teknium
4738fabd57
convert fundamentals prediction env to use managed server
2025-11-14 09:48:56 +00:00
teknium
ff46cfff44
convert letter_counting_environment to use managed server
2025-11-14 09:44:20 +00:00
pre-commit-ci[bot]
aae4432a58
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-11-14 06:55:56 +00:00
teknium
76fec8b919
convert rlaif_server to managedserver
2025-11-14 06:53:16 +00:00
teknium
d8c68a93e3
convert tool_calling_server to managedserver
2025-11-14 06:48:07 +00:00
pre-commit-ci[bot]
0a3c15c7ad
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-11-14 06:14:21 +00:00
teknium
be74c759e5
convert swe_rl to managedserver
2025-11-14 06:13:02 +00:00
pre-commit-ci[bot]
9d3dbd1a73
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-11-14 00:10:43 +00:00
teknium
e28297b625
support managedserver in mcqa thinking
2025-11-14 00:10:04 +00:00
teknium
f0fee7fba6
revert zip change
2025-11-14 00:03:06 +00:00
pre-commit-ci[bot]
d5e6793f02
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-11-13 22:41:42 +00:00
teknium
28468bcae5
fix sampling temps
2025-11-13 22:41:04 +00:00
teknium
73e8ee2475
make evals also use managed
2025-11-13 22:39:21 +00:00
teknium
1ccf3b54e3
remove unused import
2025-11-13 08:40:30 +00:00
Teknium
77fab3e895
Merge pull request #279 from NousResearch/fix-things
...
fix some issues
2025-11-13 00:34:20 -08:00
teknium
db1d094386
fix some issues
2025-11-13 08:33:03 +00:00
pre-commit-ci[bot]
b03b8d3808
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-11-13 08:04:40 +00:00
teknium
3f6265563f
convert gsm8k server
2025-11-13 08:03:00 +00:00
hjc-puro
a341252a21
Merge pull request #274 from dhyaneesh/dump-evaluate-config-yaml
...
feat: dump evaluate subcommand config to YAML in env save dir
2025-11-10 20:01:22 -05:00
Dhyaneesh DS
9000c10869
Merge branch 'main' into dump-evaluate-config-yaml
2025-11-11 00:47:30 +05:30
dmahan93
e2c9ff5e9c
Merge pull request #272 from ninastef/main
...
refactor: Refactor scored data handling into reusable helper
2025-11-10 09:37:29 -08:00
dmahan93
2779aacc54
Merge pull request #275 from NousResearch/pre-commit-ci-update-config
...
[pre-commit.ci] pre-commit autoupdate
2025-11-10 09:22:21 -08:00
pre-commit-ci[bot]
705a0f743b
[pre-commit.ci] pre-commit autoupdate
...
updates:
- [github.com/psf/black-pre-commit-mirror: 25.9.0 → 25.11.0](https://github.com/psf/black-pre-commit-mirror/compare/25.9.0...25.11.0 )
- [github.com/astral-sh/ruff-pre-commit: v0.14.3 → v0.14.4](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.3...v0.14.4 )
2025-11-10 16:43:57 +00:00
Dhyaneesh
39d5fb4452
feat: dump evaluate subcommand config to YAML in env save dir
...
Automatically save the final merged evaluate configuration to evaluate_config.yaml
in the data_dir_to_save_evals directory. This includes env config, OpenAI/server
configs, and server manager settings, enabling reproducibility and easier
debugging of evaluation runs.
The config is saved after all merging (CLI args > YAML > defaults) to capture
the exact configuration used for the evaluation.
2025-11-08 23:46:13 +05:30
dmahan93
b4080a4f37
Merge pull request #273 from NousResearch/add-vllm-manager-fn
...
add managed vllm server
2025-11-07 14:22:07 -08:00
Dakota
e6ac3abdcb
add managed vllm server
2025-11-07 13:06:49 -06:00
pre-commit-ci[bot]
9bef7a1b46
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-11-07 18:10:40 +00:00
Nina
722385e715
Update server.py
2025-11-07 19:06:03 +01:00
Nina
b3feae5eef
Update server.py
2025-11-07 19:03:54 +01:00
Nina
74b5412c2b
Update server.py
2025-11-07 19:02:26 +01:00
Nina
16a40a5617
Update server.py
2025-11-07 19:01:48 +01:00
Nina
97107ca868
Update server.py
2025-11-07 19:01:09 +01:00
Nina
a5a8b07848
Update server.py
2025-11-07 19:00:32 +01:00
dmahan93
c96b8a1255
Merge pull request #267 from bobtajson/main
...
fix: correct typo and improve code quality
2025-11-06 18:36:37 -08:00
dmahan93
fe6be37ba2
Merge pull request #269 from NousResearch/pre-commit-ci-update-config
...
[pre-commit.ci] pre-commit autoupdate
2025-11-03 15:04:14 -08:00
pre-commit-ci[bot]
a896383c1a
[pre-commit.ci] pre-commit autoupdate
...
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.14.2 → v0.14.3](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.2...v0.14.3 )
2025-11-03 16:43:01 +00:00
dmahan93
b1e164eef5
Merge pull request #264 from NousResearch/add-logprob-server-manager-fn
...
add sglang specific token level logprob handling and server manager/b…
2025-10-29 13:53:39 -07:00
Dakota
578175a709
fix pre-commit
2025-10-29 14:47:50 -05:00
Dakota
3c8fc32288
fix test case
2025-10-29 14:38:16 -05:00
Dakota
5d6d6bb0dc
add docs :)
2025-10-29 11:26:43 -05:00
Dakota
c3a118f50d
fix tests
2025-10-29 10:55:10 -05:00
Dakota
d5400460e8
made masked logprobs coherently decided on
2025-10-29 10:52:38 -05:00
Dakota
e57c396f86
ran pre-commit
2025-10-29 10:45:27 -05:00
Dakota
17bb7bdf15
revert base.py
2025-10-29 10:11:05 -05:00
dmahan93
c044f1d82c
Merge pull request #268 from NousResearch/pre-commit-ci-update-config
...
[pre-commit.ci] pre-commit autoupdate
2025-10-29 07:40:50 -07:00
pre-commit-ci[bot]
4173adb7b6
[pre-commit.ci] pre-commit autoupdate
...
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.14.1 → v0.14.2](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.1...v0.14.2 )
2025-10-27 16:41:06 +00:00
dmahan93
c483840f59
set prompt logprobs to a masked value
2025-10-26 11:58:55 -07:00
dmahan93
c22f8ca81b
Merge remote-tracking branch 'origin/add-logprob-server-manager-fn' into add-logprob-server-manager-fn
2025-10-24 23:18:37 -07:00
dmahan93
5d662bf1aa
add chat example and fix bug in managed_server
2025-10-24 23:15:56 -07:00
pre-commit-ci[bot]
0d80da5146
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-10-24 20:10:29 +00:00
