Teknium
62cee8ac66
Merge pull request #209 from NousResearch/add-pairwise-judge-environment
...
Add LLM as a judge environment for eval and train based on RewardBench
2025-07-16 13:37:09 -07:00
pre-commit-ci[bot]
6455c305e6
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-07-16 17:51:15 +00:00
teknium
542185bbcc
Merge branch 'add-pairwise-judge-environment' of https://github.com/NousResearch/atropos into add-pairwise-judge-environment
2025-07-16 17:48:44 +00:00
teknium
a43520e619
one last linter...
2025-07-16 17:48:43 +00:00
pre-commit-ci[bot]
eab2c938ea
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-07-16 16:58:42 +00:00
teknium
18f228615d
linter stuff
2025-07-16 16:57:51 +00:00
pre-commit-ci[bot]
ffc210e470
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-07-16 16:51:19 +00:00
teknium
2f37714e84
Merge branch 'add-pairwise-judge-environment' of https://github.com/NousResearch/atropos into add-pairwise-judge-environment
2025-07-16 16:50:04 +00:00
teknium
0113dc906b
add a bunch of extra debugging traces - configurable
2025-07-16 16:49:42 +00:00
Skylar Ray
e889324171
fix: correct quantum environment repository URL
2025-07-16 11:00:45 +03:00
pre-commit-ci[bot]
1af508b27f
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-07-16 07:46:52 +00:00
teknium
10bb22f557
adding debugging
2025-07-16 07:46:17 +00:00
pre-commit-ci[bot]
7d980372d3
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-07-15 18:40:26 +00:00
teknium
02ad3e8661
Merge branch 'add-pairwise-judge-environment' of https://github.com/NousResearch/atropos into add-pairwise-judge-environment
2025-07-15 18:39:52 +00:00
teknium
8aa540275b
add to the envs readme
2025-07-15 18:39:50 +00:00
pre-commit-ci[bot]
9f3e2ee460
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-07-15 18:24:13 +00:00
teknium
856a8455b1
please the precommit gods
2025-07-15 18:20:44 +00:00
pre-commit-ci[bot]
c053a9f134
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-07-15 11:40:22 +00:00
teknium
ce1f72059c
Merge branch 'add-pairwise-judge-environment' of https://github.com/NousResearch/atropos into add-pairwise-judge-environment
2025-07-15 11:39:46 +00:00
teknium
47c396c43f
switch to chat completions endpoint to eval closed lab stuff
2025-07-15 11:39:29 +00:00
pre-commit-ci[bot]
818ec9d7c1
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-07-15 11:18:03 +00:00
teknium
982645ce73
Implement proper ties category scoring
2025-07-15 11:16:15 +00:00
pre-commit-ci[bot]
41c847ddf4
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-07-14 09:43:20 +00:00
teknium
ef04098718
glitch
2025-07-14 09:42:44 +00:00
teknium
51d4d52765
Merge branch 'add-pairwise-judge-environment' of https://github.com/NousResearch/atropos into add-pairwise-judge-environment
2025-07-14 09:42:21 +00:00
teknium
9607880f3d
Lots of updates to the environment to cleanup, add more metrics, make more robust - ties has an issue though
2025-07-14 09:39:00 +00:00
pre-commit-ci[bot]
107809260d
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-07-12 11:23:13 +00:00
teknium
e83d796c74
add pairwise judgement environment
2025-07-12 11:15:56 +00:00
hjc-puro
75a4264f8d
Merge pull request #208 from NousResearch/2025-07-08-evals
...
Add `evaluate_log` method, gsm8k example
2025-07-12 06:45:05 +08:00
hjc-puro
6e9baaf9d8
table
2025-07-11 09:52:19 +00:00
hjc-puro
352e1b8f88
comments
2025-07-11 03:55:16 +00:00
hjc-puro
b06332623d
move time import
2025-07-11 00:45:24 +00:00
Philip Lippmann
17faebae03
Enhanced Pydantic Schema Following Environment with Dynamic Error Introduction and Editing Task Support ( #185 )
...
* New JSON env and documentation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Removed seperate JSON generation environment
* Updated pydantic environment with edit functionality
* Error helper function
* Updated README
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fixed pre-commit issues
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-07-11 10:44:16 +10:00
hjc-puro
3e1eba6e92
rename method
2025-07-11 00:39:28 +00:00
pre-commit-ci[bot]
5aacb1ea9b
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-07-11 00:26:48 +00:00
hjc-puro
e601251893
gsm8k eval example
2025-07-11 00:22:36 +00:00
hjc-puro
afe4fc318f
sample level logging
2025-07-10 02:03:07 +00:00
hjc-puro
f4de3ad6f5
add printing
2025-07-09 23:35:26 +00:00
Dakota
6386a5e185
fix precommit
2025-07-08 13:46:35 -05:00
dmahan93
0328680cbe
Merge pull request #205 from futreall/main
...
chore: spelling cleanup
2025-07-08 13:40:11 -05:00
pre-commit-ci[bot]
9340791a6c
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-07-07 21:53:41 +00:00
futreall
1f52b347ae
chore: spelling cleanup
2025-07-07 19:17:29 +03:00
dmahan93
ecc5eebeca
Merge pull request #167 from paulsengh/feat/pay-to-play-env
...
feat: add pay-to-play environment with mixture of judges and micropay…
2025-07-07 09:02:46 -05:00
dmahan93
58446dbcb1
Merge pull request #204 from NousResearch/multienv-enforce-mins
...
Multienv with enforced minimum samples in a batch
2025-07-07 08:53:43 -05:00
Dakota
08e14cc745
feat: add minimum batch allocation support for environments
...
- Add min_batch_allocation parameter to ensure environments contribute minimum proportion to each batch
- Implement grab_batch_with_minimum_allocations function with proper scaling when allocations exceed 100%
- Add mixed-size group buffering to handle variable-sized data submissions
- Update server to use minimum allocation logic when any env has min_batch_allocation set
- Add comprehensive tests for minimum allocation scenarios
- Update documentation in API README and CONFIG.md
- Update example environments to demonstrate the feature
This feature allows critical environments to guarantee they contribute at least a specified proportion (0.0-1.0) to each training batch, ensuring important data sources are always represented during training.
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 08:50:28 -05:00
emmmm
10f1b466b6
Update curriculum.py
2025-07-02 10:22:33 +02:00
Alex Pikme
ba992757e1
fix dead link README.md
2025-06-25 10:51:21 +02:00
dmahan93
5b2b5e9947
Merge pull request #192 from rnkrtt/main
...
Fix typo in author name Gurning -> Gurung in community README
2025-06-23 10:15:58 -05:00
Merkel Tranjes
af1c98d7a8
Update README.md
2025-06-23 16:23:02 +02:00
Tomass
14917440db
fix duplicate plot.py
2025-06-23 15:32:54 +02:00
