joesharratt1229
43c739cb3e
Feat/curr adj ( #394 )
2025-04-02 06:39:14 +01:00
Roman Machacek
2c52f33c3a
CodeIO HQ Dataset ( #382 )
...
* ADD: CodeIO high quality dataset
Based on the dataset for CodeI/O. Annotated using Qwen-Coder and filtered based on the various metrics resulting in high quality filtered dataset, where approx 50% of the original data is kept.
* ADD: Compressed version
* Delete pure json version
2025-04-01 22:34:33 +02:00
Oliver Stanley
415bcb5ace
use constant for dataset name in arc agi ( #404 )
2025-04-01 22:06:31 +02:00
Zafir Stojanovski
ec485cce24
Add banner in README.md ( #399 )
2025-04-01 21:29:41 +02:00
Adefioye
e3af2dd2bd
Add curriculum for arc_agi ( #372 )
...
* Add curriculum for arc_agi
* Resolve conflicts
* Remove code smell
* Remove unwanted code
2025-04-01 21:17:52 +02:00
Zafir Stojanovski
50846c3534
fix(env): ARC 1D curriculum ( #402 )
...
* Add arc_1d curriculum
* Add difficulty to metadata
* use range attribute instead of scalar
---------
Co-authored-by: abdulhakeem <abdulhakeemadefioye@gmail.com>
Co-authored-by: Oliver Stanley <olivergestanley@gmail.com>
2025-04-01 13:01:15 +02:00
vncntt
cd85c2d632
add knights knaves curriculum ( #401 )
...
* add knights knaves curriculum
* add metadata + width constraints
2025-04-01 12:20:58 +02:00
Oliver Stanley
ea10a0f932
update task count in readme ( #400 )
...
* update task count in readme
* fix link
2025-04-01 10:51:36 +02:00
Zafir Stojanovski
8c45571a48
visualize heatmap sorted by overall performance ( #397 )
2025-04-01 00:08:39 +02:00
Zafir Stojanovski
c6663cdb81
fix(training): Prepend <think> token in format reward ( #396 )
...
* prepend think token in format reward
* pre commit + fix some default vals
* add checkpoint config
2025-03-28 09:45:17 +01:00
Tiago Serafim
7ae2942c34
Fix small bit of old code ( #386 )
...
Removed the `sat_utils.` that was left there from the original zebra package.
This allows the zebra generator to be used with the full set of available clues.
2025-03-24 09:00:28 +00:00
Zafir Stojanovski
2a970390e8
remove coach and update tests ( #393 )
2025-03-24 00:10:01 +01:00
Oliver Stanley
eb69916c1b
initial verl training codebase ( #389 )
...
* fixes for latest verl
* composite dataset training experiment
* use stateful dataloaders to match verl changes
* training readme
* add formatting reward
* length reward impl
* standalone reasoning_gym config section
* curriculum learning, new length reward, more config
2025-03-20 15:04:57 +00:00
Zafir Stojanovski
ce0a6c4878
fix(envs): Add source dataset and index to metadata ( #388 )
...
* add source dataset and index to metadata
* fix typo
* fix coach class and its test
2025-03-20 11:12:14 +00:00
Oliver Stanley
7475a20700
include ranges rather than sampled values in difficulty metadata dicts ( #387 )
...
* update difficulty metadata for logic datasets
* update difficulty metadata for graph datasets
* update difficulty metadata for geometry datasets
* update difficulty metadata for games datasets
* update difficulty metadata for cognition datasets
* update difficulty metadata for arithmetic datasets
* update difficulty metadata for arc datasets
* update difficulty metadata for algorithmic datasets
* update difficulty metadata for algebra datasets
* use tuples
* update tests
* update tests
2025-03-20 10:27:03 +01:00
Rich Jones
b69c35818a
fix figlet font curr imports ( #383 )
2025-03-18 23:38:17 +01:00
Zafir Stojanovski
814da6e08a
add difficulty metadata
2025-03-18 13:51:43 +01:00
Zafir Stojanovski
29bf78293f
figlet font curriculum
2025-03-18 13:51:43 +01:00
joesharratt1229
9234aa77bf
Feat/open instruct example ( #381 )
...
* added open-instruct
* fixed hooks
* GRPO
---------
Co-authored-by: Andreas Koepf <andreas.koepf@provisio.com>
2025-03-17 23:20:11 +01:00
Andreas Koepf
1511c5e301
don't pass answer value to eval
2025-03-17 23:13:53 +01:00
Jean Kaddour
d6aad5a329
fix: add score_answer() to number_sorting ( #380 )
...
* fix: add score_answer() to number_sorting
* chore: run pre-commit
* fix: use json.loads()
* fix: run isort()
2025-03-17 23:04:13 +01:00
Oliver Stanley
1c6f2d01ee
use StatefulDataLoader in veRL examples ( #378 )
2025-03-17 07:28:10 +01:00
Andreas Koepf
c760da6b1b
adapt needle_haystack test to curriculum changes
2025-03-16 17:35:16 +01:00
Andreas Koepf
f25820a0c7
bump version
2025-03-16 15:54:03 +01:00
Andreas Koepf
08b422f673
reduce values in NeedleHaystackCurriculum
2025-03-16 15:52:48 +01:00
Andreas Köpf
d2c895f1d3
Refactor Curriculum Attributes ( #335 )
...
* remove min_value from AttributeDefinition
* remove type from AttributeDefinition
* Add CurriculumContext
* add ensure_interval option for RangeAttributes
* docs: Add legend explaining curriculum indicators in dataset gallery
* update GALLERY.md
2025-03-16 15:40:28 +01:00
Andreas Köpf
4e7d9296ee
feat: Add comprehensive visualization script for evaluation results ( #376 )
2025-03-16 12:26:27 +01:00
Andreas Köpf
d6f399b8e4
Add eval configs, small fixes to eval script & rush-hour score_answer
2025-03-16 09:18:05 +01:00
Andreas Koepf
27cdd25548
add gemma-3-27b & qwq-32b configs
2025-03-15 20:47:51 +01:00
Andreas Koepf (aider)
f373881c06
feat: Add fallback to first non-None model answer when best_answer is unset
2025-03-15 16:52:50 +01:00
Andreas Köpf
bfa5f8078b
Eval N completions per prompt ( #374 )
...
* feat: Add support for generating multiple completions per prompt
* feat: Track best and mean scores for multiple completions per prompt
* feat: Add checkpoint and resume functionality to evaluation script
2025-03-15 16:39:36 +01:00
Oliver Stanley
bd13b1b92a
Fix chain sum veRL example for latest veRL ( #371 )
...
* fixes for latest verl
* add balance_batch cofg
* 1 -> 2 gpu
* tweaks
* also add raw ids to server script
2025-03-14 20:15:54 +01:00
Adefioye
8a0cacc054
Add jugs curriculum ( #369 )
2025-03-14 18:04:33 +01:00
Andreas Koepf
8c12fe86e2
bump version
2025-03-14 16:21:34 +01:00
Andreas Koepf
79ba55aae3
update gallery
2025-03-14 16:18:43 +01:00
Zafir Stojanovski
f72ecaf411
circuit logic curriculum ( #368 )
2025-03-14 16:17:00 +01:00
Rich Jones
2efe577238
Add curricula status to gallery ( #367 )
...
* Add curricula status to gallery
* lint
2025-03-14 16:16:26 +01:00
Rich Jones
e7d05d6510
GoL-Halt Curricula ( #366 )
...
* GoL-Halt Curricula
* trivial
2025-03-14 16:15:45 +01:00
Zafir Stojanovski
d33c667c3d
feat(env): Propositional Logic Curriculum ( #365 )
...
* propositional logic curriculum
* lint
* difficulty meta
2025-03-14 16:12:39 +01:00
Zafir Stojanovski
41f3ef876c
time intervals curriculum ( #363 )
2025-03-14 16:11:55 +01:00
Oliver Stanley
a71994ad03
add rush hour curriculum ( #362 )
...
* add rush hour curriculum
* add to __init__.py
2025-03-14 16:11:23 +01:00
Oliver Stanley
b5651e5e2c
add word ladder curriculum ( #361 )
...
* add word ladder curriculum
* add to __init__.py
2025-03-14 16:10:52 +01:00
Oliver Stanley
8f8bd9d756
add sudoku curriculum ( #360 )
...
* sudoku curriculum
* add test
* add to __init__.py
2025-03-14 16:09:49 +01:00
joesharratt1229
80ac138455
added family relatinship curriculum ( #359 )
2025-03-14 16:08:49 +01:00
joesharratt1229
fa7d8e66b3
added boxnet generator and curr ( #364 )
2025-03-14 08:48:57 +01:00
Adefioye
5bfe730c3b
Simple equations curriculum ( #340 )
...
* Add curriculum to simple_equations and unit tests
* Add difficulty to metadata
2025-03-13 21:17:14 +01:00
joesharratt1229
6354ca5d35
added rearc curr ( #358 )
2025-03-13 21:14:03 +01:00
Zafir Stojanovski
099ea88402
feat(env): Rubiks Cube Curriculum ( #357 )
2025-03-13 21:12:32 +01:00
joesharratt1229
d603d8b72b
added aiw curric ( #356 )
...
added metadata
2025-03-13 21:10:52 +01:00
Zafir Stojanovski
93793f5416
feat(env): Needle in Haystack Curriculum ( #355 )
...
* needle haystack curriculum
2025-03-13 21:09:45 +01:00
