Oliver Stanley
5726034a26
fix color_cubes answer strings, update gallery with latest envs ( #464 )
...
* update gallery with latest envs
* fix regression where answer str is wrong in color_cubes
* re-update gallery
2025-06-08 13:16:54 +02:00
Oliver Stanley
1a727ecf4e
support python 3.10 ( #450 )
...
* support python 3.10
* add 3.10 to tests
* new StrEnum
2025-06-04 10:34:01 +01:00
Zafir Stojanovski
6d9e14729a
fix rubiks cube prompt ( #433 )
2025-05-15 21:31:21 +01:00
joesharratt1229
d0ef136d5b
Feat/intragen experiments ( #414 )
...
* added curriculum
* readapted readme
* corrected small errors
* Delete eval/eval/r1/algorithmic/word_sorting.json
* removed redundant argument
* added spell
* removed duplicated fit
* changed config
* added composite changes
* added composite changes
* updated yaml
* added spell backward
* updated read me
* added qwen2.5
* added
* Add files via upload
* updated missing trainer func
* updated curr
* updated spell back
* updated correctness score func
* updated configs
* added local evals
* added updates
* updated datasets
* added fsdp to hf utility
* added algorithmic qwen 3b yaml
* updated read me
* updated configs
* added preappend token
* updated with thinking token
* updated test score board
* resolved comments
* added evaluation scripts
* removed results from pr
* added config
* added partial reward scoring
* added evaluation composites
* added training configs
* added games eval
* added rubriks cube
* resolved merge cinflicts
* added games config
* added latest eval configs
* updated strucutre
* Delete training/evaluations/eval_graphs_composite.yaml
---------
Co-authored-by: joesharratt1229 <joesharrat1229@gmail.com>
2025-04-16 08:04:52 +02:00
Zafir Stojanovski
290bfc4fdd
(evals): Medium configs ( #415 )
...
* updated medium configs
* fix problematic curriculum values / small issues causing exceptions to be raised
* optimus alpha config
* all configs so far
* fix tests
2025-04-14 08:25:31 +02:00
Zafir Stojanovski
dced3bfc45
fix(curriculum): Make boundaries in curriculum more sensible ( #407 )
...
* init
* fix tests
* unify codeio
* filtered for libraries not present in reasoning-gym
* fix more bounds
* puzzle24
* knight swap curriculum
* fix number sorting
* fix attributes
* add validation of config in creation of dataset
* dry run for instantiating and validating the datasets
* remove unused imports
* fix curriculum tests to reference newly updated attribute names
2025-04-04 20:24:14 +02:00
Zafir Stojanovski
ce0a6c4878
fix(envs): Add source dataset and index to metadata ( #388 )
...
* add source dataset and index to metadata
* fix typo
* fix coach class and its test
2025-03-20 11:12:14 +00:00
Oliver Stanley
7475a20700
include ranges rather than sampled values in difficulty metadata dicts ( #387 )
...
* update difficulty metadata for logic datasets
* update difficulty metadata for graph datasets
* update difficulty metadata for geometry datasets
* update difficulty metadata for games datasets
* update difficulty metadata for cognition datasets
* update difficulty metadata for arithmetic datasets
* update difficulty metadata for arc datasets
* update difficulty metadata for algorithmic datasets
* update difficulty metadata for algebra datasets
* use tuples
* update tests
* update tests
2025-03-20 10:27:03 +01:00
Rich Jones
b69c35818a
fix figlet font curr imports ( #383 )
2025-03-18 23:38:17 +01:00
Zafir Stojanovski
814da6e08a
add difficulty metadata
2025-03-18 13:51:43 +01:00
Zafir Stojanovski
29bf78293f
figlet font curriculum
2025-03-18 13:51:43 +01:00
Andreas Koepf
08b422f673
reduce values in NeedleHaystackCurriculum
2025-03-16 15:52:48 +01:00
Andreas Köpf
d2c895f1d3
Refactor Curriculum Attributes ( #335 )
...
* remove min_value from AttributeDefinition
* remove type from AttributeDefinition
* Add CurriculumContext
* add ensure_interval option for RangeAttributes
* docs: Add legend explaining curriculum indicators in dataset gallery
* update GALLERY.md
2025-03-16 15:40:28 +01:00
Zafir Stojanovski
099ea88402
feat(env): Rubiks Cube Curriculum ( #357 )
2025-03-13 21:12:32 +01:00
Zafir Stojanovski
93793f5416
feat(env): Needle in Haystack Curriculum ( #355 )
...
* needle haystack curriculum
2025-03-13 21:09:45 +01:00
Zafir Stojanovski
9fcc277101
color cube rotation curriculum ( #347 )
2025-03-13 21:04:34 +01:00
joesharratt1229
0dce7adbad
Curriculum/cognition ( #314 )
...
* added rectangle count curriculum
* added number sequences
* registered curriculum
2025-03-11 00:10:28 +01:00
Rich Jones
e62b45d61c
BF Curricula and More ( #309 )
...
* bf curricula
* modulo grid curricula
* minor changes to how difficulty is stored
---------
Co-authored-by: Andreas Koepf <andreas.koepf@provisio.com>
2025-03-09 18:22:22 +01:00
Rich Jones
cbfdf097a0
Add Modulo Grid Task ( #273 )
...
* add modulo_grid dataset
* ensure the pattern is mathematical, not just spatial
---------
Co-authored-by: Andreas Koepf <andreas.koepf@provisio.com>
2025-03-07 11:11:41 +01:00
Andreas Köpf
5d7fbac0ad
Minor question template & score_answer improvements ( #261 )
...
* math prompt improvements
* ignore brackets in complex_arithmetic results
* improve additional instruction in prompt of polynomial_equations
* more strict tests for score_answer in polynomial_equations
* simplify special reward handling
* fix test_intermediate_integration
* fix sokoban dataset
* add common dataset score_answer consistency test
2025-03-04 21:55:09 +01:00
Zafir Stojanovski
01e1c8f9af
fix: Unify Prompts ( #254 )
...
* remove cot
* fix prompt template
* fix pool matrix
* spiral matrix fixed
2025-03-03 21:55:53 +01:00
Andreas Köpf
c98cc5fcd6
Merge pull request #220 from open-thought/rich/cubeinstructions
...
Make Rubiks Cube Output Format More Explicit
2025-02-27 12:16:09 +01:00
Rich Jones
253e49aecf
sm fixes
2025-02-27 11:54:04 +01:00
Rich Jones
633a1aa1ba
expand more
2025-02-27 10:41:30 +01:00
Rich Jones
f0ca949aaf
support expanded notation anyway
2025-02-26 13:17:03 +01:00
Rich Jones
285e2b20cc
rubiks cube instructions
2025-02-26 13:07:17 +01:00
Rich Jones
229086131a
fix CCC scoring
2025-02-26 12:54:40 +01:00
Andreas Koepf
eeb9fa31d5
more native type hints
2025-02-21 21:23:14 +01:00
Andreas Köpf
a6a5d30f1c
Merge pull request #175 from AhmedSaif2/fix-format
...
Add score_answer function to handle comma-formatted numbers
2025-02-21 15:36:21 +01:00
Andreas Koepf
3e7ff3b084
use native types List->list, Dict->dict, Set->set, Tuple->tuple
2025-02-21 15:15:38 +01:00
Andreas Köpf
994ffa8459
Merge pull request #170 from open-thought/rich/needle
...
Adds Needle in a Haystack problems
2025-02-20 22:12:47 +01:00
Rich Jones
2188c53308
wiggle imports
2025-02-20 16:23:40 +01:00
Rich Jones
39b739917e
move assert to config
2025-02-20 12:45:09 +01:00
Rich Jones
621c20d8d8
adds Needle in a Haystack problems
2025-02-20 12:28:30 +01:00
Rich Jones
a9bbdd292a
rc gallery format
2025-02-20 11:26:05 +01:00
joesharratt1229
52cfca5f73
added colour cube score answer impl
2025-02-19 18:12:31 +00:00
Zafir Stojanovski
c344a4de35
fix prompt
2025-02-17 16:12:50 +01:00
Andreas Koepf
979b6ba4ef
lazy import of wordle_words
2025-02-14 12:06:44 +01:00
Rich Jones
c2fb8bb6cc
add rectangle count dataset
2025-02-11 13:56:27 +01:00
Andreas Koepf
60effc6e7a
move arc_1d into from cognition into arc folder
2025-02-08 19:37:26 +01:00
Andreas Köpf
3f6b2fc807
Add Coaching & ScoreBoard class (result tracking) ( #72 )
...
* feat: Add Coach and ScoreBoard classes for performance tracking and difficulty adjustment
* feat: Add GroupedScores class to wrap aggregated scores
* refactor: Create ScoreStats class with tuple-based score statistics
* feat: Add unit test for Coach with CompositeDataset and multiple datasets
* fix: Add difficulty metadata to leg counting dataset
* feat: Add clear() method to ScoreBoard to reset all stored data
* feat: Add __len__ method to ScoreBoard to return number of scores
* feat: Add update_dataset_config method to CompositeDataset
* cleanup __init__ & imports
2025-02-06 23:15:28 +01:00
Andreas Koepf
3aeec71523
add attribution for arc-1d and unit tests
2025-02-02 23:45:25 +01:00
Andreas Koepf (aider)
978a0879f7
feat: Add mirrored and inverse task variations to ARC_1D_TASKS
2025-02-02 23:21:46 +01:00
Andreas Koepf
9a1270dd95
add arc_1d dataset
2025-02-02 23:03:56 +01:00
Andreas Koepf (aider)
a060348a9c
fix: Resolve undefined task function references in arc_1d.py
2025-02-02 22:49:28 +01:00
Andreas Koepf (aider)
b599d6e1a2
feat: Add Arc1D dataset with comprehensive task generation and configuration
2025-02-02 22:49:00 +01:00
Andreas Koepf (aider)
84e4f1c5bc
feat: Add task augmentation functions mirror, inverse, and identity to arc_1d.py
2025-02-02 22:42:21 +01:00
Andreas Koepf
5dd4c0e831
change parameter order for basic arc tasks
2025-02-02 17:25:37 +01:00
Andreas Koepf (aider)
da16467ca7
feat: Add five new 1D ARC task generation functions
2025-02-02 16:38:14 +01:00
Andreas Koepf (aider)
3714e6c5ff
feat: Add five new 1D ARC task generation functions
2025-02-02 16:37:14 +01:00
