Commit graph

709 commits

Author SHA1 Message Date
Andreas Koepf
0a660a3409 ignore single whitespace at beginning and end of answer, use reward = len(oracle_answer) / len(answer) 2025-02-14 15:40:12 +01:00
Andreas Koepf
979b6ba4ef lazy import of wordle_words 2025-02-14 12:06:44 +01:00
Andreas Koepf
5d16a2193c import string_insertion, minor whitespace fixes, gallery update 2025-02-14 07:55:38 +01:00
Andreas Koepf (aider)
d60d608192 feat: Add comprehensive unit tests for Game of Life dataset 2025-02-14 07:28:41 +01:00
Andreas Koepf
8960d66beb move game_of_life from games to algorithmic 2025-02-14 07:26:47 +01:00
Andreas Köpf
312291f2a7
Merge pull request #126 from zafstojano/env/string-synthesis
String Synthesis
2025-02-14 07:11:25 +01:00
Andreas Köpf
31f2dae3cf
Merge pull request #120 from open-thought/rich/graphcolor
Add Graph Coloring Problems
2025-02-14 07:09:44 +01:00
Andreas Köpf
b64d0af2bc
Merge branch 'main' into rich/graphcolor 2025-02-14 07:09:38 +01:00
Andreas Köpf
1ccc95eb61
Merge pull request #124 from Adefioye/koko/eval
Add useful instructions to question template of some datasets
2025-02-14 07:08:06 +01:00
abdulhakeem
ad318c5b8e Remove pre-commit config in gitignore 2025-02-13 23:15:10 -06:00
abdulhakeem
c762bae9f0 Remove pre-commit config 2025-02-13 22:52:16 -06:00
abdulhakeem
94d4bc03fc Remove play file and format with pre-commit 2025-02-13 22:39:21 -06:00
Adefioye
ecb0ee2315
Merge branch 'main' into koko/eval 2025-02-13 22:27:16 -06:00
abdulhakeem
58e03067f9 Fix formatting of added instructions 2025-02-13 21:26:58 -06:00
abdulhakeem
fc5fb85537 Fix more conflict 2025-02-13 21:26:54 -06:00
abdulhakeem
b7a721fce0 Fix more conflict 2025-02-13 21:24:05 -06:00
abdulhakeem
3493703c33 Fix conflict during rebasing 2025-02-13 21:21:11 -06:00
Andreas Koepf (aider)
6ec029c030 test: Add test to verify perfect score for PolynomialEquationsDataset 2025-02-13 21:15:17 -06:00
joesharratt1229
90a923c78e updated async impl and added r1 2025-02-13 21:15:17 -06:00
Zafir Stojanovski
6f9036631a lint 2025-02-13 21:15:17 -06:00
Zafir Stojanovski
12af794381 string insertion 2025-02-13 21:15:17 -06:00
Andreas Koepf
9ae8a29ff2 use json formatting for initial state of game-of-life board 2025-02-13 19:06:20 +01:00
Andreas Koepf
5410bb78a0 add ProductsDataset (multiplication tasks) 2025-02-13 17:59:02 +01:00
Andreas Koepf (aider)
ce30536627 test: Add scoring tests for Products dataset 2025-02-13 17:52:32 +01:00
Andreas Koepf (aider)
bdcaeff42a feat: Add ProductsDataset with configurable terms and digits 2025-02-13 17:50:19 +01:00
Andreas Koepf
3ead141db5 feat: Add PowerFunctionConfig and PowerFunctionDataset to arithmetic module exports 2025-02-13 17:50:17 +01:00
Andreas Köpf
de8101aef1
Merge pull request #125 from open-thought/rich/gol2
Game of Life Output Formatting
2025-02-13 17:42:22 +01:00
Zafir Stojanovski
3a2edf71c3 string synthesis 2025-02-13 16:33:28 +01:00
Rich Jones
48e19795ad
Merge branch 'main' into rich/graphcolor 2025-02-13 13:43:34 +01:00
Rich Jones
deba814211 Merge branch 'main' into rich/graphcolor 2025-02-13 13:42:54 +01:00
Rich Jones
c4c546fc82 address comments abt formatting 2025-02-13 13:42:34 +01:00
Rich Jones
0adde9853f json gol 2025-02-13 13:25:38 +01:00
Andreas Köpf
304489fed4
Merge pull request #118 from zafstojano/env/string-insertion
String Insertion
2025-02-13 13:07:36 +01:00
Andreas Köpf
971782308f
Merge branch 'main' into env/string-insertion 2025-02-13 13:07:29 +01:00
Andreas Köpf
9dc031e704
Merge pull request #119 from zafstojano/env/string-modification
String Manipulation
2025-02-13 13:03:42 +01:00
Andreas Koepf
66df77aca6 format answer in polynomial_equations to match expected format, add unit test 2025-02-13 12:09:20 +01:00
Andreas Koepf (aider)
997f4fc829 test: Add test to verify perfect score for PolynomialEquationsDataset 2025-02-13 11:48:16 +01:00
Andreas Köpf
e14e61824f
Merge pull request #123 from joesharratt1229/feat/r1-evals
Added r1 async implementation and algorithmic config
2025-02-13 11:37:07 +01:00
abdulhakeem
8811598a17 Add useful instructions to question template of some datasets 2025-02-13 00:49:31 -06:00
joesharratt1229
b2e3ccf3d6 updated async impl and added r1 2025-02-13 03:51:01 +00:00
Rich Jones
3ec4c15c51 add graph coloring 2025-02-13 01:28:09 +01:00
Zafir Stojanovski
7a12f45d53 string manipulation 2025-02-12 22:28:23 +01:00
Zafir Stojanovski
50f5b50845 lint 2025-02-12 17:26:23 +01:00
Andreas Koepf
ab9f781d97 use *args param for _define_attributes() 2025-02-12 16:59:09 +01:00
Zafir Stojanovski
a79d3d06f2 string insertion 2025-02-12 15:18:51 +01:00
Andreas Köpf
c9b39fdab1
Merge pull request #116 from zafstojano/env/pool_matrix
(Max/Average) Pool Matrix
2025-02-12 14:07:38 +01:00
Andreas Köpf
c73d241ae1
Merge branch 'main' into env/pool_matrix 2025-02-12 14:07:25 +01:00
Andreas Köpf
8e5caf6efb
Merge pull request #117 from zafstojano/fix/eval-system-prompt
fix(eval): System prompt and parse response
2025-02-12 14:05:34 +01:00
Zafir Stojanovski
58a641e59f lint 2025-02-12 11:21:46 +01:00
Zafir Stojanovski
3be5252d92 min matrix size, fixed question template 2025-02-12 11:20:55 +01:00