24 commits

Author	SHA1	Message	Date
Zafir Stojanovski	98e976642d	gemini flash and o3 mini configs (#425)	2025-04-20 21:24:52 +02:00
Adefioye	169d8c3aec	Evals: Sonnet-3.7-eval partial results (#423) ... * Add results for sonnet and config * Make some cleanup	2025-04-18 10:31:40 +02:00
Zafir Stojanovski	290bfc4fdd	(evals): Medium configs (#415) ... * updated medium configs * fix problematic curriculum values / small issues causing exceptions to be raised * optimus alpha config * all configs so far * fix tests	2025-04-14 08:25:31 +02:00
Andreas Köpf	cd1a9ea58b	add llama-4-maverick eval yaml config (#413)	2025-04-06 21:10:00 +01:00
Zafir Stojanovski	dced3bfc45	fix(curriculum): Make boundaries in curriculum more sensible (#407) ... * init * fix tests * unify codeio * filtered for libraries not present in reasoning-gym * fix more bounds * puzzle24 * knight swap curriculum * fix number sorting * fix attributes * add validation of config in creation of dataset * dry run for instantiating and validating the datasets * remove unused imports * fix curriculum tests to reference newly updated attribute names	2025-04-04 20:24:14 +02:00
Andreas Köpf	4e7d9296ee	feat: Add comprehensive visualization script for evaluation results (#376)	2025-03-16 12:26:27 +01:00
Andreas Köpf	d6f399b8e4	Add eval configs, small fixes to eval script & rush-hour score_answer	2025-03-16 09:18:05 +01:00
Andreas Koepf	27cdd25548	add gemma-3-27b & qwq-32b configs	2025-03-15 20:47:51 +01:00
Andreas Köpf	bfa5f8078b	Eval N completions per prompt (#374) ... * feat: Add support for generating multiple completions per prompt * feat: Track best and mean scores for multiple completions per prompt * feat: Add checkpoint and resume functionality to evaluation script	2025-03-15 16:39:36 +01:00
joesharratt1229	105374183f	Algebra/curr (#320) ... * add polynomial equation curriculum * added simple integration * addded metadata to config	2025-03-11 00:17:07 +01:00
Andreas Koepf	4109b5b72c	update eval yaml config files	2025-03-10 00:48:32 +01:00
Andreas Köpf	5b8d1b5175	Generate eval config tool (#240) ... * feat: Add generate_config.py script to create eval configurations	2025-02-27 21:40:53 +01:00
Andreas Köpf	850c1cf6f4	Eval script consolidation (#238) ... The script now supports: - YAML and JSON configurations - Dataset-specific parameters - Overriding configuration via command line - Detailed logging and error handling	2025-02-27 17:39:14 +01:00
Andreas Koepf	acb2d7eb53	add llama-3.3-70b-instruct eval yaml files	2025-02-26 20:54:07 +01:00
Andreas Köpf	5b89a3a2d0	Merge pull request #217 from open-thought/feat/o3-mini-eun ... added o3 mini yaml rconfiguration	2025-02-26 09:38:11 +01:00
vncntt	29179f783e	fix sonnet eval_dir (#216) ... * fix eval_dir * add logging	2025-02-26 09:37:09 +01:00
joesharratt1229	7d7e44d1af	added o3 mini yaml	2025-02-26 08:09:12 +00:00
Andreas Koepf	6d5168d1e5	add llama-3.3-70b-instruct algebra, algorithmic eval configs	2025-02-25 23:43:29 +01:00
Andreas Koepf	791f16ec0f	use results folder name for eval results	2025-02-25 19:41:21 +01:00
joesharratt1229	56cc111ab3	Merge remote-tracking branch 'origin/consolidate_eval_script' into fix/eval	2025-02-25 18:10:07 +00:00
joesharratt1229	9ac6ea4eb2	changed structure	2025-02-25 16:32:42 +00:00
joesharratt1229	046c46c0bb	updated read me	2025-02-25 15:46:43 +00:00
Andreas Koepf	878f9bbc76	move r1 configs into r1 yaml/r1 subfolder	2025-02-25 16:24:30 +01:00
Andreas Koepf	e7ae82a831	consolidate eval scripts to have single eval.py	2025-02-25 16:13:22 +01:00