17 commits

Author	SHA1	Message	Date
Andreas Koepf	27cdd25548	add gemma-3-27b & qwq-32b configs	2025-03-15 20:47:51 +01:00
Andreas Köpf	bfa5f8078b	Eval N completions per prompt (#374) ... * feat: Add support for generating multiple completions per prompt * feat: Track best and mean scores for multiple completions per prompt * feat: Add checkpoint and resume functionality to evaluation script	2025-03-15 16:39:36 +01:00
joesharratt1229	105374183f	Algebra/curr (#320) ... * add polynomial equation curriculum * added simple integration * addded metadata to config	2025-03-11 00:17:07 +01:00
Andreas Koepf	4109b5b72c	update eval yaml config files	2025-03-10 00:48:32 +01:00
Andreas Köpf	5b8d1b5175	Generate eval config tool (#240) ... * feat: Add generate_config.py script to create eval configurations	2025-02-27 21:40:53 +01:00
Andreas Köpf	850c1cf6f4	Eval script consolidation (#238) ... The script now supports: - YAML and JSON configurations - Dataset-specific parameters - Overriding configuration via command line - Detailed logging and error handling	2025-02-27 17:39:14 +01:00
Andreas Koepf	acb2d7eb53	add llama-3.3-70b-instruct eval yaml files	2025-02-26 20:54:07 +01:00
Andreas Köpf	5b89a3a2d0	Merge pull request #217 from open-thought/feat/o3-mini-eun ... added o3 mini yaml rconfiguration	2025-02-26 09:38:11 +01:00
vncntt	29179f783e	fix sonnet eval_dir (#216) ... * fix eval_dir * add logging	2025-02-26 09:37:09 +01:00
joesharratt1229	7d7e44d1af	added o3 mini yaml	2025-02-26 08:09:12 +00:00
Andreas Koepf	6d5168d1e5	add llama-3.3-70b-instruct algebra, algorithmic eval configs	2025-02-25 23:43:29 +01:00
Andreas Koepf	791f16ec0f	use results folder name for eval results	2025-02-25 19:41:21 +01:00
joesharratt1229	56cc111ab3	Merge remote-tracking branch 'origin/consolidate_eval_script' into fix/eval	2025-02-25 18:10:07 +00:00
joesharratt1229	9ac6ea4eb2	changed structure	2025-02-25 16:32:42 +00:00
joesharratt1229	046c46c0bb	updated read me	2025-02-25 15:46:43 +00:00
Andreas Koepf	878f9bbc76	move r1 configs into r1 yaml/r1 subfolder	2025-02-25 16:24:30 +01:00
Andreas Koepf	e7ae82a831	consolidate eval scripts to have single eval.py	2025-02-25 16:13:22 +01:00