Andreas Köpf
bfa5f8078b
Eval N completions per prompt ( #374 )
...
* feat: Add support for generating multiple completions per prompt
* feat: Track best and mean scores for multiple completions per prompt
* feat: Add checkpoint and resume functionality to evaluation script
2025-03-15 16:39:36 +01:00
joesharratt1229
6770ee3eef
updated for config by dataset ( #257 )
...
* updated for config by dataset
* updated read me
2025-03-03 21:58:32 +01:00
Andreas Köpf
4ad9d22fa3
Add base_url and api_key command line args for eval.py script ( #244 )
...
* feat: Add base URL command line parameter to eval.py script
* feat: Add API key parameter and CLI option to AsyncModelEvaluator
2025-02-28 18:32:58 +01:00
Andreas Köpf
b4207162ff
Eval sampling settings for generation (temperature, top-p, max_tokens) ( #242 )
...
* feat: Add sampling parameters to eval configuration and API call
* feat: Add support for system_prompt_id and optional system_prompt configuration
2025-02-28 11:48:37 +01:00
Andreas Köpf
5b8d1b5175
Generate eval config tool ( #240 )
...
* feat: Add generate_config.py script to create eval configurations
2025-02-27 21:40:53 +01:00
Andreas Köpf
850c1cf6f4
Eval script consolidation ( #238 )
...
The script now supports:
- YAML and JSON configurations
- Dataset-specific parameters
- Overriding configuration via command line
- Detailed logging and error handling
2025-02-27 17:39:14 +01:00
Andreas Koepf
791f16ec0f
use results folder name for eval results
2025-02-25 19:41:21 +01:00
joesharratt1229
ffe60ef112
finalised readme
2025-02-25 18:14:39 +00:00
joesharratt1229
9ac6ea4eb2
changed structure
2025-02-25 16:32:42 +00:00
joesharratt1229
52c3c430b9
updated config and read me
2025-02-25 16:25:16 +00:00
joesharratt1229
7b39f4a3c7
updated read me
2025-02-25 15:51:31 +00:00
joesharratt1229
046c46c0bb
updated read me
2025-02-25 15:46:43 +00:00
Andreas Koepf (aider)
e48c1f82cd
docs: Update installation instructions in eval README
2025-02-25 15:37:09 +01:00
Andreas Koepf (aider)
a1b0a0414e
docs: Add dependency installation step to eval README setup instructions
2025-02-25 15:19:38 +01:00
Andreas Koepf (aider)
205174c532
docs: Add info about reasoning-gym-eval repository for evaluation results
2025-02-25 10:53:21 +01:00
rishabhranawat
fb40c8ca55
[eval-v1] add a simple readme with some details
2025-02-10 21:57:00 -08:00
