open-thought/reasoning-gym
1
0
Fork 0
mirror of https://github.com/open-thought/reasoning-gym.git synced 2026-04-19 12:58:07 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Eval sampling settings for generation (temperature, top-p, max_tokens) (#242)

Browse source 
* feat: Add sampling parameters to eval configuration and API call
* feat: Add support for system_prompt_id and optional system_prompt configuration
This commit is contained in:
Andreas Köpf 2025-02-28 11:48:37 +01:00 • committed by GitHub
parent b1c8840129
commit b4207162ff
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 83 additions and 22 deletions
Download patch file Download diff file Expand all files Collapse all files

5
eval/README.md
View file

@ -44,6 +44,11 @@ output_dir: "results"
max_concurrent: 10
default_size: 20 # Default size for all datasets
default_seed: 42 # Default seed for all datasets
max_tokens: 32768 # Maximum generation length (optional)
temperature: 0.6 # Generation temperature (optional)
top_p: 0.95 # Top-p sampling parameter (optional)
system_prompt_id: "default" # Use a predefined system prompt by ID (optional)
# system_prompt: "Your custom system prompt here" # Or specify a custom system prompt directly
categories:
- category: "algebra"