open-thought/reasoning-gym
1
0
Fork 0
mirror of https://github.com/open-thought/reasoning-gym.git synced 2026-04-19 12:58:07 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

changed structure

Browse source
This commit is contained in:
joesharratt1229 2025-02-25 16:32:42 +00:00
parent ce8877167d
commit 68e8ea89d8
6 changed files with 45 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

5
eval/README.md
View file

@ -34,7 +34,7 @@ export OPENROUTER_API_KEY=your-api-key
```
4. Prepare your dataset configuration in YAML format (see examples in `yaml/algorithmic.yaml` or `yaml/logic.yaml`):
4. Prepare your dataset configuration in YAML format (see examples in `yaml/<model_name>/algorithmic.yaml` e.g `yaml/r1/algorithmic.yaml`):
```yaml
model: model-name
category: category-name
@ -130,8 +130,9 @@ python eval.py --yaml <path-to yaml file>
```
e.g
```
python eval.py --yaml yaml/algorithmic.yaml
python eval.py --yaml yaml/r1/algorithmic.yaml
```
To run r1 evaluations on algorithmic.yaml
The results of individual model on a dataset will be stored in a new folder in the directory E.g `r1/algorithmic/proposition_logic.json`