Andreas Köpf
ece6990709
Remove strip from ProceduralDataset::core score_answer() ( #250 )
...
* remove strip from ProceduralDataset::core score_answer(), strip in extract answer (optional, default=True)
* test: Move test_extract_answer() from test_dataset.py to test_utils.py
* refactor: Improve decimal reward computation with more flexible comparison
* fix: Implement rounding for format_number when round_if_needed is True
* test: Add test case for compute_decimal_reward with sign and zeros
2025-03-02 08:46:36 +01:00
Andreas Köpf
1b1c04bb70
feat: Add category property to ProceduralDataset to extract category name ( #248 )
2025-03-01 23:11:40 +01:00
Andreas Köpf
1ea9a657a7
Eval script consolidation ( #238 )
...
The script now supports:
- YAML and JSON configurations
- Dataset-specific parameters
- Overriding configuration via command line
- Detailed logging and error handling
2025-02-27 17:39:14 +01:00
Andreas Koepf
ff5b210106
use native types List->list, Dict->dict, Set->set, Tuple->tuple
2025-02-21 15:15:38 +01:00
Zafir Stojanovski
a1e2e2324c
strip answer and solution
2025-02-16 15:39:10 +01:00
Andreas Koepf
5803a2962e
more tolerant parsing of futoshiki answers
2025-02-16 14:23:40 +01:00
Andreas Koepf
2726caf2fe
ignore single whitespace at beginning and end of answer, use reward = len(oracle_answer) / len(answer)
2025-02-14 15:40:12 +01:00
Andreas Koepf
28a7f7f532
add simple dataset gallery generation script
2025-01-30 22:30:26 +01:00
Andreas Koepf (aider)
7b9f1a3b76
feat: Add seed wrapping at 2^32 to prevent unbounded growth
2025-01-30 22:05:14 +01:00
Andreas Koepf (aider)
42f6a742c9
refactor: Use self.dataset.seed directly for chunk seed generation
2025-01-30 22:02:21 +01:00
Andreas Koepf (aider)
df153d0dde
feat: Add score_answer method to ReseedingDataset
2025-01-30 21:59:50 +01:00
Andreas Koepf (aider)
4cb17ffcbd
feat: Add ReseedingDataset wrapper for infinite procedural datasets
2025-01-30 21:56:43 +01:00
Andreas Koepf
1bc56b8559
extract answer from last answer tag
2025-01-28 16:37:19 +00:00
Andreas Koepf
655de7a7f3
add first example with OpenRLHF
2025-01-28 14:40:06 +00:00
Andreas Koepf
0d2d8ba6a0
pass config to ProceduralDataset base
2025-01-25 00:23:05 +01:00
Andreas Koepf (aider)
aad0285252
feat: Add dataset factory with registration and creation functions
2025-01-25 00:00:22 +01:00
Andreas Koepf
aaabc05ace
formatting
2025-01-24 10:34:07 +01:00
Andreas Koepf (aider)
a20a9e9650
feat: Add return type annotation to ProceduralDataset.__next__()
2025-01-24 10:16:27 +01:00
Andreas Koepf (aider)
3a8a613573
feat: Add Sized and Iterable base classes to ProceduralDataset
2025-01-24 10:14:42 +01:00
Andreas Koepf (aider)
c0552097c3
feat: Add ProceduralDataset abstract base class for dataset generators
2025-01-24 09:51:04 +01:00
