NousResearch/atropos
1
0
Fork 0
mirror of https://github.com/NousResearch/atropos.git synced 2026-04-19 12:57:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

[pre-commit.ci] auto fixes from pre-commit.com hooks

Browse source 
for more information, see https://pre-commit.ci
This commit is contained in:
pre-commit-ci[bot] 2026-02-20 04:58:43 +00:00
parent ccdd5a1ca6
commit 60fb6cae11
11 changed files with 221 additions and 136 deletions
Download patch file Download diff file Expand all files Collapse all files

4
README.md
View file

@ -294,10 +294,10 @@ Distillation is configured in `BaseEnvConfig` and available via CLI under `--env
Both setups are supported:
- **Self-distillation (same model family for teacher and student)**
- **Self-distillation (same model family for teacher and student)**
Point `teacher_base_url` to a server running the same model (or equivalent checkpoint family) as the student. This is the most stable setup for token-level alignment.
- **Cross-model distillation (different teacher and student models)**
- **Cross-model distillation (different teacher and student models)**
Also supported, but tokenization compatibility becomes more important. If token vocabularies/template behavior differ significantly, alignment quality may degrade.
In practice, self-distillation is usually easiest to bring up first, then cross-model can be layered in once your pipeline is stable.