qwen tokenizer wrapper & fixed jinja template for tool handling (#224)

Browse source

* added qwen tokenizer wrapper & fixed jinja template for tool handling issues in the official HF one

* moved jinja template into it's own file

...

This commit is contained in:

shannonsands 2025-07-30 11:57:15 +10:00 • committed by GitHub

parent 56fb50a503

commit 9f23c732dd

No known key found for this signature in database

GPG key ID: B5690EEEBB952194

3 changed files with 117 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

0

atroposlib/utils/tokenizers/__init__.py Normal file

Unescape Escape View file