NousResearch/atropos
1
0
Fork 0
mirror of https://github.com/NousResearch/atropos.git synced 2026-04-30 17:40:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

remove bloated endpoints

Browse source
This commit is contained in:
Jai Suphavadeeprasit 2026-02-03 09:46:55 -05:00
parent 672cdbaea8
commit 4f6f2faac8
5 changed files with 37 additions and 701 deletions
Download patch file Download diff file Expand all files Collapse all files

6
example_trainer/vllm_manager.py
View file

@ -118,9 +118,9 @@ def launch_vllm_server(
Launch a vLLM server process using our custom vllm_api_server.py.
Uses the custom server instead of standard vLLM because:
- Standard vLLM only has /v1/completions (OpenAI-compatible)
- Our custom server has /generate endpoint needed by VLLMServer class
- This allows proper tokens_and_logprobs_completion support
- Streamlined API: Only /generate endpoint (provides logprobs)
- Weight bridge support: /bridge/* endpoints for shared memory mode
- LoRA hot-swap: /lora/* endpoints for adapter loading/unloading
Args:
config: Training configuration