thinking-machines-lab/tinker
1
0
Fork 0
mirror of https://github.com/thinking-machines-lab/tinker.git synced 2026-04-19 12:58:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Sync contents

Browse source
This commit is contained in:
Daniel Xu 2025-12-15 01:07:10 +00:00
parent e7a0d0ca2d
commit 2d8e9d5e00
3 changed files with 99 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/api/samplingclient.md
View file

@ -11,14 +11,14 @@ Client for text generation and inference from trained or base models.
The SamplingClient lets you generate text tokens from either a base model or from weights
you've saved using a TrainingClient. You typically get one by calling
`service_client.create_sampling_client()` or `training_client.save_weights_and_get_sampling_client()`.
Key methods:
- sample() - generate text completions with customizable parameters
- compute_logprobs() - get log probabilities for prompt tokens
Args:
- `holder`: Internal client managing HTTP connections and async operations
Create method parameters:
- `model_path`: Path to saved model weights (starts with 'tinker://')
- `base_model`: Name of base model to use for inference
- `base_model`: Name of base model to use for inference (e.g., 'Qwen/Qwen3-8B')
- `retry_config`: Configuration for retrying failed requests
Example: