mirror of
https://github.com/thinking-machines-lab/tinker.git
synced 2026-04-19 12:58:01 +00:00
Sync contents
This commit is contained in:
parent
e7a0d0ca2d
commit
2d8e9d5e00
3 changed files with 99 additions and 7 deletions
|
|
@ -1,4 +1,4 @@
|
||||||
{
|
{
|
||||||
"last_synced_sha": "6752e178932fa060b6a916ff0e2aefd1d0410970",
|
"last_synced_sha": "e91a52a27e4676c1c349cdc2da15dc89685770cd",
|
||||||
"last_sync_time": "2025-12-15T01:00:20.438218"
|
"last_sync_time": "2025-12-15T01:07:10.226751"
|
||||||
}
|
}
|
||||||
|
|
@ -11,14 +11,14 @@ Client for text generation and inference from trained or base models.
|
||||||
The SamplingClient lets you generate text tokens from either a base model or from weights
|
The SamplingClient lets you generate text tokens from either a base model or from weights
|
||||||
you've saved using a TrainingClient. You typically get one by calling
|
you've saved using a TrainingClient. You typically get one by calling
|
||||||
`service_client.create_sampling_client()` or `training_client.save_weights_and_get_sampling_client()`.
|
`service_client.create_sampling_client()` or `training_client.save_weights_and_get_sampling_client()`.
|
||||||
|
|
||||||
Key methods:
|
Key methods:
|
||||||
- sample() - generate text completions with customizable parameters
|
- sample() - generate text completions with customizable parameters
|
||||||
- compute_logprobs() - get log probabilities for prompt tokens
|
- compute_logprobs() - get log probabilities for prompt tokens
|
||||||
|
|
||||||
Args:
|
Create method parameters:
|
||||||
- `holder`: Internal client managing HTTP connections and async operations
|
|
||||||
- `model_path`: Path to saved model weights (starts with 'tinker://')
|
- `model_path`: Path to saved model weights (starts with 'tinker://')
|
||||||
- `base_model`: Name of base model to use for inference
|
- `base_model`: Name of base model to use for inference (e.g., 'Qwen/Qwen3-8B')
|
||||||
- `retry_config`: Configuration for retrying failed requests
|
- `retry_config`: Configuration for retrying failed requests
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
|
|
|
||||||
|
|
@ -20,6 +20,38 @@ Coefficient used for computing running averages of gradient square
|
||||||
|
|
||||||
Term added to the denominator to improve numerical stability
|
Term added to the denominator to improve numerical stability
|
||||||
|
|
||||||
|
#### `weight_decay`
|
||||||
|
|
||||||
|
Weight decay for the optimizer. Uses decoupled weight decay.
|
||||||
|
|
||||||
|
#### `grad_clip_norm`
|
||||||
|
|
||||||
|
Gradient clip norm for the optimizer. 0.0 means no clipping.
|
||||||
|
|
||||||
|
## `SupportedModel` Objects
|
||||||
|
|
||||||
|
```python
|
||||||
|
class SupportedModel(BaseModel)
|
||||||
|
```
|
||||||
|
|
||||||
|
Information about a model supported by the server.
|
||||||
|
|
||||||
|
#### `model_name`
|
||||||
|
|
||||||
|
The name of the supported model.
|
||||||
|
|
||||||
|
## `GetServerCapabilitiesResponse` Objects
|
||||||
|
|
||||||
|
```python
|
||||||
|
class GetServerCapabilitiesResponse(BaseModel)
|
||||||
|
```
|
||||||
|
|
||||||
|
Response containing the server's supported models and capabilities.
|
||||||
|
|
||||||
|
#### `supported_models`
|
||||||
|
|
||||||
|
List of models available on the server.
|
||||||
|
|
||||||
## `OptimStepResponse` Objects
|
## `OptimStepResponse` Objects
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
|
@ -405,7 +437,7 @@ class ForwardBackwardOutput(BaseModel)
|
||||||
|
|
||||||
#### `loss_fn_output_type`
|
#### `loss_fn_output_type`
|
||||||
|
|
||||||
The type of the ForwardBackward output. Can be one of [...] TODO
|
The class name of the loss function output records (e.g., 'TorchLossReturn', 'ArrayRecord').
|
||||||
|
|
||||||
#### `loss_fn_outputs`
|
#### `loss_fn_outputs`
|
||||||
|
|
||||||
|
|
@ -444,6 +476,58 @@ class CreateSamplingSessionResponse(BaseModel)
|
||||||
|
|
||||||
The generated sampling session ID
|
The generated sampling session ID
|
||||||
|
|
||||||
|
## `ModelData` Objects
|
||||||
|
|
||||||
|
```python
|
||||||
|
class ModelData(BaseModel)
|
||||||
|
```
|
||||||
|
|
||||||
|
Metadata about a model's architecture and configuration.
|
||||||
|
|
||||||
|
#### `arch`
|
||||||
|
|
||||||
|
The model architecture identifier.
|
||||||
|
|
||||||
|
#### `model_name`
|
||||||
|
|
||||||
|
The human-readable model name.
|
||||||
|
|
||||||
|
#### `tokenizer_id`
|
||||||
|
|
||||||
|
The identifier of the tokenizer used by this model.
|
||||||
|
|
||||||
|
## `GetInfoResponse` Objects
|
||||||
|
|
||||||
|
```python
|
||||||
|
class GetInfoResponse(BaseModel)
|
||||||
|
```
|
||||||
|
|
||||||
|
Response containing information about a training client's model.
|
||||||
|
|
||||||
|
#### `type`
|
||||||
|
|
||||||
|
Response type identifier.
|
||||||
|
|
||||||
|
#### `model_data`
|
||||||
|
|
||||||
|
Detailed metadata about the model.
|
||||||
|
|
||||||
|
#### `model_id`
|
||||||
|
|
||||||
|
Unique identifier for the model.
|
||||||
|
|
||||||
|
#### `is_lora`
|
||||||
|
|
||||||
|
Whether this is a LoRA fine-tuned model.
|
||||||
|
|
||||||
|
#### `lora_rank`
|
||||||
|
|
||||||
|
The rank of the LoRA adaptation, if applicable.
|
||||||
|
|
||||||
|
#### `model_name`
|
||||||
|
|
||||||
|
The name of the model.
|
||||||
|
|
||||||
## `Cursor` Objects
|
## `Cursor` Objects
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
|
@ -470,7 +554,15 @@ class CreateModelRequest(StrictBase)
|
||||||
|
|
||||||
#### `base_model`
|
#### `base_model`
|
||||||
|
|
||||||
Optional metadata about this model/training run, set by the end-user
|
The name of the base model to fine-tune (e.g., 'Qwen/Qwen3-8B').
|
||||||
|
|
||||||
|
#### `user_metadata`
|
||||||
|
|
||||||
|
Optional metadata about this model/training run, set by the end-user.
|
||||||
|
|
||||||
|
#### `lora_config`
|
||||||
|
|
||||||
|
LoRA configuration
|
||||||
|
|
||||||
## `Datum` Objects
|
## `Datum` Objects
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue