Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions docs/cli/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,19 @@ See [Signing & verification](../guides/signing.md).

These exec `basert-<cmd>`. Full flags in their guides.

**Selecting a quant variant.** For the model-taking commands (`serve`, `chat`,
`complete`, `bench`, `profile`), choose which quant build to load with an inline
`:variant` on the hub id, or the `--variant` flag:

```sh
basert serve basecompute/gemma-4-E2B-it:default-q8
basert serve basecompute/gemma-4-E2B-it --variant default-q8 # equivalent
```

Run `basert list` to see installed variants (`default-q4`, `default-q8`, …). An
uninstalled variant is fetched on demand. The inline `:variant` wins if both are
given.

### `basert serve`

OpenAI-compatible HTTP server. See [Serving an API](../guides/serving.md).
Expand Down
14 changes: 14 additions & 0 deletions docs/guides/serving.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,20 @@ basert serve --model Qwen/Qwen3-4B --model Qwen/Qwen3-0.6B --api-key "$KEY"

See the full list of endpoints in the [Server API reference](../reference/server-api.md).

## Choosing a quant build

Pick which quant variant to serve with an inline `:variant` on the id, or the
`--variant` flag. Dense models ship `default-q4` and `default-q8`; MoE models are
q4-only.

```sh
basert serve basecompute/gemma-4-E2B-it:default-q8 --max-context 16000
# equivalently:
basert serve basecompute/gemma-4-E2B-it --variant default-q8 --max-context 16000
```

`basert list` shows installed variants; an uninstalled one is pulled on demand.

## Core flags

| Flag | Default | Purpose |
Expand Down
Loading