Skip to content

docs: revamp README (+ pi plugin, fix docs link) #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
prabod merged 1 commit into from
Jun 30, 2026
Merged

docs: revamp README (+ pi plugin, fix docs link) #12

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 76 additions & 57 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,55 +1,63 @@
<div align="center">

# BaseRT

**A fast LLM inference runtime for Apple Silicon (Metal).** Pull a model, chat
with it, or serve an OpenAI-compatible API — all from one CLI, `basert`.
**A fast LLM inference runtime for Apple Silicon.**

Pull a model from HuggingFace, chat with it, or serve an OpenAI-compatible API — all from one CLI.

This repository is the open ecosystem around the engine: the `basert` CLI
(model hub + converter), the `.base` model format, the public C API, and
language bindings for Python, Node, Rust, and Swift.
[![Docs](https://img.shields.io/badge/docs-docs.basecompute.co-5b8fa8)](https://docs.basecompute.co)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](LICENSE)
[![Platform](https://img.shields.io/badge/platform-Apple%20Silicon-black)](#requirements)
[![Bindings](https://img.shields.io/badge/bindings-Python%20%C2%B7%20Node%20%C2%B7%20Rust%20%C2%B7%20Swift-444)](docs/bindings/index.md)

> **Full documentation:** [docs/](docs/), also published at
> [basecompute.co/docs](https://basecompute.co/docs).
</div>

---

## Quickstart
BaseRT runs large language models locally on Apple Silicon, accelerated by
hand-written Metal kernels. This repository is the open ecosystem around the
engine: the `basert` CLI (model hub + converter), the `.base` model format, the
public C API, and language bindings for Python, Node, Rust, and Swift.

### 1. Install
- **One CLI for everything** — `basert pull`, `chat`, `serve`, `convert`.
- **OpenAI-compatible server** — chat, completions, embeddings, transcription,
tool calls, continuous batching, paged-KV, and prefix caching.
- **Multimodal** — text, vision, and audio on supported models.
- **Its own `.base` format** — affine quantization (Q2–Q8), optional AWQ
calibration, and signed bundles.
- **Bindings everywhere** — Python, Node, Rust, and Swift over a stable C API.
- **A drop-in coding-agent backend** — see [below](#use-it-as-a-coding-agent).

```sh
curl -LsSf https://basecompute.co/install.sh | sh
```
> **Documentation:** **[docs.basecompute.co](https://docs.basecompute.co)** (source in [`docs/`](docs/index.md)).

This downloads the prebuilt engine + the `basert` CLI to `~/.basert` and adds it
to your `PATH`. Requires Apple Silicon (M1+) and macOS 14+. Restart your shell
afterward (or `export PATH="$HOME/.basert:$PATH"`).
## Requirements

<details>
<summary>Build from source instead</summary>
- Apple Silicon (M1 or later), macOS 14+.

```sh
# Prebuilt engine (libbaseRT.dylib + basert-* tools), into build/
gh release download --repo basecompute/baseRT --pattern 'basert-engine-macos-arm64*.tar.gz'
mkdir -p build && tar -xzf basert-engine-macos-arm64*.tar.gz -C build
## Quickstart

# The basert CLI (model hub + converter + launcher) — needs Rust 1.80+
cd base-convert && cargo build --release && cd ..
### 1. Install

export PATH="$PWD/build:$PWD/base-convert/target/release:$PATH"
```sh
curl -LsSf https://basecompute.co/install.sh | sh
```
</details>

See [docs/getting-started/installation.md](docs/getting-started/installation.md)
for details.
This installs the prebuilt engine and the `basert` CLI to `~/.basert` and adds
it to your `PATH`. Restart your shell afterward (or
`export PATH="$HOME/.basert:$PATH"`). See
[Installation](docs/getting-started/installation.md) for building from source.

### 2. Pull a model

```sh
basert pull Qwen/Qwen3-4B # download from HuggingFace + convert to .base
basert list # show installed models
basert pull Qwen/Qwen3-4B # download from HuggingFace + convert to .base
basert list # show installed models
```

Models are cached under `~/.cache/baseRT/models` (`$BASERT_MODELS_DIR`).
Models are cached under `~/.cache/baseRT/models` (override with
`$BASERT_MODELS_DIR`). To pick a specific quant build, append `:default-q8` to
the id (or pass `--variant default-q8`).

### 3. Chat

Expand All @@ -61,33 +69,48 @@ basert chat Qwen/Qwen3-4B

```sh
basert serve Qwen/Qwen3-4B --api-key "$(uuidgen)" --port 8080
# load several models at once with --model (repeatable):
# basert serve --model models/a.base --model models/b.base
```

```sh
curl http://127.0.0.1:8080/v1/chat/completions \
-H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \
-d '{"model":"Qwen3-4B","messages":[{"role":"user","content":"Hello!"}]}'
-d '{"model":"Qwen/Qwen3-4B","messages":[{"role":"user","content":"Hello!"}]}'
```

### 5. Or call it from your language

```python
import baseRT
m = baseRT.Model("models/your-model.base") # kernels embedded; no metallib needed
print(m.generate_text("The capital of France is", max_tokens=64))

model = baseRT.Model("models/your-model.base")
print(model.generate_text("The capital of France is", max_tokens=64))
```

Node, Rust, and Swift bindings work the same way — see
[docs/bindings/](docs/bindings/index.md).
[Bindings](docs/bindings/index.md).

---
## Use it as a coding agent

BaseRT runs as a local backend for the [pi](https://pi.dev) coding agent through
the **[pi-basert](https://github.com/basecompute/pi-basert)** extension, which
auto-discovers whatever models your server is running.

```sh
# 1. Serve a model (a dense model is best for tool calling)
basert serve basecompute/gemma-4-E2B-it

# 2. Install the extension and launch pi
pi install git:github.com/basecompute/pi-basert
pi
```

Inside pi, run `/model` to pick a served model. See the
[pi-basert README](https://github.com/basecompute/pi-basert) for details.

## The `basert` CLI

`basert` is a single front-end. Model-management commands run natively; runtime
commands are forwarded to the matching `basert-<cmd>` engine binary:
commands are forwarded to the matching `basert-<cmd>` engine binary.

| Command | What it does |
| --- | --- |
Expand All @@ -101,34 +124,30 @@ commands are forwarded to the matching `basert-<cmd>` engine binary:
| `basert inspect <model>` | Dump a `.base` header + tensor inventory |
| `basert sign` / `verify` / `keygen` | ed25519 signing of `.base` bundles |

Full flag reference: [docs/cli/reference.md](docs/cli/reference.md).
Full flag reference: [CLI reference](docs/cli/reference.md).

## What's in this repo

- **`base-convert/`** — the `basert` CLI (Rust): model hub (`base-hub`),
converter, and launcher. Includes the `.base` format spec
([FORMAT.md](base-convert/FORMAT.md)), the quantization spec
([CANONICAL_QUANT_SPEC.md](base-convert/CANONICAL_QUANT_SPEC.md)), and the
generic quant profiles.
- **`include/baseRT/`** — the stable public C API (`baseRT.h`, `types.h`).
- **`bindings/`** — Python, Node, Rust, Swift.
- **`benchmarks/`** — scripts + reference results.
- **`docs/`** — the documentation site.
| Path | Contents |
| --- | --- |
| [`base-convert/`](base-convert/) | The `basert` CLI (Rust): model hub, converter, launcher — plus the [`.base` format](base-convert/FORMAT.md) and [quantization](base-convert/CANONICAL_QUANT_SPEC.md) specs and the generic quant profiles. |
| [`include/baseRT/`](include/baseRT/) | The stable public C API (`baseRT.h`, `types.h`). |
| [`bindings/`](bindings/) | Python, Node, Rust, and Swift. |
| [`benchmarks/`](benchmarks/) | Scripts and reference results. |
| [`docs/`](docs/index.md) | The documentation site. |

Everything in this repository is open source (Apache-2.0). The engine ships as
a separate prebuilt binary under a proprietary license — see
[docs/reference/engine-releases.md](docs/reference/engine-releases.md).
This repository — the CLI, `.base` format, public headers, and bindings — is
licensed under **Apache-2.0**. The engine ships as a separate prebuilt binary
under its own license; see
[Engine releases](docs/reference/engine-releases.md).

## Links

- [Documentation](docs/index.md)
- [Server API](docs/reference/server-api.md)
- [`.base` format](base-convert/FORMAT.md)
- [Security](SECURITY.md) · [Contributing](CONTRIBUTING.md)
- **[Documentation](https://docs.basecompute.co)** · [Server API](docs/reference/server-api.md) · [`.base` format](base-convert/FORMAT.md)
- [Security policy](SECURITY.md) · [Contributing](CONTRIBUTING.md)

## License

This repository (the CLI, `.base` format, public headers, and bindings) is
licensed under Apache-2.0 — see `LICENSE` and `NOTICE`. The prebuilt engine
Apache-2.0 — see [`LICENSE`](LICENSE) and [`NOTICE`](NOTICE). The prebuilt engine
binary distributed via [Releases](https://github.com/basecompute/baseRT/releases)
is proprietary and ships under its own license.
Loading