Run a local model on your own GPU or CPU and open a coding agent on it. Local AI picks a model that was tested on your card (NVIDIA, Intel Arc Pro, AMD) or an x86-64 AVX2 CPU, downloads the weights from a pinned Hugging Face revision and checks them, and serves the model in Docker behind a keyed, OpenAI-compatible gateway on 127.0.0.1. From the bar you open pi, Claude Code, Codex, OpenCode, omp, Crush, Grok, Copilot or Hermes on it in a folder you choose, without editing their config. The panel has two tabs (home: your activity, what runs, your pinned models; models: every model this machine can run, to search, download, start and pin), a hardware page with each card's memory and temperature, a full-screen view with each model's Hugging Face card, keyboard navigation, and can share a running model on your tailnet.
omarchy plugin add https://github.com/sybil-solutions/omarchy-local-ai --enableThen open Local AI in the bar and click Set up Local AI (once per machine; Omarchy asks for your password), pick a model on the models tab (or just type its name) and Start it, and open a coding agent on it. Install says what setup does, Requirements what it needs, and Remove how to take it all back out.
omarchy plugin add https://github.com/sybil-solutions/omarchy-local-ai --enableOpen Local AI in the bar and click Set up Local AI. A terminal opens and runs two of Omarchy's own commands; Local AI itself runs nothing as root, and adding or updating the plugin never runs setup.
omarchy-setup-security-sudoless-docker, Omarchy's Sudoless Docker: it explains what that means, asks you to confirm, then asks "Password for you to turn on Sudoless Docker for Local AI". This login reaches Docker through the new group at once, so Local AI needs no logout or reboot (Omarchy may still suggest one at its next update).- NVIDIA only:
omarchy-pkg-add nvidia-container-toolkit, asking "Password for you to install NVIDIA container support for Local AI"; already installed, it asks nothing. The package describes your GPUs to Docker in/etc/cdi/nvidia.yaml, which Docker reads when a model starts, so Docker's configuration is untouched and it is not restarted.
Setup ends by checking what the panel will show. Sharing a model on your tailnet needs you as Tailscale's operator; Omarchy's Tailscale install sets that, and setup tells you when it is not. After setup, starting, stopping, sharing, refreshing and removing never ask for a password. Sudoless Docker is root-equivalent, as Omarchy's own warning says; turn it off in Setup > Security.
- Omarchy with the Quattro shell and its bar. Local AI calls Omarchy's own commands:
omarchy-setup-security-sudoless-docker,omarchy-pkg-add,omarchy-hw-nvidia,omarchy-launch-tui,omarchy-launch-browser,omarchy-notification-sendandomarchy-cmd-present. - Docker,
jq,curl,flockandsha256sum. - A supported GPU (below), or an x86-64 CPU with AVX2 and enough RAM for its recipe. NVIDIA needs the driver and
nvidia-smi; setup adds the container toolkit. Docker 25 or later (Omarchy ships 29). AMD needs ROCm'samd-smi. - Optional: Tailscale, to share a model; a Hugging Face token in
~/.cache/huggingface/token, used for downloads when it exists. - Network access to
huggingface.co(weights),ghcr.io/sybil-solutions, historicalghcr.io/0xseropins, andghcr.io/ggml-org(the engine and gateway images, pinned by digest), andapi.github.comandraw.githubusercontent.com(the automatic catalog check),huggingface.coagain for model cards in full screen, andusage.sybilsolutions.ai(the install count, below).
Qwen3.8-Flash-Next on one Intel Arc Pro B70 32 GB can keep its routed experts and complete n-gram table in host RAM. This recipe needs 128 GB available RAM and 85.14 GB disk, has a 64K context window, and does not require NVMe. It supports images, not video. Its memory and private shared-memory limits are carried by the v3 catalog; the panel checks host RAM before starting it. Update to 6.13.4 or newer to get this engine repository in the bundled catalog. Automatic catalog refreshes intentionally cannot introduce an engine repository an older plugin version has never shipped.
- Validated models per card, or one across several.
recipes.jsonholds, for each supported hardware kind, every recipe accepted on that exact card or across 2 or 4 of them in local-ai-registry: download, load, a correctness check and speed at several context lengths. EXL3 weights on SGLang or vLLM come first and are recommended. The models tab lists every model this machine can run, on whichever of its cards it lands, as one searchable table; home shows the ones you pinned (running or downloading one pins it). A machine with no tested model says so and links the supported list. - What the machine needs. A recipe that keeps experts in system RAM or reads from disk while it serves (Qwen3.8-Flash-Next on a 3090 or a B70) says how much free RAM and disk it needs and whether the models folder must be on NVMe; it is offered only on a machine that has them; the others are listed as too big, saying what is missing. A recipe may also name the CPU threads its engine pins work to.
- Weights are downloaded as you, from the pinned revision, and every file's size is checked against Hugging Face, and the SHA-256 of every large (LFS) file, before it is used. A matching copy in your Hugging Face cache is reused.
- Containers. The engine runs on a private network with no published port and
no-new-privileges. A keyed gateway runs as you on127.0.0.1, speaks the OpenAI, Anthropic and Responses APIs, and logs one line per answer; the tokens, speeds, charts and the activity grid on Home come from that log, summed once per new line rather than on every refresh. - Agents. pi, Claude Code, Codex, OpenCode, omp, Crush, Grok, Copilot and Hermes open in a terminal, in the folder you pick, pointed at the gateway. Nothing in their own config is touched. A model's details list the agents once, the chosen one checked; on the agents page the chosen one is the default for new models. Update to appears when mise or npm has a newer version of the agent. Click the folder row to open a folder picker. Open launches the chosen agent from home or the model's page. Agent choices on a running model do not change the default; the last folder picked remains the folder default. Updates use mise when it manages that agent, the native updater for standalone omp, Hermes, Claude Code and OpenCode, or npm for an existing npm installation. Existing sessions keep running.
- Share a running model on your tailnet with
tailscale serve(tailnet only, still keyed). Stop sharing on its page removes that share. Tailscale must be running and logged in. Setup preserves another account’s operator; ask that account to manage sharing. A failed unshare is logged and does not prevent stopping the model.
Supported: x86-64 AVX2 CPUs (LFM2.5-2.6B in system RAM), NVIDIA RTX 30, 40 and 50 series, RTX A6000, RTX Ada and RTX Pro Blackwell, Intel Arc Pro B70, and AMD Instinct MI300X and Radeon RX 6800 XT (with ROCm's amd-smi).
bin/omarchy-local-ai setup # one-time machine setup
bin/omarchy-local-ai registry # refresh the validated catalog
bin/omarchy-local-ai forget <recipe> # remove stopped managed weights
bin/omarchy-local-ai snapshot # what the card draws, as JSON
bin/omarchy-local-ai run <recipe> <gpu>[,<gpu>] # e.g. run qwen38-27b-exl3-3bpw-rtx3090-sglang-tp1 nvidia:0
bin/omarchy-local-ai stop <recipe>
bin/omarchy-local-ai open <recipe> # the chosen agent on it, in a terminal
bin/omarchy-local-ai update <agent> # update the installed harness
bin/omarchy-local-ai set agent|folder <value> [recipe]
bin/omarchy-local-ai set ping on|off # whether this install is counted
bin/omarchy-local-ai share <recipe> [off]
bin/omarchy-local-ai logCPU recipes use system RAM and start without GPU devices. Recipes that offload GPU work into RAM or NVMe remain hardware-specific and list their requirements.
Each running model answers on http://127.0.0.1:<port>/v1 (ports from 12434); the key is in ~/.local/state/omarchy/local-ai/gateway.key. Local AI itself, and the agents it opens, send the key only to the gateway's address on the model's private Docker network, which no other account on the machine can take over.
bin/omarchy-remove-ai-local stops this user’s managed models and deletes their containers, engine images, weights and settings. Then omarchy plugin remove sero.local-ai.
| File | Role |
|---|---|
bin/omarchy-local-ai |
The backend, one bash file |
Model.js |
Pure functions: the snapshot in, the view out |
Panel.qml |
Draws the view and runs the backend's verbs |
recipes.json |
The vendored recipes, one card kind per line (make sync) |
docs/design.md has the design and test/all runs the tests.
Images are pinned by SHA-256. The gateway runs as the calling user, with no Linux capabilities, a read-only root filesystem, a bounded temporary directory and external DNS disabled. Docker service discovery still resolves the engine; usage records remain writable in their dedicated directory. Existing containers receive these restrictions when stopped and started again.
Model engines still have writable container filesystems and outbound network access. Digest pins and build attestations establish image identity; they do not make untrusted code safe. Engine restrictions need per-engine hardware validation before rollout. Disabling gateway DNS does not block outbound IP connections.
To count how many machines use it, Local AI sends one small message each time it
checks the registry for new models (every 12 hours while the bar runs) to
https://usage.sybilsolutions.ai/v1/local-ai/ping:
{"id": "9f2c…", "version": "6.13.0", "hw": ["rtx-3090"], "running": 1}id is a random number made on first run and kept in
~/.local/state/omarchy/local-ai/install-id; it is not derived from the machine
or you, and Remove deletes it. hw lists the card kinds Local AI has recipes for
that it found, running how many models run. The server keeps those four fields
and the day, and no IP address or other header. Its code, the D1 migrations and the
deploy config are in usage/ (the worker local-ai-usage and its usage.sybilsolutions.ai route), folded in
from the former local-ai-usage repository; the public totals are the same worker's GET /v1/local-ai/stats.
Turn it off with omarchy-local-ai set ping off (or DO_NOT_TRACK=1 in the
bar's environment). A failed or refused message changes nothing.
New models arrive by themselves: once the catalog is 12 hours old, Local AI checks
the latest published registry (at most once an hour) and says so only when it
brought new models; the catalog lives in your cache (~/.cache/omarchy/local-ai/v3). Downloads are pinned to a registry commit,
validated before an atomic replacement, and failures keep the previous catalog.
Refreshing does not stop running models or download model weights. Offload
recipes are offered only when their RAM, disk and storage requirements fit.
A recipe that generates a pack first declares its output budget separately from raw weights. Local AI prepares with the pinned serving image, verifies the pack and publishes it from private staging. Each reuse is verified. Stop retains the completed pack; Remove download removes it only when no managed model uses it. The v3 contract supports these operations; research models still require their own accepted image, hardware and fidelity evidence before they appear here.
On the models tab, open a model's row to Start it, Download it without starting, Pin it to home, or see its details. Remove download deletes that model's managed weights after it is stopped; shared weights in use by another managed model are protected. The recipe stays available to download again. Files in your separate Hugging Face cache are kept, so hard-linked files there may continue to occupy disk space.
The equivalent commands are omarchy-local-ai registry, download <recipe> [off],
pin <recipe> [off] and forget <recipe> (or the plugin's bin/omarchy-local-ai).
MIT. Model-maker marks in Logos.js come from lobe-icons (MIT) and, for Intel and AMD, Simple Icons (CC0). The agent, model-maker and GPU-maker logos are trademarks of their owners, shown to identify compatibility only; Local AI is not affiliated with or endorsed by any of them.