Skip to content

Latest commit

 

History

History
370 lines (290 loc) · 19 KB

README.md

File metadata and controls

370 lines (290 loc) · 19 KB

Verifying OpenMausBot

OpenMausBot has one development control surface: pnpm control:omb. It runs the published openmausbot verbs (docs/headless-cli.md), which call the MCP tools in scripts/mcp-server.ts, so verification uses the same URL validation, task pinning, bounded transcripts, wait states, and redaction as external MCP clients and scripts.

Launch

Start a fixture in one terminal:

node --experimental-strip-types scripts/control-omb.ts launch

Run the foreground launcher directly rather than through pnpm; this ensures it receives Ctrl-C and can stop its child before removing the temporary data.

It gives the child a temporary data directory and home, chooses a free harness/webhook port pair, installs only the repository's fake engine, prints the URL, PID, data directory, and persistent log path, then stays attached to that exact child. The parent shell and the user's OpenMausBot data are untouched. Only FAKE_CLAUDE_* variables cross from the launcher's environment into that child, so a recipe can script the fake engine's mode, replies and tool calls without writing a wrapper CLI. Live calls add one exception: OMB_OPENAI_LIVE_URL crosses when it is a loopback http://127.0.0.1:PORT (the fake GPT-Live that node --experimental-strip-types server/testing/fake-openai-live.ts prints), and OMB_OPENAI_LIVE_KEY crosses only with it, so that key can only ever reach the fake.

Pass the printed URL explicitly from a second terminal:

pnpm control:omb doctor --url http://127.0.0.1:PORT

Mutating commands refuse silent port discovery. This prevents a verification recipe from sending messages to the user's running app by accident.

Drive

Use only mapped, tested commands:

control-omb ui (Chat UI, driven headlessly) drives the real renderer in a headless Chrome by accessible name, so composer sends, transcript rows, tool chips and server feature flags are provable from the command line. Other renderer-only behavior—Settings, sidebar drag-and-drop, the VM modal, the built-in browser panel, and updater UI—is still not proven by the harness. Use the relevant Electron/package smoke test and state that limitation. Add a map entry only after the shared control surface can really drive it.

The switchable local environments recipe verifies creating, switching between and recovering named local environments by hand on a packaged desktop install; the control surface cannot drive it, so it is not in the map above.

The talk shortcut recipe checks hold-to-talk from any app, the reply box and the Quick asks chat by hand on a packaged side-by-side build; the control surface cannot drive a global shortcut or the microphone, so it is not in the map above.

The move to My Cloud fixture runs this computer's server and a Cloud home with the desktop's own wiring between them, and runs and checks the one-time move as main does.

The server of the person's own as home fixture runs this computer's server and servers of the person's own with main's wiring for Make my home: the move, routines that run once, this computer then lent to that server by itself, and none of it for a server shared with others.

The MCP sign-in fixture checks remote sign-in, callback URL paste-back, MCP tools and logout cancellation through the real settings panel and a synthetic OAuth provider in a disposable workspace.

The Whop fixture checks its one-click setup, cancellation, denied consent, discovery failures, reconnect and disconnect using that same registry and a synthetic provider, without accessing a real Whop account.

The bot contact cards fixture checks the AgentMail, AgentPhone, Vapi and Telnyx cards: a pasted key that is refused, replaced and deleted on Disconnect, kept in a header and out of every listing, against a loopback stand-in for Vapi.

The Postiz fixture checks the Apps page's Postiz card in the Social category: Postiz Cloud's sign-in and an API key for a person's own Postiz, the bot's turn getting the postiz tools with the token kept out of every answer, transcript and log, plain refusals, revocation and disconnect, against loopback stand-ins for Postiz.

The X card fixture checks the Apps page's X card: a plan's included calls with the month's use, the person's own treg key (saved, tested, removed, used first) and a used-up month, against loopback stand-ins for treg and the relay.

The Connected Apps OAuth fixture checks blocked-popup recovery, safe authorization links and abandoned-account retries through real renderer components and isolated HTTP routes.

The desktop server connection smoke mounts the real Settings connection component in disposable Electron windows.

The loading screen and tray smoke checks the startup close button, hidden handoff, tray restore, and Quit in a disposable Electron profile.

The optional organization connection smoke checks the real Settings panel and production desktop client against a synthetic Admin server, including cancellation, revocation and unchanged normal startup.

The personal Cloud account smoke checks optional browser sign-in, server-verified Pro status and sign-out in a disposable Electron profile.

The rail of Clouds checks the session's list of Clouds, which pages may read the rail, opening a place by its id, hiding the rail, and buying another Cloud, with a loopback Admin and the real app in a disposable preview.

The embedded server recovery smoke crashes real Electron-owned fixture servers, verifies bounded recovery and private access, and proves quit cancels recovery without replaying an interrupted fixture turn.

The Tailscale discovery fixture checks standalone macOS CLI mode and HTTP tailnet endpoint refresh without touching a real Tailscale installation.

The external runtimes recipe proves a bot's standing comms capability from external-runtimes.json against a disposable server and the fake engine: scope, on-demand token reads, coordinate_bots roots polled through assignment_status, and immediate dispatch to a free teammate.

The cloud preview fixture mounts the real Computer panel against an isolated server for image decoding, loading, and recovery UI checks.

The VPS recovery fixtures reproduce preview/startup contention and Docker-over-SSH timeout cleanup without contacting a real server.

The live browser fixture mounts the real Browser panel with an explicitly selected native engine and Chrome in a disposable home, covering watching, takeover, input, and profile switching.

The page health and recorded browser steps check drives the real engine against a fixture page with failing requests and page errors, and reads the recorded steps back through the server.

The Local VM resume fixture checks idle stop, restart recovery, guarded resume and the stopped-to-ready Computer panel flow.

The local computer launch regression starts the host CUA gate through real Electron in a disposable home, without opening the desktop app or controlling the user's computer.

The remote desktop viewer fixture checks the bundled noVNC page, authenticated WebSocket proxy, desktop input, reconnect and logout against an isolated server, synthetic desktop and VPS SSH forward.

The bot settings fixture checks profile saves, standing instructions, history restore, skill/memory refresh, and stale-response isolation.

The hosted Slack management fixture checks the agent settings link to Admin: hosted-only availability, the member-readable route module, and stale-response isolation.

The chat and settings polish fixture exercises attachment galleries, opt-in video playback, persisted tool details, and responsive settings through the real renderer in an isolated fake-engine workspace.

The rich tables fixture checks Markdown tables and message-scoped CSV/TSV previews, large-file virtualization, sorting, search, export and keyboard interaction in a disposable workspace.

The recent files fixture mounts the Files panel with synthetic digest rows and real message-scoped downloads from a disposable workspace.

The Data viewer fixture checks real DuckDB loading, querying, export, restart persistence, result history and the full renderer's 100,000-row grid in a disposable workspace with synthetic data.

The people invitation fixture checks hosted workspace sign-in, roles and device revocation through the real HTTP API with a stubbed email service. It does not drive the People settings UI through control-omb.

The sidebar fixture checks archive and delete confirmations, their default focus, keyboard wrapping and focus return against two disposable bots; its departments recipe drags bots to reorder them and into new departments.

The sidebar attention geometry fixture measures the Active Threads popover's width and inset in the open sidebar and its column of avatars in a headless Electron window, so the menu never crosses the window's left edge and loses its title.

The avatar provider fixture checks image-provider settings, keyless local generation, saved-key handling, and safe errors with a local fake API.

The independent threads fixture checks nested sidebar navigation, per-thread models, simultaneous direct conversations and thread-scoped Stop.

The custom ACP model fixture checks native model and thinking choices, independent bot defaults and thread overrides, and retained settings and catalogs after restart with an offline ACP engine.

The GitHub Copilot engine checks cover its ACP catalog, model and effort selection, sign-in detection, and moving a bot or conversation between Claude, Copilot and a custom ACP agent with fake engines.

The mobile generated-image checks cover native image attachments and message-scoped download authorization using isolated fixtures.

The guarded external messages fixture checks atomic branch and approval preconditions, retry receipts, and refusal to queue or steer messages from external interfaces. It also verifies bounded request lineage snapshots, Chief continuations and exact-execution Stop without interrupting a newer request.

The Live calls fixture checks the call id on spoken requests, the one "call" row a finished call leaves, its generated title and the desktop's record row, against the fake GPT-Live in a disposable workspace. control-omb cannot start a call, so the recipe also calls the harness's own routes. It proves no real audio, OpenAI key or phone.

The right-to-left fixture checks per-block direction in bot replies and per-line direction in sent turns, with code pinned left-to-right.

The routines fixture checks confirmed proposals, manual and scheduled runs, central run logs, List/Calendar views, and bot-scoped routines using the real renderer and an isolated fake-engine server.

The interval restrictions recipe checks weekday and time-window limits on scheduled routines in that same disposable fixture.

The automation reports recipe checks that every run reports into the bot's chat as one captioned bot message, that quiet runs write nothing, that a run's question is answered in that chat, and the concurrency rule with the person's own turn, in the real renderer and end to end.

The server settings recipe checks browser provider sign-in with an offline CLI and custom-domain validation without touching live accounts.

The engine library fixture checks onboarding and Settings cards, responsive layouts, theme contrast, and status refreshes without losing drafts.

The Claude account recipe checks sign-out, cancellation and retry against an offline Claude CLI confined to a disposable home.

The Claude custom endpoints checks a per-bot DeepSeek/GLM endpoint in Settings and its turn against the fake Claude CLI: its own directory, its model and routing, the token kept out of the engine list, and edits that keep the saved token.

The provider recovery recipe verifies real Grok image transport and Claude authentication against loopback APIs, plus scoped thread approvals and provider safety errors in an isolated desktop UI.

The automatic recovery fixture checks opt-in, one-shot backup model dispatch after proven pre-prompt ACP failures, preserving the conversation, approval level and Stop/new-message precedence.

The skill approval lifecycle recipe checks Deny, missing staged records and active-thread deletion in two isolated app windows, including the surviving conversation and sending again without deleting the bot.

The Codex account recipe checks account switching against an offline Codex CLI whose identity is synthetic and whose credential directory is empty.

The ChatGPT plan recipe checks the separate official OAuth flow, credential rotation, account ownership, and native Responses routing.

The mention fixture checks candidate selection, composer highlighting, sent mentions, multiline scrolling and responsive wrapping in real chat views.

The Group and Goal Local VM recipe checks per-speaker desktop routing, cancellation, and computer authority cleanup.

Evidence

The Gemini speech fixture checks write-only credentials, voice selection and WAV synthesis against an offline provider in an isolated server, with manual steps for real Google access and playback.

The persistence responsiveness benchmark measures synthetic message/log bursts and history scans, with isolated chat, visibility and packaged-worker checks. It is not a production capacity qualification.

The backup responsiveness benchmark compares in-process and worker exports using synthetic data, with isolated backup workflow checks.

The Japanese desktop font recipe checks real Firefox and XFCE glyph rendering in disposable managed desktops, including fresh recreation.

The optional Podman full-stack acceptance recipe checks the Compose deployment with a fresh home, fake engine, and two desktops. It includes workspace ownership, persistence, and proxy authentication checks.

The Podman Firefox sandbox recipe checks the capability set a managed desktop keeps so Firefox can start, with before/after acceptance evidence.

The Hetzner launch record is a dated self-hosting run on a disposable VPS: what passed, what was corrected, and what it does not prove.

Keep the JSON from wait and messages, the exact command sequence, and the fixture's printed log path. Evidence must show both the action and the resulting state. A green unit test alone does not prove a user workflow.

Cleanup

Interrupt the launch process with Ctrl-C. It stops the exact child it owns and removes only its temporary data directory. The server log remains at the printed path. Never kill processes by name and never delete a broad temp root.