The ui group of control-omb drives the real React renderer — the same
<App/> the desktop shell loads, mounted by scripts/testing/threads-preview.tsx
— in a headless Chrome through the agent-browser binary the harness pins
(server/browser-engine-release.ts, implementation in
scripts/testing/control-omb-ui.ts). Everything it touches is disposable: the
fake-engine fixture from launch, a Vite preview of the app, and one browser
session whose HOME is the fixture's data directory. The user's app on port
8799 and ~/.openmausbot are never involved.
node --experimental-strip-types scripts/control-omb.ts ui launch \
--tool-calls '[{"name":"Bash","input":{"command":"echo hi"},"ok":true}]'Run it in the foreground so Ctrl-C reaches it. On first use it downloads the
pinned agent-browser release (size and SHA-256 verified) and its Chrome for
Testing into .omb-scratch/verify-tools (gitignored); later launches reuse
them. OMB_AGENT_BROWSER_PATH and AGENT_BROWSER_EXECUTABLE_PATH take
precedence when set. The launcher then starts the fixture, pins its language
to English (PATCH /api/config), creates Pepper through the same new-bot
path as Chat turns, mounts the preview, opens it in a headless
session named omb-ui-<port>, and prints a handle:
{
"ok": true,
"ui": "/tmp/openmausbot-verify-data-XXXXXX/ui.json",
"url": "http://127.0.0.1:PORT",
"previewUrl": "http://127.0.0.1:5178/__threads.html",
"botId": "…", "dataDir": "…", "logPath": "…"
}--tool-calls and --mode script the fake engine (FAKE_CLAUDE_TOOL_CALLS
and FAKE_CLAUDE_MODE in server/testing/fake-claude-cli.ts). Pass ui.json
to every other verb as --ui; there is no discovery, so a recipe cannot drive
a browser it did not launch.
H=/tmp/openmausbot-verify-data-XXXXXX/ui.json
pnpm control:omb ui flag --ui $H --set features.showToolCalls=true --dry-run
pnpm control:omb ui flag --ui $H --set features.showToolCalls=true
pnpm control:omb ui snapshot --ui $H --interactive
pnpm control:omb ui type --ui $H --name "Message Pepper" --text hello
pnpm control:omb ui press --ui $H --keys Enter
pnpm control:omb ui wait-settle --ui $H --timeout 60
pnpm control:omb ui snapshot --ui $H
pnpm control:omb ui click --ui $H --name "Save as skill"
pnpm control:omb ui eval --ui $H --js "document.querySelector('textarea[aria-label=\"Message Pepper\"]').value"snapshot returns the accessibility tree with @eN refs and a refs table
of accessible names and roles. click and type take --ref @eN or the
exact --name, and refuse an ambiguous name by listing the candidates. flag
patches server feature flags (PATCH /api/config with features); the
renderer picks the change up over SSE. wait-settle succeeds only when the
shared wait tool reports the seeded bot settled, no bot in GET /api/bots
is busy, the transcript shows the newest server message (its data-mid row)
and the browser reports network idle; on timeout it exits non-zero with the
last state it saw.
Expected: the interactive snapshot has exactly one textbox "Message Pepper".
After the turn, the log "Conversation with Pepper" landmark contains
StaticText "hello", a StaticText "Bash" tool chip (the scripted call) and
StaticText "hello from fake claude" (the fake engine's reply). The chip is
present only because showToolCalls is on; the sidebar row previews the reply
too, so read the transcript landmark, not the whole tree.
pnpm control:omb ui screenshot --ui $H --out .omb-scratch/verify-evidence/chat-ui.png
pnpm control:omb ui console --ui $H
pnpm control:omb ui eval --ui $H --js "document.title"Keep the wait-settle JSON, both snapshots, the screenshot and the printed
server log path. The console output must contain no error entries.
What the screenshot looks like when the recipe passes (evidence/chat-ui/chat-ui.png):
The permanent form of this recipe is scripts/testing/control-omb-ui.e2e.test.ts:
The fixture also hit-tests the rendered empty band beside This run. It must target the conversation rather than the transparent dock, while points inside the card and composer still target those controls. This checks browser pointer targeting, not native OS wheel delivery.
OMB_UI_E2E=1 pnpm exec vitest run scripts/testing/control-omb-ui.e2e.test.tsThe asserted recipe now also covers the floating This run card with a
successful, failed, and dry-run command. The card records every shell command
the bot ran in the current ask — the commands after the person's last message —
and marks the ones that went through the control CLI verified; reads
(cat, git log, gh pr view, a curl that only fetches) and computer-use
are left out, and a run of one unverified command shows no card. Those tool
outcomes are simulated provider events, not executions of the commands
written in the chips; the card's summary reads 3 steps · 3 verified · 1 failed · 1 dry run. The recipe then presses Save as skill, which fills the
composer in one of two shapes. A run with a verified step opens with the
trigger phrase (Create a verification skill from the run below.), then
Goal: hello (the person's request), the rule not to re-run, and one line per
step with the verified ones tagged (verified). A run with no verified step
asks in plain words instead — Save the steps below as a reusable skill for my review., then Goal: <request>, then "Keep the exact commands and note the
failed ones as gotchas. Do not re-run anything.", then the step lines; the
server (server/skill-learn.ts) expands a turn that opens with that sentence
into the same skill-authoring turn as /learn, so nobody sees or types a slash
command. In both shapes the caret is in the composer and no new user message
was sent — the transcript still holds exactly one StaticText "hello". The person adds any notes and
sends as usual; the card never sends on its own. Separately, the recipe runs a
real fixture health check and verifies that clicking a deliberately missing
control fails. The card remains collapsible; the old execution timeline is no
longer shown above chat.
For activity detail, click More → Inspector → Run Log. It shows the selected
conversation's recorded commands, statuses and timestamps; command previews
may be shortened. Events and Raw retain the underlying technical views.
The recipe checks tab switching and saves run-log.png alongside chat-ui.png.
Copy redacted run log copies only the displayed activity (up to 200 entries),
not chat text or raw protocol data. Review copied logs before sharing: automatic
redaction is best effort. Neither this log nor a successful command proves an
unasserted user outcome.
scripts/testing/usage-details-ui.e2e.test.ts checks the header's More menu,
usage breakdown, clipboard success and refusal, and responsive geometry at
390, 800, 1100 and 1600 px. It checks that opening bot settings folds the sidebar
without changing its saved density and closing settings restores it. Evidence
includes screenshots and a .usage-details.json next to the fixture log.
These are renderer checks, not native Windows caption-button verification.
It runs when an agent-browser binary resolves and is skipped with a printed
reason otherwise; OMB_UI_E2E=1 forces the verified download. The ui-smoke
job in .github/workflows/ci.yml runs it on Ubuntu 24.04 and uploads the
screenshot; it is not one of the required checks.
OMB_UI_E2E=1 pnpm exec vitest run scripts/testing/mobile-code-blocks-ui.e2e.test.ts
pnpm exec vitest run src/components/ChatMarkdown.block-actions.test.ts src/components/ChatMarkdown.test.ts src/lib/copy-text.test.tsThe browser recipe sends a synthetic reply through a disposable fake-engine server and renders it in the real chat. Its five blocks cover plaintext, highlighted code, an unknown language, an omitted language, and indented code. Copy preserves only the selected block's text, including whitespace; Wrap and Unwrap affect only that block and leave the copied source unchanged.
At narrow widths and on coarse-pointer or no-hover devices, the labeled actions occupy their own row with at least 44px touch targets. Check 320px and 390px portrait, landscape and tablet widths, and a desktop with a fine pointer. Long tokens must wrap within the bubble when Wrap is selected; unwrapped code can scroll horizontally inside its block without widening the page. Copy keeps a stable mobile label while its icon shows success or failure; a persistent live region exposes the feedback to assistive technology. Keyboard focus has an outline.
A hosted Linux runner has no mouse, so its headless Chrome reports no hover. The recipe starts Chrome with a fine, hovering pointer and runs the desktop pass before touch emulation starts: once touch emulation is switched on or off, Chrome goes back to the host's own pointer.
This is mobile web renderer coverage in an isolated Chromium profile. It does not verify the separately maintained native iOS or Android clients, physical devices, or Safari. Clipboard permissions are confined to the fixture origin; the refusal case replaces only that disposable page's clipboard writer.
| Before: small icon-only actions | After: labeled touch controls |
|---|---|
![]() |
![]() |
scripts/testing/thinking-timer-ui.e2e.test.ts holds the working row's
elapsed readout to the server's turnStartedAt stamp. It launches the full
app with the fake engine in hang mode, so a sent turn stays officially in
flight and the bot stays busy with no reply arriving. A second thread is
created through the same POST /api/bots/:id/tasks the sidebar's
New thread dispatches, the bot's thread list is expanded through its
chevron, and the test switches to the idle thread and back mid-turn. The
readout before the switch, the readout after the return, and the stamp on
the wire are compared: the count must resume from the stamp (13+ seconds
in), never from the moment of re-selection, and the stamp itself must not
move while the turn runs. A screenshot of the anchored readout is kept as
evidence.
Groups never showed the readout at all — their turns run on the group's
busy slot, not on a member's task, so there was no stamp to count from.
The second test in the same file gives groups their own: a one-member
group is created through the same POST /api/groups the sidebar's group
creation dispatches (with setup completed, so the composer is live at
once), a message routes to the default responder, and the group's claim
of its speaker — the busyBotId transition the server stamps as
turnStartedAt on the group, cleared again when the group goes idle —
must appear in the readout. The test switches to the member's 1:1 thread
and back mid-turn and holds the resumed readout to the same claim stamp,
keeping a second screenshot.
OMB_UI_E2E=1 pnpm exec vitest run scripts/testing/thinking-timer-ui.e2e.test.tsThe desktop shows a reply once it is finished; while a turn runs, the chat
shows the bot's busy state. The renderer keeps no streamed text:
runtimeFrameAction in src/state/store.tsx passes only the model-variant
frames on, and src/state/store.test.ts checks that a reply or reasoning
delta leaves the store state unchanged. src/components/ChatView.follow.test.ts
checks that the finished reply still moves a following transcript to its end.
pnpm exec vitest run src/state/store.test.ts src/components/ChatView.follow.test.tsWhile a step runs, the working line names it ("Running scrape.py", "Searching the web for “Acme construction leads”", "Reading linkedin.com"), with the whole command on hover. Once the step has run past the quiet notice it adds how it is doing: the time since its last output, or what its own processes show ("waiting on the network", "busy, no output yet", "idle, no output yet"). A click on the line, or Enter on it, unfolds the Now card: the current ask's steps, a running command's last lines of output and Open activity, which opens the Activity panel at this run. A second click or Escape folds it. A working bot's sidebar row shows the same step in place of its last message.
A scripted call with hold stays running until FAKE_CLAUDE_RELEASE exists;
hold: "busy", "idle" or "network" also keeps a stand-in process that
burns CPU, sleeps or holds a loopback connection, as the call's command would.
The quiet-status knobs cross into the fixture, so the sign of life shows in
seconds:
OMB_ACP_QUIET_NOTICE_MS=10000 OMB_ACP_QUIET_TICK_MS=2000 FAKE_CLAUDE_RELEASE=/tmp/release \
node --experimental-strip-types scripts/control-omb.ts ui launch --tool-calls \
'[{"name":"WebSearch","input":{"query":"Acme construction leads"},"ok":true},
{"name":"Bash","input":{"command":"cd ~/acme && python3 scrape.py --pages 40"},"hold":"busy"}]'Expected after a sent message: Working · 15s · Running scrape.py · busy, no output yet, the button What Pepper is doing with aria-expanded="false",
and, once clicked, a card listing Searched the web for “Acme construction leads” and
Running scrape.py. Claude Code prints a command's output only when it ends,
so streamed output needs an engine that sends it: --engine codex adds the
fake Codex app-server, and FAKE_CODEX_COMMAND, FAKE_CODEX_COMMAND_OUTPUT
(JSON chunks), FAKE_CODEX_COMMAND_OUTPUT_MS and FAKE_CODEX_COMMAND_GATE
script its command. A bot moved to the codex instance then shows last output just now and the command's last two lines in the card.
The before-and-after screenshots were removed because they showed real example
data; run the scripts/control-omb.ts ui launch command above to see the
working line and the Now card.
The checks are server/step-summary.test.ts (the words and their redaction),
server/harness/live-steps.test.ts and server/harness/bus.test.ts (one path
for every engine, the sign of life, the output's last lines),
server/bot-visibility.test.ts (a step reaches only someone who may see the
chat), src/components/TurnPresence.test.ts,
src/components/ChatView.live-status.test.ts (the line and the card) and
src/components/Sidebar.simple-mode.test.ts (the sidebar row).
scripts/testing/bot-tools-ui.e2e.test.ts uses the same full-app launcher and
optional OMB_UI_E2E=1 gate. It verifies profile-only role creation, closing
and reopening the dialog during a slow creation without duplicate submissions, recovery
when the preset PATCH fails after creation, the composer’s Tools shortcut,
optional setup ideas, a pasted config added from Add MCP server that stays off when it doesn't answer, refreshed
per-bot MCP switches and saved opt-outs, and modal Tab/Escape containment.
The MCP command is an inert fixture command; no real accounts are connected.
The advisory renderer job runs both recipes. Runtime mounting, direct/channel
turns, busy-state rejection and revoked-session imports are separately exercised
by server/mcp-selection.e2e.test.ts against disposable fake-engine servers.
Interrupt ui launch with Ctrl-C. It closes the browser session (waiting
until agent-browser no longer lists it), then the preview, then the fixture,
and removes only its data directory; the server log stays at the printed path
and the tools directory keeps the downloads. Every verb refuses a handle whose
launch has stopped.
The queued-message Edit action must remove the server's held send before
returning its text to the same thread's draft, ahead of any existing text.
To exercise the desktop Claude update card without an actual provider,
launch an isolated UI with FAKE_CLAUDE_MODE=api-error and
FAKE_CLAUDE_API_ERROR="API Error: 400 Claude Code 2.1.268 does not support this model; version 2.1.280 or newer is required.".
Send one short message, then click Update Claude for me. The fake updater
must report its synthetic version and the card must offer Retry, even
when a digest follows the error. This proves the update request and recovery
UI, not a real Claude installation or a successful provider retry.
Proven: the real composer sends a turn on Enter, the fixture runs the scripted fake-engine turn, the transcript renders the sent text, the tool chip and the reply, a server-side feature flag reaches the renderer live, and Save as skill fills the composer without sending — all in a Chromium page, through accessibility names, with no mouse coordinates.
Not proven: the Electron shell (menus, preload bridge, screen capture,
dictation), a real provider, Settings, sidebar drag-and-drop, the VM modal, the
browser panel and updater UI, and anything Show threads gates (that toggle is
renderer localStorage, outside ui flag). AGENT_BROWSER_HEADLESS=1 is set
for consistency with the harness, but agent-browser 0.37.0 is headless by
default and reads AGENT_BROWSER_HEADED to opt out.


