Replace the agentic browser fleet's Chromium engine with CloakBrowser

CLAUDE.md

@@ -82,7 +82,7 @@ Only after doing all of the above should you begin writing code.
 # Important commands and conventions:
 
 - Never run `uv sync`, always run `uv sync --all-packages` instead
-- For browser automation, Playwright's Python API is in the root venv (`from playwright.sync_api import sync_playwright`, run via `uv run python`). Chromium installs asynchronously on first container boot (the one-shot `deferred-install` program), so in a fresh workspace confirm it finished -- `supervisorctl status deferred-install` or the marker `/var/lib/minds/deferred-install/done.playwright` -- before launching, or the launch fails with a clear error. It runs as-is under the docker provider's gVisor runtime; if you hit a "No usable sandbox!" error on a runtime without unprivileged user namespaces, pass `chromium.launch(args=["--no-sandbox"])`. See `libs/bootstrap/README.md` for the full deferral contract.
+- For browser automation, Playwright's Python API is in the root venv (`from playwright.sync_api import sync_playwright`, run via `uv run python`). The engine is CloakBrowser (a stealth-patched Chromium fork), not Playwright's own managed Chromium -- pass `executable_path="/opt/cloakbrowser/chrome"` explicitly to `chromium.launch(...)`, since Playwright's own browser-cache lookup only auto-discovers builds it downloaded itself. It installs asynchronously on first container boot (the one-shot `deferred-install` program), so in a fresh workspace confirm it finished -- `supervisorctl status deferred-install` or the marker `/var/lib/minds/deferred-install/done.cloakbrowser` -- before launching, or the launch fails with a clear error. It runs as-is under the docker provider's gVisor runtime; if you hit a "No usable sandbox!" error on a runtime without unprivileged user namespaces, pass `chromium.launch(executable_path="/opt/cloakbrowser/chrome", args=["--no-sandbox"])`. See `libs/bootstrap/README.md` for the full deferral contract.
 
 # Always remember these guidelines:
 

libs/bootstrap/README.md

@@ -53,8 +53,16 @@ The `deferred-install` program in `supervisord.conf` runs
 bake into the Docker image but aren't required by any boot-time service.
 Currently it covers:
 
-- Playwright's Chromium browser + its apt system libraries
-  (`uv run playwright install --with-deps chromium`).
+- Chromium's apt system libraries (`uv run playwright install-deps chromium`
+  -- libs only, no browser download).
+- CloakBrowser, a from-source C++ (Blink/V8) stealth-patched Chromium fork,
+  fetched from a pinned GitHub release + SHA256-verified per-arch (Linux
+  x64/arm64), unpacked to `/opt/cloakbrowser/`. This is the one Chromium
+  binary in the image -- both the agentic browser fleet and any agent's own
+  direct Playwright calls (`chromium.launch(executable_path=...)`) use it;
+  Playwright's own managed-Chromium download is not installed. See
+  `libs/browser/CHROMIUM_ENGINE_DESCRIPTION_AND_FUTURE_GUIDELINES.md` for the
+  full swap-engine/upgrade-version contract.
 
 (The publish-inspiration scan gate's two secret-scanner binaries --
 `betterleaks`, `kingfisher` -- are NOT deferred: they are baked into the

libs/browser/CHROMIUM_ENGINE_DESCRIPTION_AND_FUTURE_GUIDELINES.md

@@ -0,0 +1,103 @@
+# Chromium engine: description and future guidelines
+
+## What this is, in three sentences
+
+The browser fleet drives [CloakHQ/CloakBrowser](https://github.com/CloakHQ/CloakBrowser)
+(a from-source C++/Blink/V8 stealth-patched Chromium fork), pulled from a
+pinned GitHub release under their free "delayed release" tier -- chosen over
+`tiliondev/fortress` specifically because Fortress ships no Linux arm64 build
+(would break the desktop/Lima path on Apple Silicon), while CloakBrowser does.
+The binary lands at `/opt/cloakbrowser/chrome`, fetched and SHA256-verified by
+`scripts/deferred_install.sh`'s `_install_cloakbrowser` on first container
+boot, and pre-baked into the box image ahead of time on cloud slices (see
+`slice_provider.py::_build_cloakbrowser_derived_image` in `mngr`). Exactly
+which named things changed to point at it -- and, just as important, which
+same-named things belong to a *different* library and did **not** change --
+is spelled out explicitly below; don't guess from the diff.
+
+## Exactly which variables changed -- and whose they are
+
+Two different libraries are involved here, Playwright and browser-use, and
+they are **not the same thing** even though one property and one keyword
+argument happen to share the identical name `executable_path`. Mixing them up
+is the easiest way to misread this code. Table, in the order they appear in
+`libs/browser/src/browser/session.py`:
+
+| # | Name | File | Whose namespace | What happened |
+|---|---|---|---|---|
+| 1 | `_PLAYWRIGHT_MARKER` → `_CLOAKBROWSER_MARKER` | `session.py` | **Ours** -- a plain module-level constant we invented, not part of any library's API | Renamed. A `pathlib.Path` gating `deferred_install_ready()`. Value changed from `Path("/var/lib/minds/deferred-install/done.playwright")` to `Path("/var/lib/minds/deferred-install/done.cloakbrowser")`. |
+| 2 | `playwright.chromium.executable_path` | `session.py` | **Playwright's own property**, on Playwright's `BrowserType` object (`playwright.chromium`) | **Deleted, no longer read at all.** This is Playwright's own API -- a read-only attribute that returns wherever Playwright's *own* browser-management system downloaded *its own* managed Chromium binary. It has nothing to do with browser-use. We used to read this to find a Chromium to launch; we don't anymore. |
+| 3 | `chromium_path` | `session.py` | **Ours** -- an ordinary local Python variable inside `LiveBrowser.start()`, belongs to neither library | Still exists, but its source changed: it used to be assigned from reading #2 (Playwright's property); it's now assigned from #4 (our own constant). |
+| 4 | `_CLOAKBROWSER_EXECUTABLE` | `session.py` | **Ours** -- new module-level constant | New. A plain string, `"/opt/cloakbrowser/chrome"`. This is what #3 is now set to. |
+| 5 | `executable_path=` | `session.py` | **browser-use's own keyword argument**, on the `browser_use.BrowserSession(...)` constructor call inside `_build_bu_session` | **Still used, value changed.** This is browser-use's own API -- a constructor parameter that tells browser-use which literal binary file to launch and drive. This is the one that actually matters for what engine runs. It used to receive whatever #2 (Playwright's property) returned; it now receives #3/#4 (our CloakBrowser path). Despite sharing the string `"executable_path"` with #2, #2 and #5 are unrelated attributes on two different objects from two different libraries -- #2 is Playwright reporting its own state, #5 is browser-use accepting a value *we* hand it. |
+| 6 | `_CLOAKBROWSER_VERSION`, `_CLOAKBROWSER_INSTALL_DIR`, `_CLOAKBROWSER_RELEASE_URL`, `_CLOAKBROWSER_SHA256_ARM64`, `_CLOAKBROWSER_SHA256_X64` | `scripts/deferred_install.sh` | **Ours** -- bash `readonly` variables | New. Control what gets fetched, verified, and where it's unpacked. |
+| 7 | `_CLOAKBROWSER_VERSION`, `_CLOAKBROWSER_SHA256_X64`, `_CLOAKBROWSER_INSTALL_DIR`, `_CLOAKBROWSER_RELEASE_URL` | `mngr`'s `slice_provider.py` | **Ours** -- Python `Final[str]` constants | New. Independently-pinned mirror of #6 for the cloud box pre-bake (x64 only, no arm64 constant there -- cloud slices are x86_64 bare metal). Kept in sync with #6 **by hand**, not by shared code -- see the upgrade steps below. |
+
+The `playwright` Python **package** itself is still a dependency and still
+installed (`pyproject.toml`, unchanged) -- it's still used for the CDP
+*observer* connection (`playwright.chromium.connect_over_cdp`, protocol-level,
+engine-agnostic) and by any agent's own direct scripted use
+(`from playwright.sync_api import sync_playwright`). Only its *browser*
+download (row 2, and the `deferred_install.sh` step that used to fetch it) is
+gone.
+
+## Upgrading CloakBrowser to a new version
+
+1. Browse <https://github.com/CloakHQ/CloakBrowser/releases> and pick a tag
+   that actually has downloadable `cloakbrowser-linux-x64.tar.gz` /
+   `-linux-arm64.tar.gz` assets attached -- not just `SHA256SUMS` /
+   `SHA256SUMS.sig`. Their newest major version is routinely Pro-gated (no
+   public binary at all); the release you want is the newest one *with* real
+   assets, which may be one or more majors behind their latest tag.
+2. Download that release's `SHA256SUMS` file and copy the `x64`/`arm64` hash
+   lines out of it.
+3. In `scripts/deferred_install.sh`, update the three constants together:
+   `_CLOAKBROWSER_VERSION` (the release tag), `_CLOAKBROWSER_SHA256_X64`, and
+   `_CLOAKBROWSER_SHA256_ARM64`.
+4. In `mngr`'s `slice_provider.py`, update the matching
+   `_CLOAKBROWSER_VERSION` and `_CLOAKBROWSER_SHA256_X64` (no arm64 constant
+   there -- see row 7 above). These are a **separate, manually-duplicated
+   pin** -- forgetting this step means cloud slices keep baking the old
+   version while desktop/Lima gets the new one.
+5. Nothing else needs to change. The binary always unpacks to the same
+   `/opt/cloakbrowser/chrome` path regardless of version -- verify this stays
+   true for the new release by actually downloading and `tar tzf`-listing it
+   (don't assume; CloakBrowser ships a flat archive with the binary literally
+   named `chrome`, but a future release could restructure that).
+6. Re-run `libs/browser/browser_test.py` and, ideally, boot a real workspace
+   off the change and drive the fleet once before merging.
+
+## When Fortress ships Linux arm64 (or any other engine swap)
+
+Fortress (`tiliondev/fortress`) was the first candidate considered here and
+was rejected on exactly one blocking fact: no Linux arm64 build, native or
+Docker. Their own roadmap lists `linux/arm64 Docker image` as an unshipped
+item. **When that changes, it's worth actively re-evaluating**, not just
+noting the option -- Fortress's stealth claims (0% CreepJS headless/stealth,
+a published gauntlet, monthly Chromium rebase) are stronger than CloakBrowser's
+free tier, which trails their paid tier by roughly one to two Chromium majors
+at any given time. Check <https://github.com/tiliondev/fortress/releases> and
+their roadmap section for arm64 status.
+
+If/when a swap is worth doing, same two touch points as any engine swap, no
+architecture change required:
+
+1. **Download and inspect the actual release tarball first.** Do not assume
+   the binary's name or directory layout from a README. (CloakBrowser turned
+   out to be a flat archive with the binary literally named `chrome`;
+   Fortress's own quick-start docs implied a `tilion-fortress/` wrapper
+   directory with a `tilion` binary inside -- these are not the same shape,
+   and guessing wrong here fails silently until the fleet tries to launch.)
+2. In `scripts/deferred_install.sh`, replace the CloakBrowser-specific bits in
+   (or rename) `_install_cloakbrowser`: the release URL, asset name(s), and
+   SHA256 pin(s) -- rows 6 above.
+3. In `session.py`, update `_CLOAKBROWSER_EXECUTABLE` (row 4) to the new
+   binary's real path.
+4. Mirror both of the above in `mngr`'s `slice_provider.py`
+   (`_build_cloakbrowser_derived_image`) for the cloud pre-bake -- row 7.
+
+That's the entire contract. Any Chromium-family fork that accepts standard
+launch flags (`--headless`, `--user-data-dir`, `--remote-debugging-port`,
+`--no-sandbox`) and speaks CDP satisfies it -- browser-use's
+`BrowserSession(executable_path=...)` (row 5) and the CDP observer connection
+don't know or care which engine is behind that path.

libs/browser/README.md

@@ -8,12 +8,15 @@ agent, identified by its `MNGR_AGENT_ID`, or the human).
   thread-per-connection) that owns every browser. browser_use, Playwright (async),
   and the per-browser ownership state machine run on one background asyncio event
   loop, reached from the Flask threads through a single `run_coroutine_threadsafe`
-  bridge. Each browser is a headless Chromium driven by `browser_use.BrowserSession`, observed
-  over the same CDP endpoint to stream a live view (`Page.startScreencast` ->
-  base64 JPEG frames over a WebSocket) and inject human input. Each browser is
-  addressed by a random ~2-word english NAME (e.g. `alex-smith`), generated on
-  demand and never reused; the fleet starts empty and there is no default
-  browser.
+  bridge. Each browser is a headless CloakBrowser -- a from-source C++
+  (Blink/V8) stealth-patched Chromium fork at `/opt/cloakbrowser/chrome` (see
+  `scripts/deferred_install.sh`), not Playwright's own managed Chromium --
+  driven by `browser_use.BrowserSession` (`executable_path` pinned to that
+  fixed path), observed over the same CDP endpoint to stream a live view
+  (`Page.startScreencast` -> base64 JPEG frames over a WebSocket) and inject
+  human input. Each browser is addressed by a random ~2-word english NAME
+  (e.g. `alex-smith`), generated on demand and never reused; the fleet starts
+  empty and there is no default browser.
 - **Ownership** is one locked, compare-and-set state machine per browser. Agents
   never preempt each other -- a second agent waits in a FIFO queue
   (monitor-and-wait). The human can take control from the UI at any time, which

libs/browser/browser_test.py

@@ -76,11 +76,11 @@ def test_deferred_install_ready_gates_on_marker(
     monkeypatch: pytest.MonkeyPatch, tmp_path: Path
 ) -> None:
     monkeypatch.delenv("BROWSER_SKIP_INSTALL_CHECK", raising=False)
-    play = tmp_path / "done.playwright"
-    monkeypatch.setattr(bsession, "_PLAYWRIGHT_MARKER", play)
+    marker = tmp_path / "done.cloakbrowser"
+    monkeypatch.setattr(bsession, "_CLOAKBROWSER_MARKER", marker)
     ready, _ = bsession.deferred_install_ready()
     assert ready is False
-    play.write_text("")
+    marker.write_text("")
     ready, reason = bsession.deferred_install_ready()
     assert ready is True
     assert reason == "ready"

libs/browser/src/browser/session.py

@@ -105,10 +105,16 @@
 # tab feels snappier. Slightly more bandwidth than skipping frames.
 _SCREENCAST_EVERY_NTH_FRAME = 1
 
-# Deferred-install marker (see scripts/deferred_install.sh). Chromium installs
-# asynchronously on first container boot; launching a browser before it exists
-# fails, so callers gate on this. No Xvfb: CDP streaming/input are headless.
-_PLAYWRIGHT_MARKER = Path("/var/lib/minds/deferred-install/done.playwright")
+# Deferred-install marker (see scripts/deferred_install.sh). CloakBrowser
+# installs asynchronously on first container boot; launching a browser before
+# it exists fails, so callers gate on this. No Xvfb: CDP streaming/input are
+# headless.
+_CLOAKBROWSER_MARKER = Path("/var/lib/minds/deferred-install/done.cloakbrowser")
+
+# CloakBrowser's fixed install path (see scripts/deferred_install.sh's
+# _install_cloakbrowser). A stealth, C++-patched Chromium fork -- replaces
+# vanilla Chromium as the engine for every browser the fleet launches.
+_CLOAKBROWSER_EXECUTABLE = "/opt/cloakbrowser/chrome"
 
 # Default model. browser-use's own default LLM is ChatBrowserUse (its hosted
 # model), so to drive with the user's Anthropic key we pass ChatAnthropic
@@ -329,7 +335,7 @@ def deferred_install_ready() -> tuple[bool, str]:
     """Return ``(ready, reason)`` once Chromium is installed."""
     if os.environ.get("BROWSER_SKIP_INSTALL_CHECK") == "1":
         return True, "ready"  # host/CI testing without the deferred-install marker
-    if not _PLAYWRIGHT_MARKER.exists():
+    if not _CLOAKBROWSER_MARKER.exists():
         return False, "Chromium is still installing in this workspace; try again in a minute."
     return True, "ready"
 
@@ -542,7 +548,11 @@ async def start(
         """
         self._playwright = playwright
         self._input_enabled.set()
-        chromium_path = playwright.chromium.executable_path
+        # Fixed CloakBrowser path, not playwright.chromium.executable_path --
+        # the fleet's engine is CloakBrowser, not Playwright's own managed
+        # Chromium (which vanilla Playwright calls elsewhere in this image
+        # still use). See _CLOAKBROWSER_EXECUTABLE.
+        chromium_path = _CLOAKBROWSER_EXECUTABLE
         profile_dir = _profile_dir(self.browser_id)
         profile_dir.mkdir(parents=True, exist_ok=True)
         _clear_stale_singleton(profile_dir)  # a prior hard kill may have orphaned a lock

libs/browser/test_browser_integration.py

@@ -415,7 +415,7 @@ def test_http_release_requires_ownership(monkeypatch: pytest.MonkeyPatch) -> Non
 
 def test_http_new_browser_blocked_until_chromium_installed(monkeypatch: pytest.MonkeyPatch) -> None:
     monkeypatch.delenv("BROWSER_SKIP_INSTALL_CHECK", raising=False)
-    monkeypatch.setattr(bsession, "_PLAYWRIGHT_MARKER", bsession.Path("/nonexistent/marker"))
+    monkeypatch.setattr(bsession, "_CLOAKBROWSER_MARKER", bsession.Path("/nonexistent/marker"))
     client = runner.application.test_client()
     resp = client.post("/browsers")
     assert resp.status_code == 503