sparklayer-io · rlweb · Jun 15, 2026 · Apr 15, 2026 · Apr 21, 2026 · Apr 21, 2026
diff --git a/.env.example b/.env.example
@@ -11,6 +11,18 @@ ANTHROPIC_API_KEY=sk-ant-...
 # Google Generative AI API key (used for step execution, assertions, extraction)
 GOOGLE_GENERATIVE_AI_API_KEY=AIza...
 
+# =============================================================================
+# Optional: Cache provider
+# =============================================================================
+
+# Cache backend for step caching and global state: redis | file | none
+# Defaults to redis. Use "file" to persist the cache to disk (e.g. committed
+# to Git for CI), or "none" to disable caching entirely.
+# CACHE_PROVIDER=redis
+
+# Directory for the file-based cache (only used when CACHE_PROVIDER=file)
+# CACHE_DIR=.passmark-cache
+
 # =============================================================================
 # Optional: AI Gateway
 # =============================================================================
@@ -25,6 +37,13 @@ GOOGLE_GENERATIVE_AI_API_KEY=AIza...
 # Required only if ai.gateway is set to "openrouter" in configure()
 # OPENROUTER_API_KEY=sk-or-...
 
+# =============================================================================
+# Optional: OpenCode Zen
+# =============================================================================
+
+# Required only if ai.gateway is set to "opencodezen" in configure()
+# OPENCODEZEN_API_KEY=
+
 # =============================================================================
 # Optional: Cloudflare AI Gateway
 # =============================================================================
@@ -36,6 +55,16 @@ GOOGLE_GENERATIVE_AI_API_KEY=AIza...
 # CLOUDFLARE_AI_GATEWAY=
 # CLOUDFLARE_AI_GATEWAY_API_KEY=
 
+# =============================================================================
+# Optional: CUA mode (OpenAI computer-use agent)
+# =============================================================================
+
+# Required only if ai.mode is set to "cua" in configure().
+# CUA requires direct OpenAI access (gateway: "none") and an API key with
+# access to the CUA model (default: gpt-5.5) and the built-in `computer`
+# tool on the Responses API.
+# OPENAI_API_KEY=sk-...
+
 # =============================================================================
 # Optional: Telemetry (Axiom)
 # =============================================================================

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -9,8 +9,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- **OpenCode Zen gateway support**: set `gateway: "opencodezen"` in `configure()` and provide `OPENCODEZEN_API_KEY` to route all model requests through [OpenCode Zen](https://opencode.ai/docs/ko/zen/) (`https://opencode.ai/zen/v1`), an OpenAI-compatible gateway with 30+ curated models including Claude, Gemini, GPT, Qwen, and more.
+- **OpenAI Support**: Direct integration with OpenAI models via `@ai-sdk/openai` and `OPENAI_API_KEY`
 - `maxRetries` option to `AssertionOptions` (default: `1`) to control how many times a failed assertion is retried with a fresh page snapshot and screenshot. Setting it to `0` disables retries.
 - `onRetry` callback to `AssertionOptions` that fires before each retry, receiving the retry index and the full `AssertionResult` from the previous attempt for debugging flaky assertions.
+- **CUA mode** (`configure({ ai: { mode: "cua" } })`): execute `runSteps` and `runUserFlow` through OpenAI's Responses API with the built-in `computer` tool. Screenshot-driven, coordinate-based actions via Playwright's `page.mouse` / `page.keyboard`. Requires `OPENAI_API_KEY` and `gateway: "none"`; Redis step caching is skipped in this mode because coordinate actions aren't portable across viewport sizes.
+- `cua` model slot in `ModelConfig` (default: `gpt-5.5`). For now, you cannot override the CUA model.
+- `getMode()` helper and `AIMode` type exported from `src/config.ts`.
 
 ## [1.0.0] - 2026-03-27
 

diff --git a/README.md b/README.md
@@ -31,9 +31,9 @@ ANTHROPIC_API_KEY=sk-ant-...
 GOOGLE_GENERATIVE_AI_API_KEY=AIza...
 ```
 
-Alternatively, you can use an AI gateway like Vercel AI Gateway or OpenRouter to route requests to multiple providers without managing individual API keys. If you choose this option, set `AI_GATEWAY_API_KEY` (for Vercel) or `OPENROUTER_API_KEY` (for OpenRouter) instead.
+Alternatively, you can use an AI gateway like Vercel AI Gateway, OpenRouter, or OpenCode Zen to route requests to multiple providers without managing individual API keys. If you choose this option, set `AI_GATEWAY_API_KEY` (for Vercel), `OPENROUTER_API_KEY` (for OpenRouter), or `OPENCODEZEN_API_KEY` (for OpenCode Zen) instead.
 
-You can also route requests through Cloudflare AI Gateway for observability, caching, and rate limiting. Unlike Vercel/OpenRouter, Cloudflare is a proxy (not a reseller), so you still need your own `ANTHROPIC_API_KEY` / `GOOGLE_GENERATIVE_AI_API_KEY` alongside `CLOUDFLARE_ACCOUNT_ID` and `CLOUDFLARE_AI_GATEWAY` (and `CLOUDFLARE_AI_GATEWAY_API_KEY` if the gateway has authentication enabled).
+You can also route requests through Cloudflare AI Gateway for observability, caching, and rate limiting. Unlike Vercel/OpenRouter/OpenCode Zen, Cloudflare is a proxy (not a reseller), so you still need your own `ANTHROPIC_API_KEY` / `GOOGLE_GENERATIVE_AI_API_KEY` alongside `CLOUDFLARE_ACCOUNT_ID` and `CLOUDFLARE_AI_GATEWAY` (and `CLOUDFLARE_AI_GATEWAY_API_KEY` if the gateway has authentication enabled).
 
 Set your Playwright project to read `.env` by adding the following to `playwright.config.ts`  (after `import { defineConfig, devices } from '@playwright/test';`):
 
@@ -82,8 +82,9 @@ import { runSteps, configure } from "passmark";
 
 configure({
   ai: {
-    gateway: "vercel" // or "openrouter" or "cloudflare"
-    // Set AI_GATEWAY_API_KEY (Vercel), OPENROUTER_API_KEY (OpenRouter), or
+    gateway: "vercel" // or "openrouter", "opencodezen", or "cloudflare"
+    // Set AI_GATEWAY_API_KEY (Vercel), OPENROUTER_API_KEY (OpenRouter),
+    // OPENCODEZEN_API_KEY (OpenCode Zen), or
     // CLOUDFLARE_ACCOUNT_ID + CLOUDFLARE_AI_GATEWAY (+ CLOUDFLARE_AI_GATEWAY_API_KEY
     // if the gateway is authenticated) in your .env file. Cloudflare also requires
     // the upstream provider keys (ANTHROPIC_API_KEY, GOOGLE_GENERATIVE_AI_API_KEY).
@@ -99,11 +100,76 @@ npx playwright test example.spec.ts --project chromium
 
 After the test completes, you can run `npx playwright show-report` to see a detailed report of the test execution, including an AI summary at the top, provided by Passmark.
 
+### Using CUA mode (OpenAI computer-use agent)
+
+By default Passmark uses ARIA accessibility snapshots. For visual, screenshot-driven automation via OpenAI's computer-use agent, opt in with `mode: "cua"`:
+
+```typescript
+import { configure } from "passmark";
+
+configure({
+  ai: {
+    mode: "cua",
+    gateway: "none", // CUA requires direct OpenAI access
+  },
+});
+```
+
+Set `OPENAI_API_KEY` in your `.env`. Then you can write tests like this:
+
+```typescript
+test("Shopping cart tests", async ({ page }) => {
+  await runSteps({
+    page,
+    userFlow: "Add product to cart",
+    steps: [
+      { description: "Navigate to https://demo.vercel.store" },
+      { description: "Click Acme Circles T-Shirt" },
+      { description: "Select color", data: { value: "White" } },
+      { description: "Add to cart", waitUntil: "My Cart is visible" },
+    ],
+    test,
+    expect,
+  });
+});
+```
+
+Notes:
+
+- CUA mode uses OpenAI's `gpt-5.5` + built-in `computer` tool. The CUA model is currently locked and not user-configurable.
+- Redis step caching is skipped in CUA mode because coordinate actions aren't portable across viewport sizes.
+- `gateway: "vercel" | "openrouter" | "cloudflare"` is not compatible with CUA — the Responses-API `computer` tool is only exposed on direct OpenAI access.
+- Account requirements: your OpenAI API key must have access to the CUA model and the built-in `computer` tool on the Responses API.
+
+#### Per-step overrides (hybrid runs)
+
+The same `ai` shape accepted by `configure()` can also be passed at the `runSteps`/`runUserFlow` call level **and** on individual `Step`s. This lets you mix snapshot steps (cheap, cacheable, OpenRouter/Vercel/etc.) with CUA steps (visual, direct OpenAI) in a single run. Precedence: `step.ai` ▶ call-level `ai` ▶ global `configure()`.
+
+```typescript
+configure({ ai: { gateway: "openrouter" } }); // most steps go through OpenRouter
+
+await runSteps({
+  page, test, expect,
+  userFlow: "Buy product on sale",
+  steps: [
+    { description: "Navigate to /products" },                     // OpenRouter snapshot
+    {
+      description: "Drag the price slider to $40",
+      ai: { mode: "cua", gateway: "none" },                       // CUA for this step only
+    },
+    { description: "Click Add to cart" },                         // back to OpenRouter snapshot
+  ],
+});
+```
+
+Set `OPENAI_API_KEY` whenever any step opts into `mode: "cua"`. CUA steps still require `gateway: "none"`; mixing CUA with a non-`none` gateway throws at the per-step level for the same reason it does globally.
+
 ## Features
 
 - **Core Execution** — `runSteps()` and `runUserFlow()` for flexible test orchestration in natural language, with smart caching and auto-healing
 - **Multi-Model Assertion Engine** — Consensus-based validation using Claude and Gemini, with an arbiter model to resolve disagreements
-- **Pluggable Step Caching** — Cache-first execution with AI fallback and automatic self-healing. Supports Redis, file-based, or custom cache backends.
+- **Video Assertions** — Opt in per-assertion to record the full step run and evaluate the assertion against the whole video via Gemini's Files API. Useful for ephemeral UI (toasts, snackbars) that a single screenshot may miss
+- **Pluggable Step Caching** — Cache-first execution with AI fallback and automatic self-healing. Backed by Redis or a file-based store (`CACHE_PROVIDER=redis|file|none`)
 - **Configurable AI Models** — 8 dedicated model slots for step execution, assertions, extraction, and more
 - **AI Gateway Support** — Route requests through Vercel AI Gateway, OpenRouter, Cloudflare AI Gateway, or connect directly to provider SDKs
 - **Dynamic Placeholders** — Inject values at runtime with `{{run.*}}`, `{{global.*}}`, `{{data.*}}`, and `{{email.*}}` expressions for repeatable and data-driven tests
@@ -151,7 +217,7 @@ const result = await runUserFlow({
 
 ### `assert(options: AssertionOptions)`
 
-Multi-model consensus assertion. Runs Claude and Gemini in parallel; if they disagree, a third model arbitrates.
+Multi-model consensus assertion. Runs Claude and Gemini in parallel; if they disagree, a third model arbitrates (configurable — see [Consensus Policy](#consensus-policy)).
 
 ```typescript
 const result = await assert({
@@ -161,6 +227,53 @@ const result = await assert({
 });
 ```
 
+### Consensus Policy
+
+When the primary (Claude) and secondary (Gemini) assertion models reach the same verdict, the result is used directly. When they **disagree**, you choose how Passmark resolves it:
+
+| Policy | Behavior |
+|---|---|
+| `consult-arbiter-on-disagreement` *(default)* | Calls the arbiter model (Gemini 3.1 Pro) to break the tie. |
+| `fail-on-disagreement` | Treats any disagreement as a failure immediately — no arbiter call. The returned reasoning includes both models' takes so you can inspect what they saw differently. |
+
+Pick `fail-on-disagreement` when you'd rather surface ambiguity/flakiness in the UI under test than let a single model swing the result. Pick the default when you trust the arbiter to make the final call.
+
+```typescript
+configure({
+  assertions: {
+    consensusPolicy: "fail-on-disagreement",
+  },
+});
+```
+
+### Video Assertions
+
+For UI that's only visible for a second or two — toast messages, snackbar confirmations, transient banners — a single end-of-flow screenshot often misses the evidence. Set `video: true` on an assertion inside `runSteps` and Passmark will record the entire step run with `page.screencast`, upload the resulting `.webm` to Gemini's Files API, and evaluate the assertion against the full video:
+
+```typescript
+await runSteps({
+  page,
+  userFlow: "Add to cart",
+  steps: [
+    { description: "Click Acme Circles T-Shirt" },
+    { description: "Add to cart" },
+  ],
+  assertions: [
+    { assertion: "An 'Added to cart' toast appears", video: true },
+  ],
+  test,
+  expect,
+});
+```
+
+Notes:
+
+- Recording spans the **entire** step run (start of first step to end of last step). One recording is shared across all `video: true` assertions in the same `runSteps` call.
+- The video file is written to `/tmp/passmark-recordings/` by default and deleted automatically after the assertions consume it. Override via `configure({ videoDir: "/your/path" })`.
+- This path uses **only Gemini** (no Claude/Gemini consensus) since Claude doesn't accept video. The model is `gemini-3-flash-preview`.
+- Video assertions go **directly** to Gemini's Files API regardless of any configured `gateway` — file URIs are tied to the uploading Google account, so the gateway can't proxy them. You must set `GOOGLE_GENERATIVE_AI_API_KEY` (or `GEMINI_API_KEY`) even when the rest of your stack runs through Vercel / OpenRouter / Cloudflare.
+- If `page.screencast.start()` fails (rare), video assertions silently fall back to the regular screenshot/snapshot path so the run still completes.
+
 ## Configuration
 
 Call `configure()` once before using any functions:
@@ -170,7 +283,7 @@ import { configure } from "passmark";
 
 configure({
   ai: {
-    gateway: "none", // "none" (default), "vercel", "openrouter", or "cloudflare"
+    gateway: "none", // "none" (default), "vercel", "openrouter", "opencodezen", or "cloudflare"
     models: {
       stepExecution: "google/gemini-3-flash",
       utility: "google/gemini-2.5-flash",
@@ -184,18 +297,20 @@ configure({
 
 | Variable | Required | Default | Description |
 |----------|----------|---------|-------------|
-| `CACHE_PROVIDER` | No | - | Cache backend: `redis`, `file`, or `none`. Falls back to Redis if `REDIS_URL` is set. |
+| `CACHE_PROVIDER` | No | `redis` | Cache backend: `redis`, `file`, or `none`. Defaults to Redis. |
 | `CACHE_DIR` | No | `.passmark-cache` | Directory for file-based cache (when `CACHE_PROVIDER=file`) |
-| `REDIS_URL` | No | - | Redis connection URL for step caching and global state |
+| `REDIS_URL` | No | - | Redis connection URL for step caching and global state. Can also be set via `configure({ redis: { url } })`, which takes precedence. |
 | `ANTHROPIC_API_KEY` | Yes | - | Anthropic API key for Claude models |
-| `GOOGLE_GENERATIVE_AI_API_KEY` | Yes | - | Google API key for Gemini models |
+| `GOOGLE_GENERATIVE_AI_API_KEY` | Yes | - | Google API key for Gemini models. Also required for `video: true` assertions regardless of gateway (file URIs are tied to the uploading account). |
+| `OPENAI_API_KEY` | No | - | OpenAI API key for OpenAI models (required for CUA mode; must have Responses-API `computer` tool access) |
 | `AI_GATEWAY_API_KEY` | If gateway=vercel | - | Vercel AI Gateway API key |
 | `OPENROUTER_API_KEY` | If gateway=openrouter | - | OpenRouter API key |
+| `OPENCODEZEN_API_KEY` | If gateway=opencodezen | - | OpenCode Zen API key |
 | `CLOUDFLARE_ACCOUNT_ID` | If gateway=cloudflare | - | Cloudflare account ID that owns the AI Gateway |
 | `CLOUDFLARE_AI_GATEWAY` | If gateway=cloudflare | - | Cloudflare AI Gateway name (slug) |
 | `CLOUDFLARE_AI_GATEWAY_API_KEY` | If gateway=cloudflare and the gateway is authenticated | - | Cloudflare AI Gateway token (sent as `cf-aig-authorization`) |
-| `AXIOM_TOKEN` | No | - | Axiom token for OpenTelemetry tracing |
-| `AXIOM_DATASET` | No | - | Axiom dataset for trace storage |
+| `AXIOM_TOKEN` | No | - | Axiom token for OpenTelemetry tracing. Can also be set via `configure({ telemetry: { axiomToken } })`, which takes precedence. |
+| `AXIOM_DATASET` | No | - | Axiom dataset for trace storage. Can also be set via `configure({ telemetry: { axiomDataset } })`, which takes precedence. |
 | `PASSMARK_LOG_LEVEL` | No | `info` | Log level: `debug`, `info`, `warn`, `error`, `silent` |
 
 ## Model Configuration
@@ -211,6 +326,7 @@ All models are configurable via `configure({ ai: { models: { ... } } })`:
 | `assertionSecondary` | `google/gemini-3-flash` | Secondary assertion model (Gemini) |
 | `assertionArbiter` | `google/gemini-3.1-pro-preview` | Arbiter for assertion disagreements |
 | `utility` | `google/gemini-2.5-flash` | Data extraction, wait conditions |
+| `cua` | `gpt-5.5` | CUA mode — OpenAI Responses API with the built-in `computer` tool |
 
 ## Caching
 
@@ -244,16 +360,27 @@ interface CacheStore {
 
 ### Caching Behavior
 
+Provide the connection via `configure({ redis: { url } })` or the `REDIS_URL` env var (configure value wins). Without either, caching, `{{global.*}}` placeholders, and project data are disabled.
+
 - Steps are cached by `userFlow` + `step.description`
 - Set `bypassCache: true` on individual steps or the entire run to force AI execution
 - Cache is automatically bypassed on Playwright retries
 - Caching only applies to `runSteps`. As of now, only those AI executions that are single-step are cached as multi-step actions can vary widely and are less likely to be identical on subsequent runs. We're exploring ways to safely cache multi-step flows.
 
 ## Telemetry
 
-Telemetry is opt-in. Set `AXIOM_TOKEN` and `AXIOM_DATASET` to enable OpenTelemetry tracing via Axiom. All AI calls are wrapped with `withSpan` for observability.
+Telemetry is opt-in. Either set the `AXIOM_TOKEN` and `AXIOM_DATASET` env vars, or pass them through `configure()`:
+
+```typescript
+configure({
+  telemetry: {
+    axiomToken: process.env.MY_AXIOM_TOKEN,
+    axiomDataset: "passmark-traces",
+  },
+});
+```
 
-Without these env vars, telemetry is a no-op.
+`configure()` values take precedence over env vars. Without either, telemetry is a no-op. All AI calls are wrapped with `withSpan` for observability.
 
 Configure Axiom to get a rich dashboard like this: