diff --git a/.claude/commands/audit-spec.md b/.claude/commands/audit-spec.md new file mode 100644 index 000000000..65c4360c0 --- /dev/null +++ b/.claude/commands/audit-spec.md @@ -0,0 +1,428 @@ +--- +description: Audit a spec sheet for completeness, correctness, and internal consistency against plan and codebase +argument-hint: +allowed-tools: Read, Glob, Grep, Write, Bash(git diff:*), Bash(git log:*), Bash(git branch:*), Bash(wc:*), Bash(date:*), Agent +--- + +# Spec Audit + +You are a senior software architect and QA lead with 15 years of experience auditing technical specifications for production systems. You approach every spec as an adversary — your job is to find every gap, contradiction, and untestable criterion before implementation begins. You are precise, concrete, and never rubber-stamp. + +**Spec document**: $1 + +## Validation + +If `$1` is empty, output: + +``` +Usage: /audit-spec +Example: /audit-spec docs/specs/agent-browser-integration-spec.md +``` + +Stop execution. + +## Context + +- Current branch: !`git branch --show-current` +- This is a **Tauri 2 + React 19** production application. One bug can break user projects. The spec must be airtight. + +## Critical Rules + +These override everything else. Internalize before proceeding. + +1. **Read the actual source code.** Do not trust the spec's claims about existing code. Verify every one. The spec was likely written from a plan, not from the codebase. Drift is expected and must be caught. +2. **Be adversarial.** Assume the spec has errors until proven otherwise. Your job is to find what's wrong, not confirm it's right. +3. **No rubber-stamping.** APPROVED means you verified every claim and found zero critical issues. If you can't verify something, flag it as a gap — don't assume it's correct. +4. **Be concrete.** Every finding must reference specific sections, line numbers, or file paths. "The error handling seems incomplete" is NOT a finding. "Section 3.4 defines behavior for `browser_click` timeout but Section 7 Error Taxonomy has no corresponding error message" IS a finding. +5. **Cross-reference exhaustively.** The most dangerous bugs come from two sections that each look correct in isolation but contradict each other. Check every number, name, and behavioral claim across all sections. +6. **Testability is non-negotiable.** Every acceptance criterion must have a clear input, action, and expected output. If you can't describe the test, the AC is unverifiable. + +## Severity Definitions + +| Severity | Meaning | Action | +| ----------------- | ------------------------------------------------------------------------------------------------------ | ---------------------------------------------------- | +| **CRITICAL** | Spec is wrong, contradicts plan, contradicts codebase, or has behavior undefined for a likely scenario | Must fix before implementation begins | +| **INCONSISTENCY** | Spec contradicts itself between sections, or numbers/names don't match | Must fix — implementer will hit this and guess wrong | +| **GAP** | Behavior undefined for a scenario that will occur in production | Must define before implementation | +| **UNVERIFIABLE** | Acceptance criterion cannot be tested as written | Must rewrite to be testable | +| **DRIFT** | Spec claims something about existing code that is wrong or outdated | Must correct to match reality | +| **WARNING** | Ambiguity that could lead to misinterpretation, but has a reasonable default | Should clarify | +| **NOTE** | Observation, not a problem — something the implementer should be aware of | Informational | + +## Severity Examples + +Use these to calibrate your judgment: + + + +CRITICAL — spec contradicts codebase + +| # | Severity | Section | Finding | Evidence | Fix | +|---|----------|---------|---------|----------|-----| +| 1 | CRITICAL | 3.2 | Spec claims `BrowserTool` extends `BaseTool` with a `screenshot()` method, but `BaseTool` has no such method — it was removed in commit `a3f12bc` | `src/tools/base-tool.ts:45` — class has `execute()` and `validate()` only | Update spec to use `captureScreenshot()` from `ScreenshotMixin` (line 12 of `src/tools/mixins.ts`) | + + + + +GAP — undefined failure path + +| # | Scenario | What's missing | Suggested behavior | +|---|----------|---------------|-------------------| +| 1 | User closes browser tab while `browser_click` is in-flight | No defined behavior for mid-operation tab closure. Section 3.4 only covers timeout and element-not-found. | Should emit `tool:error` with `BrowserTabClosed` error, cancel pending operation, and update tool widget to "Tab closed" state. | + + + + +WARNING — ambiguity + +| # | Section | Ambiguity | Suggested clarification | +|---|---------|-----------|------------------------| +| 1 | 3.3 | "Results should be returned promptly" — no threshold defined. Does this mean <100ms? <1s? <5s? | Specify: "Results must be returned within 500ms. If the operation exceeds 500ms, emit a `tool:progress` event at 500ms intervals." | + + + + +--- + +## Step 0: Read the Spec and Check for Prior Audits + +Read the spec document at `$1`. If the spec document doesn't exist, output "Spec document not found at `$1`" and stop. + +Derive the audit output path: `reviews/audit-{spec-base-name}.md`. Check if a prior audit exists at that path. If it does, read it — increment the pass number and verify whether previous critical/inconsistency findings have been addressed. + +Extract and verify from the spec: + +1. **Companion plan path** — usually referenced in the header +2. **Feature scope** — what's being specified +3. **Section count and structure** — list all major sections +4. **Acceptance criteria count** — total ACs with priority breakdown +5. **Out-of-scope items** — what's explicitly excluded + +If a companion plan is referenced, read it completely. The plan is the **source of truth for implementation strategy**; the spec is the **source of truth for behavior**. They must not contradict. + +--- + +## Step 1: Codebase Ground-Truth + +The spec may make claims about existing code (current tool count, existing APIs, file paths, transport mechanisms, data structures). Every such claim must be verified. + +**For each claim about existing code:** + +1. Find the actual source file +2. Read the relevant section +3. Compare against the spec's claim +4. Flag any discrepancy as **DRIFT** + +**Minimum files to verify** (adjust based on the spec's scope): + +- Every file path mentioned in the spec +- Every existing API/function/type referenced +- Every "current state" or "baseline" claim +- Every tool/command name claimed to already exist + +**If a referenced file doesn't exist**: flag as **DRIFT** — the spec claims something about a file that was moved, renamed, or deleted. Search for the functionality elsewhere and note the correct path. + +**If the companion plan doesn't exist or isn't referenced**: proceed without plan cross-validation (Step 2). Note in the audit that plan alignment was not verified and mark the Plan Alignment grade as N/A. + +**Stop when you can answer:** + +- Does the spec accurately describe the current codebase state? +- Are all file paths, function names, and API surfaces correct? + +--- + +## Step 2: Plan-Spec Cross-Validation + +Compare the spec against its companion plan: + +### 2.1 Coverage Check + +- Every phase in the plan should map to at least one AC in the spec +- Every file the plan creates/modifies should have behavior defined in the spec +- Every edge case the plan addresses should have expected behavior in the spec + +**Flag as GAP**: Any plan phase with no corresponding spec coverage. +**Flag as CRITICAL**: Any spec AC that contradicts the plan's stated approach. + +### 2.2 Numerical Consistency + +- Tool counts: Does the spec's tool inventory match the plan's tool tables? +- File counts: Do the files-to-create/modify tables match? +- Timeout values: Do spec-stated timeouts match plan-stated timeouts? +- Size limits: Do transport limits, payload caps, etc. match? + +**Flag as INCONSISTENCY**: Any numerical mismatch. + +### 2.3 Naming Consistency + +- Tool names in spec must exactly match tool names in plan +- Schema names, type names, field names must be identical +- File paths must match +- State names (enum values, state machine states) must match + +**Flag as INCONSISTENCY**: Any naming mismatch. + +--- + +## Step 3: Internal Consistency Audit + +Check the spec against itself. Read each section and cross-reference against every other section. + +### 3.1 Acceptance Criteria vs Expected Behavior + +For every AC in Section 4 (or equivalent): + +- Is the behavior it tests actually defined in Section 3 (Expected Behavior)? +- Can you write a test for this AC using ONLY the information in the spec? If not, what's missing? +- Does the AC contradict any stated behavior? + +**Flag as UNVERIFIABLE**: ACs that can't be tested as written. +**Flag as INCONSISTENCY**: ACs that contradict stated behavior. + +### 3.2 Error Taxonomy vs Expected Behavior + +- Every error in the error taxonomy should be reachable from a scenario described in the spec +- Every failure mode in expected behavior should have a corresponding error +- Error messages should match between sections + +**Flag as GAP**: Errors with no trigger scenario. Failure modes with no error message. + +### 3.3 Data Contracts vs Usage + +- Every field in a data contract should be referenced somewhere in the spec (behavior, AC, or error) +- No "orphan" fields that exist in the contract but are never mentioned +- Types must be consistent (if epoch is `number` in one place, it can't be `string` elsewhere) + +**Flag as WARNING**: Orphan fields or type mismatches. + +### 3.4 Constraints vs Everything Else + +- Every constraint should be enforceable — there must be a mechanism described somewhere +- No AC should violate a stated constraint +- No expected behavior should violate a stated constraint + +**Flag as CRITICAL**: ACs or behavior that violate constraints. + +### 3.5 Out-of-Scope vs In-Scope + +- Nothing in the expected behavior or ACs should implement an out-of-scope item +- Nothing in the out-of-scope section should be required by an in-scope AC + +**Flag as CRITICAL**: Scope contradictions. + +--- + +## Step 4: Completeness Audit + +### 4.1 Happy Path Coverage + +For every use case in the spec: + +- Is the happy path fully specified from start to finish? +- Are all intermediate states defined? +- Is the final state verifiable? + +**Flag as GAP**: Incomplete happy paths. + +### 4.2 Failure Path Coverage + +For every use case: + +- What happens if step N fails? Is recovery defined for every step? +- What happens if the user interrupts the flow midway? +- What happens on timeout, disconnect, or crash at each step? + +**Flag as GAP**: Undefined failure paths. + +### 4.3 Boundary Conditions + +- Empty inputs (empty string, empty array, null, undefined, zero) +- Maximum inputs (largest payload, most elements, longest string) +- Concurrent operations (two tools at once, two sessions, rapid-fire calls) +- State transitions (what happens in each state if an unexpected event occurs?) + +**Flag as GAP**: Undefined boundary behavior. + +### 4.4 Cross-Feature Interactions + +- How does this feature interact with existing features? +- Does the spec address side effects on other systems (e.g., does browser snapshot affect terminal, file system, chat state)? +- Does the spec account for the feature running alongside other active features? + +**Flag as GAP**: Unaddressed cross-feature interactions. + +--- + +## Step 5: Testability Audit + +For every acceptance criterion: + +1. **Input**: What exactly do you provide? +2. **Action**: What exactly do you do? +3. **Expected output**: What exactly do you verify? + +If any of these three is ambiguous, the AC is unverifiable. + +Additionally check: + +- Are performance targets measurable with the tools described? +- Are success metrics collectible without additional instrumentation not in scope? +- Can the non-functional requirements be tested before production? + +**Flag as UNVERIFIABLE**: ACs or NFRs that can't be verified with the described approach. + +--- + +## Step 6: Write Audit Report + +Derive the output filename from `$1`. Example: `docs/specs/agent-browser-integration-spec.md` → `reviews/audit-agent-browser-integration-spec.md`. + +Write the audit to `reviews/audit-{spec-base-name}.md`: + +```markdown +# Spec Audit: [Spec Title] + +**Date**: [current date] +**Spec Document**: $1 +**Companion Plan**: [path if exists] +**Branch**: [branch name] +**Pass**: [N] (increment if previous audit exists, otherwise 1) + +## Spec Summary + +[2-3 sentences: What the spec defines, its scope, and overall completeness] + +## Audit Scope + +| Check | Files read | Claims verified | +| --------------------- | -------------------------- | ----------------------- | +| Codebase ground-truth | [N files] | [N claims] | +| Plan cross-validation | [yes/no] | [N checks] | +| Internal consistency | [N sections cross-checked] | [N cross-references] | +| Completeness | [N use cases, N ACs] | [N boundary conditions] | +| Testability | [N ACs evaluated] | [N verified testable] | + +## Verdict: [APPROVED / APPROVED WITH CHANGES / NEEDS REWORK] + +[1-2 sentence justification] + +- **APPROVED**: Zero critical, zero inconsistencies, zero gaps. Warnings and notes are acceptable. +- **APPROVED WITH CHANGES**: Zero critical. Has inconsistencies, gaps, or unverifiable ACs that are minor and have clear fixes listed below. +- **NEEDS REWORK**: Has critical findings, or too many gaps/inconsistencies to fix inline. + +**Grading:** + +| Dimension | Grade | Notes | +| ---------------------------------------- | ------------ | ----- | +| Correctness (claims match codebase) | [A-F] | | +| Plan alignment (spec covers plan fully) | [A-F or N/A] | | +| Internal consistency (no contradictions) | [A-F] | | +| Completeness (no gaps in behavior) | [A-F] | | +| Testability (all ACs are verifiable) | [A-F] | | +| Error coverage (all failures handled) | [A-F] | | + +## Findings + +### Critical (Must fix before implementation) + +| # | Severity | Section | Finding | Evidence | Fix | +| --- | -------- | ------- | -------------- | -------------------------------- | ------------ | +| 1 | CRITICAL | 3.2 | [What's wrong] | [File:line or section cross-ref] | [How to fix] | + +### Inconsistencies (Must fix — implementer will guess wrong) + +| # | Section A | Section B | Mismatch | Fix | +| --- | --------- | ---------- | -------------------- | ------------------ | +| 1 | 3.1 | Appendix A | [What doesn't match] | [Which is correct] | + +### Gaps (Behavior undefined for production scenario) + +| # | Scenario | What's missing | Suggested behavior | +| --- | ------------------------- | --------------------- | --------------------- | +| 1 | [When X happens during Y] | [No defined behavior] | [Proposed resolution] | + +### Unverifiable ACs (Must rewrite) + +| # | AC | Problem | Suggested rewrite | +| --- | ---- | ------------------------ | ------------------ | +| 1 | AC-N | [Why it can't be tested] | [Testable version] | + +### Drift (Spec claims don't match codebase) + +| # | Spec claim | Actual (from codebase) | File | Fix | +| --- | ------------------- | ---------------------- | -------------- | ----------- | +| 1 | "13 existing tools" | [actual count] | `path/to/file` | Update spec | + +### Warnings (Ambiguity that should be clarified) + +| # | Section | Ambiguity | Suggested clarification | +| --- | ------- | ---------------- | ----------------------- | +| 1 | 3.3 | [What's unclear] | [How to clarify] | + +### Notes (Informational — no action required) + +- [Observation 1] +- [Observation 2] + +## Codebase Verification Log + +| Spec claim | File checked | Verified? | Notes | +| ----------------- | ------------------- | --------- | --------------------- | +| [Claim from spec] | `path/to/file:line` | Yes/No | [Details if mismatch] | + +## Plan-Spec Alignment + +| Plan phase | Spec coverage | Gap? | +| ------------------- | ----------------- | ------------------ | +| Phase 1: Runtime | Section 3.1, AC-1 | No | +| Phase 2: MCP Server | Section 3.4, AC-5 | Partial — [detail] | + +## Required Before Implementation + +- [ ] Fix Critical #1: [description] +- [ ] Fix Inconsistency #1: [description] +- [ ] Define Gap #1: [description] +- [ ] Rewrite AC-N: [description] + +## Verdict Details + +### Correctness: [PASS / CONCERNS] + +[Details — are all claims about existing code accurate?] + +### Plan Alignment: [PASS / CONCERNS / N/A] + +[Details — does the spec fully cover the plan?] + +### Internal Consistency: [PASS / CONCERNS] + +[Details — does the spec contradict itself?] + +### Completeness: [PASS / CONCERNS] + +[Details — are all production scenarios defined?] + +### Testability: [PASS / CONCERNS] + +[Details — can every AC be verified?] + +### Error Coverage: [PASS / CONCERNS] + +[Details — is every failure mode handled?] +``` + +--- + +## Step 7: Output + +After writing file: + +1. Print: `Audit written to reviews/audit-{spec-base-name}.md` +2. Print the **Verdict**, **Grading table**, and **Required Before Implementation** sections +3. Print total finding counts: + +``` +Findings: X critical, X inconsistencies, X gaps, X unverifiable, X drift, X warnings, X notes +``` + +Read `$1` now. Do not explain what you're about to do — start the audit immediately. diff --git a/.claude/commands/changelog.md b/.claude/commands/changelog.md new file mode 100644 index 000000000..93a7c40e1 --- /dev/null +++ b/.claude/commands/changelog.md @@ -0,0 +1,155 @@ +--- +description: Generate a marketing-quality changelog for a release version +argument-hint: [pr-number] +allowed-tools: Bash(gh:*), Bash(git diff:*), Bash(git log:*), Bash(git branch:*), Bash(git status:*), Read, Write, Edit, Glob, Grep +--- + +# Changelog Generator + +Generate a changelog for Orbit release `$1`, written for the marketing site. If a PR number `$2` is provided, pull its metadata from GitHub. + +## Validation + +If `$1` is empty, output: + +``` +Usage: /changelog [pr-number] +Examples: + /changelog v0.0.7 + /changelog v0.0.7 74 +``` + +Stop execution. + +## Step 0: Resolve Version + +The version is `$1`. Determine whether it's a version or a PR number: + +- **Pure integer** (e.g., `73`, `104`) → This is a PR number, not a version. Infer the version from the current branch name and use `$1` as the PR number. +- **Contains dots** (e.g., `0.0.7`, `v0.0.7`) → This is a version. Prepend `v` if missing. +- **Starts with `v`** (e.g., `v0.0.7`) → This is a version. Use as-is. + +To get the branch name for fallback: + +```bash +git branch --show-current +``` + +--- + +## Step 1: Gather Context + +Collect all raw material. Run these commands: + +```bash +git branch --show-current +git log main..HEAD --oneline +git diff main --stat +git diff main --name-only +git status --short +``` + +If a PR number is available (either `$2`, or `$1` if it was a bare number): + +```bash +gh pr view --json title,body,commits,files +``` + +Check if a changelog already exists by reading `docs/changelog-.md`. If it exists, you are **updating** — preserve existing content and add new entries. Do not rewrite what's already there. + +If the existing changelog already covers all committed and uncommitted changes, print `Changelog is up to date — nothing new to add.` and stop. + +Read `docs/changelog-v0.0.6.md` as the **tone and structure reference**. Match its style exactly. + +--- + +## Step 2: Categorize Changes + +Group everything into user-facing categories. Use your judgment — not every commit deserves a section. + +| Category | What goes here | +| ---------------- | ------------------------------------------------------------------------- | +| **New features** | Genuinely new capabilities a user would notice | +| **Improvements** | Meaningful enhancements to existing features | +| **Bug fixes** | Things that were broken and are now fixed | +| **In-progress** | Uncommitted or WIP work that's substantial enough to mention | +| **Internal** | Only if genuinely interesting (new infra, major cleanup). Skip if boring. | + +**Discard:** + +- Routine refactors that don't change behavior +- File renames, import reordering, lint fixes +- Test additions (unless they represent a new testing capability) +- Documentation changes (unless they're user-facing docs) + +--- + +## Step 3: Write the Changelog + +Create or update `docs/changelog-.md`. + +### Format + +```markdown +# Orbit + +__ + +--- + +## + +### + +<1-2 paragraphs describing what the user gets. Not what files changed.> + +--- + +## Bug Fixes + +- **** — + +--- + +## _(in progress)_ + + + +--- + +## Internal + +- +``` + +### Writing Rules + +These are non-negotiable: + +1. **Lead with what users get.** "The branch picker now always shows search and branch creation" — not "Removed conditional Select/Popover logic." + +2. **No internal naming.** Never mention token names (`orbit-100`, `surface-primary`), store names (`useGitStore`), CSS variable names (`--color-border-subtle`), file paths, or function names. + +3. **No design system implementation details** unless the design system itself is the feature. "Consistent colors across light and dark themes" is fine. "Migrated from Radix 12-step scales to a 10-step primitive scale with three semantic layers" is not. + +4. **Don't elevate routine cleanup.** Removing dead code, fixing lint, updating dependencies — these are not features. If they're significant enough (e.g., 900+ lines of dead CSS removed), fold them into the relevant section as a supporting detail, not a headline. + +5. **Bug fixes are one-liners.** State what broke and that it's fixed. Don't explain the implementation. + +6. **In-progress features get one paragraph.** What it will do, what's built so far at a high level, and that it ships later. + +7. **Internal section uses one-liners.** If there's nothing interesting, omit the section entirely. + +8. **Don't sound impressed by your own work.** No "comprehensive", "robust", "elegant", "powerful". Just say what it does. + +9. **Match the reference.** Read `docs/changelog-v0.0.6.md` before writing. Your output should feel like the same person wrote both. + +--- + +## Step 4: Output + +After writing the file: + +1. Print: `Changelog written to docs/changelog-.md` +2. Print the full changelog content +3. If updating an existing file, note what was added diff --git a/.claude/commands/implement.md b/.claude/commands/implement.md new file mode 100644 index 000000000..05afa3771 --- /dev/null +++ b/.claude/commands/implement.md @@ -0,0 +1,188 @@ +--- +description: Implement a feature from an approved plan and spec sheet, phase by phase with AC verification +argument-hint: +allowed-tools: Read, Edit, Write, Glob, Grep, Bash, Agent, AskUserQuestion, LSP +--- + +# Feature Implementation + +You are a senior implementation engineer working in a production TypeScript/React/Rust/Tauri monorepo. You implement exactly what was planned and specified — no shortcuts, no gold-plating, no improvisation. + + +$1 +$2 + + +If either path is empty, output the usage and stop: + +``` +Usage: /implement +``` + +--- + +## Principles + +These govern every decision. When in doubt, return here. + +1. **Spec = WHAT.** Behavior, ACs, data contracts, error messages. If the spec doesn't address it, ask the user. +2. **Plan = HOW.** Files, phases, architecture. Follow its structure. +3. **Codebase wins conflicts.** When plan examples contradict actual codebase conventions, match the codebase. +4. **ACs are contracts.** Never claim an AC passes without running the verification command. +5. **No gold-plating.** If it's not in the plan or spec, don't build it. +6. **Ask before guessing** on ambiguities that affect architecture or behavior. + +--- + +## AC Status Definitions + +Use these consistently in phase reports and the final report: + +- **PASS**: Verified by running a command or test +- **PENDING [Phase N]**: Cannot verify yet — depends on a later phase +- **MANUAL**: Requires hardware, UI interaction, or manual testing +- **PARTIAL**: Implemented but needs manual verification (explain what to check) +- **BLOCKED**: Cannot verify without unavailable infrastructure (explain what's needed) +- **MISSING**: Not implemented (explain why) + +--- + +## Preparation (silent — do not output analysis or status) + +Read all of these before writing any code: + +1. **Plan** — extract phases, files, architecture, inter-phase dependencies +2. **Spec** — extract ACs with priorities, data contracts, error taxonomy, constraints +3. **Relevant CLAUDE.md files** — root + sub-level for every area you'll touch +4. **Every existing file** the plan references — note actual signatures, types, patterns +5. **Neighboring files** for every new file — learn local conventions, check barrel exports + +Then map which ACs become satisfiable after which plan phase. + +--- + +## Execution + +### 1. Output the implementation sequence, then immediately start + +Do NOT wait for user confirmation. Output this once, then begin Phase 1: + +``` +## Implementation Sequence + +**Feature**: [name] +**Phases**: [N] | **ACs**: [total] (P0: n, P1: n, P2: n, P3: n) + +Phase 1: [name] → [files] → ACs: [list] +Phase 2: [name] → [files] → ACs: [list] +... +``` + +### 2. Execute every plan phase continuously + +Work through every plan phase in order. Do NOT stop between phases to ask permission — just proceed. Only stop for genuine blockers you cannot resolve (spec ambiguity, missing section, compile error after two fix attempts). + +**For each plan phase:** + +**Announce:** + +``` +## Phase [N]: [Name] +[files being created/modified] | ACs: [targeted] +``` + +**Before implementing**, identify which types, functions, or files from previous phases this phase depends on. + +**Implement** following all code quality rules from CLAUDE.md. Additionally: + +- New files: read a neighbor first for conventions; add to barrel exports (index.ts / mod.rs) +- Modified files: re-read first (may have changed from earlier phases); make minimum changes +- Use exact error message templates from the spec's Error Taxonomy +- Match the spec's data contract types exactly + +**Validate:** + +1. Run `bun run typecheck` (if TypeScript changed) and/or `cargo check` (if Rust changed) — fix errors before continuing +2. Run `bun run lint` if you created new files or made structural changes +3. For each targeted AC: run the verification command if one exists. Never claim PASS from reading code alone when a command could verify it. + +**Report** (3-5 lines, then continue to next phase): + + +Phase 2 complete. Created: `src/hooks/browser/use-browser-nav.ts`. Modified: `src/hooks/browser/index.ts`, `src/stores/browser/browser-store.ts`. +AC-4 (P0): PASS — `bun run typecheck` clean, hook returns `BrowserNavState`. AC-5 (P1): PASS — store triggers re-render (verified via test). AC-7 (P1): PENDING [Phase 4] — needs toolbar integration. +Plan drift: Plan referenced `useBrowserStore.getState().activeTab` but store uses `.tabs.active`. Adapted to match codebase. + + +--- + +## Final Validation + +After all plan phases complete: + +1. **Run full quality gates** — fix any failures: + + ``` + bun run check # typecheck + lint + tests + cargo clippy # if Rust touched + cargo test # if Rust touched + ``` + +2. **AC sweep** — walk every AC in the spec and assign a status (see AC Status Definitions above). + +3. **Output final report:** + +``` +## Implementation Complete + +**Feature**: [name] | **Branch**: [branch] | **Date**: [date] + +### Files Changed + +| File | Action | Phase | +|------|--------|-------| +| `path` | Created/Modified | Phase N | + +### Acceptance Criteria + +| AC | Pri | Status | Verification | +|----|-----|--------|--------------| +| AC-1 | P0 | PASS | `bun run typecheck` clean + test passes | + +**Summary**: [N]/[total] ACs satisfied. All P0s passing. + +### Quality Gates + +| Check | Status | +|-------|--------| +| TypeScript (`bun run typecheck`) | PASS/FAIL | +| ESLint (`bun run lint`) | PASS/FAIL | +| Frontend tests (`bun run test`) | PASS/FAIL | +| Clippy (`cargo clippy`) | PASS/FAIL or N/A | +| Rust tests (`cargo test`) | PASS/FAIL or N/A | + +### Issues & Deferred Items + +[Plan drift, skipped P3 items, observations worth noting] + +### Next Steps + +1. [e.g., manual verification for AC-12, AC-15] +2. [e.g., run /audit-spec to verify spec accuracy] +3. [e.g., create PR targeting main] +``` + +--- + +## Error Recovery + +- **Compile errors**: Check actual codebase types, not what the plan assumed. Fix simply. Note drift in phase report. +- **Plan drift**: File, function, or API doesn't exist where plan says? Search for the real location, adapt, note it. +- **Ambiguity**: Check spec's Open Questions → neighboring code patterns → ask the user. +- **Missing spec sections**: Flag the gap to the user. Do not fill it yourself. +- **Scope creep**: Do not implement it. Note under "Observations" for the user to decide. +- **Broken existing tests**: Fix them. Do not delete unless the plan explicitly says to. + +--- + +Begin by reading the plan and spec documents. diff --git a/.claude/commands/prompt.md b/.claude/commands/prompt.md new file mode 100644 index 000000000..3e9a25d44 --- /dev/null +++ b/.claude/commands/prompt.md @@ -0,0 +1,785 @@ +--- +description: Evaluate a prompt against Anthropic's prompt engineering framework, score it, and rewrite it to near-perfect quality +argument-hint: '' +allowed-tools: Read, Glob, Grep, Write, Bash(wc:*), Bash(date:*) +--- + +# Prompt Evaluator & Improver + +You are a world-class prompt engineer trained on Anthropic's official prompt engineering curriculum. Your job is to take any prompt, ruthlessly evaluate it against every known technique, score it, and rewrite it to near-perfect quality. + +**Input**: $1 + +## Validation + +If `$1` is empty, output: + +``` +Usage: /prompt +Example: /prompt "Summarize this article for me" +Example: /prompt prompts/my-system-prompt.txt +``` + +Stop execution. + +## Step 0: Acquire the Prompt + +1. If `$1` looks like a file path (contains `/` or `.txt` or `.md`), read the file and use its contents as the prompt to evaluate. +2. Otherwise, treat `$1` directly as the prompt text. +3. Identify whether the input contains: a system prompt, a user prompt, assistant prefill, or a combination. Parse each part separately. + +--- + +## Step 1: Deep Analysis + +Analyze the prompt against ALL 12 dimensions below. For each dimension, assign a score from 0-10 and note specific findings. + +--- + +### THE 12 DIMENSIONS OF PROMPT QUALITY + +These dimensions are derived from Anthropic's official prompt engineering tutorial (Chapters 1-9 + Appendices). Every rule below is a direct teaching from that curriculum. + +--- + +#### Dimension 1: CLARITY & DIRECTNESS (Chapter 2) + +**The Golden Rule**: Show the prompt to a colleague — if they're confused, Claude is confused. + +Evaluate: + +- [ ] Is every instruction explicit and unambiguous? +- [ ] Are there vague words that could be interpreted multiple ways? ("good", "nice", "appropriate", "relevant") +- [ ] Does the prompt say exactly what it wants, or does it hope Claude will guess? +- [ ] If the user wants no preamble, does the prompt say "Skip the preamble"? +- [ ] If the user wants a definitive answer, does the prompt force a choice? ("If you absolutely had to pick one...") +- [ ] If the user wants a specific length, is it stated explicitly? ("Write at least 800 words") +- [ ] If the user wants ONLY a specific output (just a name, just a number), does it say "Reply with ONLY X and nothing else"? + +**Claude has ZERO context** beyond what you literally tell it. It is not a mind reader. If you want something, ask for it directly. Don't be indirect or hope Claude infers your intent. + +**Scoring guide:** + +- 0-3: Vague, indirect, relies on Claude guessing intent +- 4-6: Mostly clear but has ambiguous words or missing specifics +- 7-9: Clear and direct with minor improvements possible +- 10: Every instruction is unambiguous; a human colleague could follow it perfectly + +--- + +#### Dimension 2: ROLE ASSIGNMENT (Chapter 3) + +**Principle**: Priming Claude with a role improves performance in writing, coding, math, logic, and domain expertise. It changes tone, style, and accuracy. + +Evaluate: + +- [ ] Does the prompt assign a specific, relevant role? (system prompt or user message) +- [ ] Is the role detailed enough? ("You are a logic bot" vs just hoping Claude reasons well) +- [ ] Does the role include relevant expertise context? ("You are a senior tax attorney specializing in Section 83(b) elections") +- [ ] Is the intended audience specified? ("You are a doctor explaining to a patient" vs "You are a doctor explaining to medical students") +- [ ] For math/logic tasks: would a role like "You are a precise mathematical reasoner" help? +- [ ] For creative tasks: would a tone-setting role help? ("You are a witty, warm children's book author") + +**Scoring guide:** + +- 0-3: No role assigned; Claude operates as generic assistant +- 4-6: Role exists but is too generic or missing expertise details +- 7-9: Specific role with good context, minor refinements possible +- 10: Role perfectly calibrated to the task with expertise, audience, and tone specified + +--- + +#### Dimension 3: DATA-INSTRUCTION SEPARATION (Chapter 4) + +**Principle**: Variable data MUST be visually and structurally separated from instructions using XML tags. Without separation, Claude confuses data for instructions. + +Evaluate: + +- [ ] Is ALL variable/dynamic content wrapped in XML tags? (``, ``, ``, ``) +- [ ] Are tag names semantically meaningful? (`` not ``) +- [ ] Can Claude clearly distinguish where data begins and instructions end? +- [ ] Are there multiple data inputs that should each have their own tags? +- [ ] Would the prompt still work correctly if the data contained instructions? (injection resistance) +- [ ] Is the prompt a template with proper `{variable}` placeholders for substitution? +- [ ] For programmatic use: are variables clearly delineated so templates can be reused with different data? + +**Important note from Anthropic**: Claude was SPECIFICALLY TRAINED to recognize XML tags as prompt organizers. There are no "magic" tag names — use whatever makes semantic sense. But DO use XML tags, not other delimiters. + +**Template pattern**: For repeatable tasks, build prompt templates with `{variable}` placeholders. Users fill in variables without seeing the full prompt. Example: `"Classify this email: {email}"` where `{email}` is substituted at runtime. + +**Scoring guide:** + +- 0-3: No separation; data and instructions are interleaved +- 4-6: Some separation but inconsistent or using non-XML delimiters +- 7-9: Good XML tag usage with minor gaps +- 10: Every variable input is cleanly wrapped in semantic XML tags; injection-resistant + +--- + +#### Dimension 4: OUTPUT FORMAT SPECIFICATION (Chapter 5) + +**Principle**: Claude can output in any format — but you must specify it. XML tags, JSON, markdown, lists, tables, specific structures. + +Evaluate: + +- [ ] Is the desired output format explicitly specified? +- [ ] Are XML output tags requested for programmatic extraction? (`"Put your answer in tags"`) +- [ ] Is JSON format requested where appropriate? (structured data, API responses) +- [ ] Is the output length specified where relevant? ("2-3 sentences", "one paragraph", "at least 500 words") +- [ ] Are unwanted elements explicitly excluded? ("Do not include explanations", "Skip the preamble") +- [ ] Could the output be more parseable with better formatting instructions? + +**Scoring guide:** + +- 0-3: No format specified; Claude uses whatever it wants +- 4-6: Partial format spec; some ambiguity about structure +- 7-9: Clear format with XML/JSON tags; minor refinements possible +- 10: Format perfectly specified; output is trivially parseable by code or perfectly readable by humans + +--- + +#### Dimension 5: PREFILLING / ASSISTANT PRIMING (Chapter 5) + +**Principle**: Putting text in the assistant turn forces Claude to continue from that point. This eliminates preamble, forces specific formats, and steers content direction. + +Evaluate: + +- [ ] Would an assistant prefill improve the output? (eliminating preamble, forcing format) +- [ ] If XML output tags are requested, is the opening tag prefilled in the assistant turn? +- [ ] If JSON is requested, is `{` prefilled? +- [ ] Could prefilling steer Claude toward a specific framing or perspective? +- [ ] Is `stop_sequences` used with the closing tag to save tokens? + +**Scoring guide:** + +- 0-3: No prefill used where it would clearly help +- 4-6: Prefill opportunity exists but isn't used, or is used suboptimally +- 7-9: Good prefill usage; minor opportunities missed +- 10: Prefill perfectly eliminates preamble and/or forces exact format; stop_sequences optimized + +--- + +#### Dimension 6: CHAIN OF THOUGHT / PRECOGNITION (Chapter 6) + +**Principle**: Giving Claude time to think OUT LOUD (not silently) improves accuracy. Thinking only counts when it's written. Claude must show its reasoning before answering. + +Evaluate: + +- [ ] Is the task complex enough to benefit from step-by-step thinking? +- [ ] Does the prompt instruct Claude to think before answering? +- [ ] Is the thinking structured in tags? (``, ``, ``, ``) +- [ ] Are specific thinking steps outlined? ("First, list pros and cons. Then, evaluate each...") +- [ ] Is the thinking placed BEFORE the final answer in the prompt's instruction order? +- [ ] For classification/analysis: does the prompt ask Claude to argue both sides before deciding? + +**Order sensitivity warning (from Anthropic)**: Claude has a bias toward the SECOND of two options. When asking "positive or negative?", the order affects the answer. Place the more likely correct answer second. + +**Scoring guide:** + +- 0-3: Complex task with no thinking instruction; Claude likely to make errors +- 4-6: Thinking instruction exists but isn't structured or doesn't use tags +- 7-9: Structured thinking with tags; minor ordering or specificity improvements +- 10: Thinking perfectly structured, tagged, ordered, with specific steps before the final answer + +--- + +#### Dimension 7: FEW-SHOT EXAMPLES (Chapter 7) + +**Anthropic's official position**: Examples are the SINGLE MOST EFFECTIVE TOOL for getting desired behavior. + +Evaluate: + +- [ ] Does the prompt include concrete examples of ideal input → output? +- [ ] Are examples wrapped in `` XML tags? +- [ ] Do examples cover the EXACT output format desired? (not just content — format too) +- [ ] Are edge cases demonstrated in examples? +- [ ] Are there enough examples? (2-3 is good; more is better for complex tasks) +- [ ] For classification: does each category have at least one example? +- [ ] For extraction: do examples show the exact extraction format? +- [ ] Do examples demonstrate the desired tone/style? + +**Two styles of few-shot (both valid):** + +1. **In-prompt examples**: Wrap in `` tags within the user message. Best for format/extraction/classification. +2. **Multi-turn examples**: Use actual `user`/`assistant` message pairs in the messages array. Best for conversational tone and style. E.g., showing a Q&A pair like `Q: Is the tooth fairy real? / A: Of course, sweetie...` then asking the real question. + +**Scoring guide:** + +- 0-3: No examples where they would clearly help +- 4-6: Has examples but they're incomplete, don't cover format, or miss edge cases +- 7-9: Good examples covering format and content; minor gaps in edge case coverage +- 10: Comprehensive examples with edge cases, exact format, and ideal quality; Claude can extrapolate perfectly + +--- + +#### Dimension 8: HALLUCINATION PREVENTION (Chapter 8) + +**Principle**: Claude fabricates when it tries to be helpful about things it doesn't know. Three defenses: give an "out", require evidence first, control temperature. + +Evaluate: + +- [ ] Does the prompt give Claude permission to say "I don't know"? ("Only answer if you know with certainty") +- [ ] For document-based tasks: does it require Claude to extract evidence/quotes BEFORE answering? +- [ ] Is there a TWO-STEP scratchpad? ("In tags, (1) pull the most relevant quotes, (2) evaluate whether those quotes actually answer the question. Then answer.") +- [ ] For factual questions: is Claude told to decline rather than guess? +- [ ] For long documents: is the question placed at the BOTTOM (after the text, not before)? This is critical — placing it at the top causes Claude to latch onto "distractor information" (similar-but-wrong data) in the document. +- [ ] Is temperature set appropriately? (0 for factual accuracy, higher for creativity) +- [ ] **Distractor defense**: When documents contain data SIMILAR to what's asked but not exactly matching (e.g., subscriber count from 2021 when the question asks about 2020), does the evidence-first step force Claude to notice the mismatch? + +**Scoring guide:** + +- 0-3: No hallucination guards; Claude will fabricate freely +- 4-6: Some guards but missing the "out" clause or evidence-first step +- 7-9: Good guards; minor improvements to evidence extraction or question placement +- 10: Multiple hallucination defenses layered; evidence-first + "out" clause + proper placement + +--- + +#### Dimension 9: PROMPT STRUCTURE & ORDERING (Chapter 9) + +**Anthropic's 10-Element Framework** — the recommended ordering for production prompts: + +``` +1. user role (always first) +2. Task context — role, goals, overview (EARLY in prompt) +3. Tone context — voice, style, audience (EARLY) +4. Detailed rules & task description (AFTER context) +5. Examples in tags (MIDDLE — single most effective tool) +6. Input data in XML tags (FLEXIBLE position) +7. Immediate task request / user's question (NEAR THE END — recency bias) +8. Chain of thought instruction (AFTER task, BEFORE output format) +9. Output formatting specification (END) +10. Prefill in assistant turn (ASSISTANT message) +``` + +**Critical ordering rules from Anthropic:** + +- Task context goes EARLY +- User's actual question goes NEAR THE BOTTOM (better results than top) +- Chain of thought goes right before output format +- Output format goes at the END +- Place the answer you think is more likely correct SECOND (bias toward second option) + +Evaluate: + +- [ ] Does the prompt follow the recommended element ordering? +- [ ] Is the user's question/task near the end (not buried at the top)? +- [ ] Are context and rules before examples? +- [ ] Are examples before the main task request? +- [ ] Is chain of thought instruction after the task but before output format? +- [ ] Is the overall flow logical: context → rules → examples → data → question → think → format? + +**Scoring guide:** + +- 0-3: Random ordering; question at top, context at bottom, examples scattered +- 4-6: Partially correct ordering; some elements misplaced +- 7-9: Good ordering with minor repositioning opportunities +- 10: Perfect element ordering following Anthropic's framework + +--- + +#### Dimension 10: SYSTEM PROMPT USAGE (Chapter 1) + +**Principle**: System prompts provide persistent context, instructions, and guidelines that shape ALL responses. They're separate from user messages and ideal for role, rules, and constraints. + +Evaluate: + +- [ ] Is there a system prompt? Should there be one? +- [ ] Are persistent rules/role/tone in the system prompt (not cluttering the user message)? +- [ ] Is the system prompt concise and focused? +- [ ] Does the system prompt set Claude's identity, constraints, and behavioral boundaries? +- [ ] Are task-specific instructions in the user message (not the system prompt)? + +**Scoring guide:** + +- 0-3: No system prompt where one would clearly help; or system prompt misused +- 4-6: System prompt exists but mixes concerns or is too generic +- 7-9: Clean separation of system vs user concerns; minor improvements +- 10: System prompt perfectly handles identity/role/rules; user message handles task/data + +--- + +#### Dimension 11: PROMPT CHAINING READINESS (Appendix 10.1) + +**Principle**: Complex workflows benefit from multi-step prompt chains — generate → validate → refine. Claude can improve its own output when asked to review it. + +Evaluate: + +- [ ] Is this prompt part of a larger workflow that could benefit from chaining? +- [ ] Could the output quality improve with a "review and fix" follow-up step? +- [ ] Are intermediate outputs structured (XML/JSON) for easy chaining? +- [ ] If asking Claude to validate, does the prompt give an "out"? ("If everything is correct, return the original") + +**Scoring guide:** + +- 0-3: Monolithic prompt trying to do everything at once; should be split +- 4-6: Could benefit from chaining but is designed as single-shot +- 7-9: Well-structured for potential chaining; outputs are parseable +- 10: Perfectly designed for single-shot OR clearly part of an optimized chain + +--- + +#### Dimension 12: SMALL DETAILS & POLISH (Chapter 4, throughout) + +**Principle from Anthropic**: "Small details matter. Claude is sensitive to patterns — it's more likely to make mistakes when you make mistakes, smarter when you sound smart, sillier when you sound silly." + +Evaluate: + +- [ ] Are there typos, grammatical errors, or sloppy formatting? +- [ ] Is the prompt professional and well-structured? +- [ ] Are placeholder variables clear and consistent? +- [ ] Does the prompt avoid contradictory instructions? +- [ ] Is there unnecessary filler that could be removed? +- [ ] Does the prompt respect Claude's strengths (pattern matching, instruction following) and weaknesses (counting, math without tools)? + +**Scoring guide:** + +- 0-3: Sloppy, typos, contradictions, unclear variables +- 4-6: Mostly clean but has formatting issues or minor contradictions +- 7-9: Professional and polished; minor cleanup opportunities +- 10: Impeccable quality; every word earns its place + +--- + +## Step 2: Score & Grade + +Calculate and present: + +``` +╔══════════════════════════════════════════════════════════════╗ +║ PROMPT EVALUATION REPORT ║ +╠══════════════════════════════════════════════════════════════╣ +║ ║ +║ DIMENSION SCORE STATUS ║ +║ ───────────────────────────────────────────────────────── ║ +║ 1. Clarity & Directness X/10 [EMOJI] ║ +║ 2. Role Assignment X/10 [EMOJI] ║ +║ 3. Data-Instruction Separation X/10 [EMOJI] ║ +║ 4. Output Format Specification X/10 [EMOJI] ║ +║ 5. Prefilling / Assistant Priming X/10 [EMOJI] ║ +║ 6. Chain of Thought X/10 [EMOJI] ║ +║ 7. Few-Shot Examples X/10 [EMOJI] ║ +║ 8. Hallucination Prevention X/10 [EMOJI] ║ +║ 9. Prompt Structure & Ordering X/10 [EMOJI] ║ +║ 10. System Prompt Usage X/10 [EMOJI] ║ +║ 11. Chaining Readiness X/10 [EMOJI] ║ +║ 12. Small Details & Polish X/10 [EMOJI] ║ +║ ───────────────────────────────────────────────────────── ║ +║ TOTAL XX/120 ║ +║ GRADE [A+ to F] ║ +║ ║ +╚══════════════════════════════════════════════════════════════╝ +``` + +**Status indicators:** + +- PASS (8-10): No changes needed +- IMPROVE (5-7): Has potential, needs work +- FAIL (0-4): Critical gap, must fix + +**Grade scale:** + +- A+ (108-120): Production-grade, Anthropic-level quality +- A (96-107): Excellent, minor polish opportunities +- B (84-95): Good foundation, several improvements needed +- C (72-83): Functional but leaves significant performance on the table +- D (60-71): Below average, missing multiple critical techniques +- F (0-59): Needs fundamental restructuring + +--- + +## Step 3: Findings (What's Wrong) + +List every specific finding, ordered by impact: + +``` +### Critical Findings (Must Fix) +1. [Specific finding with explanation of why it hurts performance] + +### High-Impact Improvements +1. [Specific improvement with expected effect] + +### Nice-to-Have Polish +1. [Minor refinement] +``` + +For each finding, explain: + +- **What**: The specific issue +- **Why**: How it hurts prompt performance (reference the Anthropic technique it violates) +- **Fix**: The exact change to make + +--- + +## Step 4: Rewrite the Prompt + +Now rewrite the prompt applying EVERY applicable technique. The rewrite should: + +1. Follow the 10-Element Framework ordering (context → rules → examples → data → question → think → format) +2. Apply every technique that scored below 8 +3. Preserve the original intent perfectly — don't change WHAT the prompt does, only HOW it instructs Claude +4. Use XML tags for all data separation +5. Include examples if the original had none and they would help +6. Add hallucination guards if the task involves factual claims +7. Add chain-of-thought if the task is complex +8. Specify output format explicitly +9. Include a prefill if it would eliminate preamble or force format +10. Fix all typos, grammar, and structural issues + +**Present the rewrite in three parts:** + +### Part A: System Prompt (if applicable) + +``` +[The system prompt, or "N/A — not needed for this task"] +``` + +### Part B: User Message + +``` +[The complete user message with all improvements] +``` + +### Part C: Assistant Prefill (if applicable) + +``` +[The prefill text, or "N/A — not needed for this task"] +``` + +### Part D: Recommended Parameters + +``` +temperature: [0.0 for factual, 0.3-0.7 for creative, 1.0 for maximum creativity] +max_tokens: [appropriate value] +stop_sequences: [closing tags if applicable] +``` + +--- + +## Step 5: Re-evaluate + +Score the rewritten prompt on the same 12 dimensions. Present a comparison: + +``` +BEFORE → AFTER +───────────────────────────────────────── +Clarity: X/10 → X/10 [+N] +Role: X/10 → X/10 [+N] +Data Separation: X/10 → X/10 [+N] +Output Format: X/10 → X/10 [+N] +Prefilling: X/10 → X/10 [+N] +Chain of Thought: X/10 → X/10 [+N] +Examples: X/10 → X/10 [+N] +Hallucination: X/10 → X/10 [+N] +Structure: X/10 → X/10 [+N] +System Prompt: X/10 → X/10 [+N] +Chaining: X/10 → X/10 [+N] +Polish: X/10 → X/10 [+N] +───────────────────────────────────────── +TOTAL: XX/120 → XX/120 [+N] +GRADE: [X] → [X] +``` + +--- + +## Step 6: Tips for This Specific Prompt + +Provide 3-5 contextual tips specific to this prompt's use case. These should go beyond the generic framework and address the particular domain, task type, or workflow. + +--- + +## Policies + +- **Be ruthless but constructive.** Every score must be justified with specific evidence. Never give a 10 to be nice — a 10 means "Anthropic would use this in their own documentation." +- **Preserve intent.** The rewrite must do EXACTLY what the original prompt intended. You're improving the vehicle, not changing the destination. +- **Not every dimension applies equally.** A simple "translate this sentence" prompt doesn't need few-shot examples or chain of thought. Score based on whether the technique WOULD help, not whether it's present. If a dimension is irrelevant, score it 8 ("not needed, correctly omitted") and note "N/A — not applicable for this task type." +- **The framework is Anthropic's, not generic.** Every rule comes from Anthropic's official prompt engineering tutorial. Reference specific chapters when explaining findings. +- **No AI slop in the rewrite.** The rewritten prompt must be tight, precise, and professional. No filler words, no redundant instructions, no "please kindly." +- **Think about the end-to-end system.** Consider whether the prompt will be used once (interactive) or in a pipeline (programmatic). Optimize accordingly. +- **Start complex, then simplify.** When rewriting, start with ALL applicable elements to get it working. Then trim unnecessary ones. This is Anthropic's recommended workflow from Chapter 9. + +--- + +## REFERENCE: Claude's Known Weaknesses + +When evaluating, flag prompts that ask Claude to do things it's bad at WITHOUT providing tool support: + +| Weakness | Why | Mitigation | +| ------------------------- | ------------------------ | ------------------------------------------------------ | +| Counting syllables | Tokenization ≠ syllables | Don't rely on syllable-accurate haiku | +| Counting words | Same tokenization issue | Give explicit word targets with overshoot buffer | +| Precise arithmetic | No calculator | Provide a calculator tool or chain a verification step | +| Counting items in lists | Attention limits | Ask Claude to number items as it lists them | +| Knowing current date/time | Training cutoff | Provide date in the system prompt | + +--- + +## REFERENCE: Golden Prompt Patterns from Anthropic's Tutorial + +Use these as templates when rewriting. These are the actual patterns from Anthropic's exercises and solutions. + +### Pattern 1: CLASSIFICATION (from Ch6-7 exercises) + +The best-performing classification prompt from the tutorial: + +``` +SYSTEM: (empty) + +USER: +Please classify this email into the following categories, and do not include explanations: + + +(A) Pre-sale question +(B) Broken or defective item +(C) Billing question +(D) Other (please explain) + + +Here are a few examples of correct answer formatting: + + + +Q: How much does it cost to buy a Mixmaster4000? +A: The correct category is: A + + +Q: My Mixmaster won't turn on. +A: The correct category is: B + + +Q: Please remove me from your mailing list. +A: The correct category is: D + + + +Here is the email for you to categorize: +{email} + +ASSISTANT PREFILL: +The correct category is: +``` + +**Why it works:** Categories are explicit + wrapped in XML. Examples show exact format. Prefill forces Claude to jump straight to the answer letter. The `{email}` variable is wrapped in `` tags for clean separation. + +--- + +### Pattern 2: DOCUMENT Q&A WITH HALLUCINATION DEFENSE (from Ch8-9) + +``` +SYSTEM: +You are an expert [domain] analyst. + +USER: +Here is the material you should use to answer the question: + + +{DOCUMENT_TEXT} + + +{QUESTION} + +In tags, pull the most relevant quotes from the document and evaluate whether they directly answer the question. If the quotes don't contain the specific information needed, acknowledge that. + +Then write your answer in tags. If there is not sufficient information in the document, say "I don't have sufficient information to answer this question." + +ASSISTANT PREFILL: + +``` + +**Why it works:** Document goes FIRST, question AFTER (recency bias). The two-step scratchpad forces Claude to (1) extract evidence and (2) evaluate it — catching distractor information. The explicit "out" clause prevents fabrication. Prefill forces immediate reasoning. + +--- + +### Pattern 3: CHATBOT / ROLEPLAY (from Ch9 career coach) + +``` +SYSTEM: (empty) + +USER: +You will be acting as an AI career coach named Joe created by the company AdAstra Careers. Your goal is to give career advice to users. You will be replying to users who are on the AdAstra site and who will be confused if you don't respond in the character of Joe. + +You should maintain a friendly customer service tone. + +Here are some important rules for the interaction: +- Always stay in character, as Joe, an AI from AdAstra Careers +- If you are unsure how to respond, say "Sorry, I didn't understand that. Could you rephrase your question?" +- If someone asks something irrelevant, say, "Sorry, I am Joe and I give career advice. Do you have a career question today I can help you with?" + +Here is an example of how to respond in a standard interaction: + +Customer: Hi, how were you created and what do you do? +Joe: Hello! My name is Joe, and I was created by AdAstra Careers to give career advice. What can I help you with today? + + +Here is the conversational history (between the user and you) prior to the question. It could be empty if there is no history: + +{HISTORY} + + +Here is the user's question: + +{QUESTION} + + +How do you respond to the user's question? Think about your answer first before you respond. Put your response in tags. + +ASSISTANT PREFILL: +[Joe] +``` + +**Why it works:** Full 10-element framework. Task context → tone → rules → examples → input data → immediate question → precognition ("think first") → output format. The prefill starts in-character. Rules include explicit "out" clauses for edge cases. + +--- + +### Pattern 4: CODE REVIEW / SOCRATIC TEACHING (from Ch9 codebot) + +``` +SYSTEM: (empty) + +USER: +You are Codebot, a helpful AI assistant who finds issues with code and suggests possible improvements. + +Act as a Socratic tutor who helps the user learn. + +You will be given some code from a user. Please do the following: +1. Identify any issues in the code. Put each issue inside separate tags. +2. Invite the user to write a revised version of the code to fix the issue. + +Here's an example: + + +def calculate_circle_area(radius): + return (3.14 * radius) ** 2 + + +3.14 is being squared when it's actually only the radius that should be squared + + +That's almost right, but there's an issue related to order of operations. It may help to write out the formula for a circle and then look closely at the parentheses in your code. + + + +Here is the code you are to analyze: + +{CODE} + + +Find the relevant issues and write the Socratic tutor-style response. Do not give the user too much help! Instead, just give them guidance so they can find the correct solution themselves. + +Put each issue in tags and put your final response in tags. +``` + +**Why it works:** Role + tone + numbered steps + example showing exact format + code in XML tags + output format specified. The example demonstrates the desired level of hint (not too much, not too little). + +--- + +### Pattern 5: EXTRACTION WITH FEW-SHOT (from Ch7) + +``` +USER: +[First passage of text with names and professions] + +1. Dr. Liam Patel [NEUROSURGEON] +2. Olivia Chen [ARCHITECT] +3. Ethan Kovacs [MUSICIAN AND COMPOSER] +4. Isabella Torres [CHEF] + + +[Second passage of text with names and professions] + +1. Oliver Hamilton [CHEF] +2. Elizabeth Chen [LIBRARIAN] +3. Isabella Torres [ARTIST] +4. Marcus Jenkins [COACH] + + +[Third passage — the actual input to extract from] + +ASSISTANT PREFILL: + +``` + +**Why it works:** Zero instructions needed. The two examples ARE the instructions — Claude sees the pattern (passage → structured extraction) and extrapolates perfectly. The prefill forces immediate extraction. This is pure few-shot power: examples replace verbose instructions. + +--- + +### Pattern 6: LEGAL / RESEARCH CITATION (from Ch9) + +``` +SYSTEM: +You are an expert lawyer. + +USER: +Here is some research that's been compiled. Use it to answer a legal question from the user. + + +{RESEARCH_DOCUMENTS} + + +When citing the legal research in your answer, please use brackets containing the search index ID, followed by a period. Put these at the end of the sentence that's doing the citing. Examples of proper citation format: + + +The statute of limitations expires after 10 years for crimes like this. [3]. +However, the protection does not apply when it has been specifically waived by both parties. [5]. + + +Write a clear, concise answer to this question: +{QUESTION} + +It should be no more than a couple of paragraphs. If there is not sufficient information in the compiled research, write "Sorry, I do not have sufficient information at hand to answer this question." + +Before you answer, pull out the most relevant quotes from the research in tags. + +Put your response in tags. + +ASSISTANT PREFILL: + +``` + +**Why it works:** Citation format shown via examples (not described — shown). Evidence-first via ``. Explicit "out" clause. Question near the bottom. Document before question. Two output sections (quotes then answer). + +--- + +## REFERENCE: The Iterative Prompt Development Workflow + +From Chapter 9 — Anthropic's recommended process: + +``` +1. START with all 10 elements (even if verbose) + ↓ +2. TEST — does it produce the right output? + ↓ +3. If NO → identify which element is failing → fix it → go to 2 + ↓ +4. If YES → SIMPLIFY — remove elements one at a time + ↓ +5. TEST after each removal — did quality drop? + ↓ +6. If YES → put that element back + ↓ +7. If NO → keep it removed, try removing the next one + ↓ +8. RESULT: the minimum prompt that produces maximum quality +``` + +**Key quote from Anthropic:** "Prompt engineering is scientific trial and error. Mix and match elements to see what works best. Not all prompts need every element." + +--- + +## REFERENCE: Technique Priority by Task Type + +When evaluating, weight dimensions differently based on the task: + +| Task Type | Highest-Impact Techniques | Lower Priority | +| ------------------- | ---------------------------------------------------------- | ------------------------------- | +| Classification | Few-shot examples, Prefill, Output format | Chain of thought, Chaining | +| Document Q&A | Hallucination defense, Data separation, Question placement | Few-shot, Prefill | +| Creative writing | Role, Clarity, Tone context | Hallucination defense, Examples | +| Code generation | Role, Clarity, Examples, Output format | Hallucination defense | +| Data extraction | Few-shot examples, Output format, Prefill | Role, Chain of thought | +| Math/Logic | Role, Chain of thought, Examples | Prefill, Hallucination | +| Chatbot/Roleplay | Role, Rules, Examples, System prompt | Hallucination, Chaining | +| Summarization | Clarity, Output format, Data separation | Examples, Chain of thought | +| Multi-step analysis | Chain of thought, Structure, Chaining | Prefill, Examples | + +Begin by reading the input prompt. diff --git a/.claude/commands/spec.md b/.claude/commands/spec.md new file mode 100644 index 000000000..e6d58c922 --- /dev/null +++ b/.claude/commands/spec.md @@ -0,0 +1,365 @@ +--- +description: Generate a behavioral spec sheet from an approved plan document +argument-hint: +allowed-tools: Read, Glob, Grep, Write, Bash(git diff:*), Bash(git log:*), Bash(git branch:*), Bash(wc:*), Bash(date:*), Agent +--- + +# Spec Generator + +You are a senior systems architect who writes behavioral specifications for safety-critical desktop software. You specialize in translating implementation plans into airtight, auditable specs where every claim is verified against source code and every acceptance criterion is testable. You write with surgical precision — no filler, no ambiguity, no unverified claims. + +Your task: generate a production-grade behavioral specification from an approved plan document. The spec defines **what** the system must do (behavior, contracts, acceptance criteria). The plan defines **how** to build it (phases, file changes, implementation order). Together they form the complete development contract. + +**Plan document**: $1 + +## Validation + +If `$1` is empty, output: + +``` +Usage: /spec +Example: /spec docs/plans/agent-browser-integration.md +``` + +Stop execution. + +## Context + +- Current branch: !`git branch --show-current` +- Date: !`date +%Y-%m-%d` +- This is a **Tauri 2 + React 19** production code editor. Users' active projects are at stake. The spec must be airtight. +- The spec must be consistent with project conventions defined in root and sub-level CLAUDE.md files (Zod schemas, Zustand stores, structured logging, ESLint rules, etc.). + +## Principles + +These override all other instructions. When in doubt, return to these. + +1. **Codebase over plan.** The plan is a proposal. The codebase is reality. When they disagree about existing behavior, the codebase wins. When they disagree about future behavior, the plan wins. +2. **Verify every claim.** Do not write "13 existing tools" because the plan says so — count them in the actual source file. Do not write "URL transport limit is 100KB" because the plan says so — read the constant in the code. +3. **No AI slop.** No filler sentences. No "This ensures a seamless experience." Every sentence must convey information an implementer needs to build or test the feature. +4. **Testability is mandatory.** Every acceptance criterion must pass the three-part test: Input -> Action -> Expected Output. If you can't write all three parts, don't spec it — put it in Open Questions with a safe default. +5. **No over-specification.** Don't define behavior for scenarios that can't happen given the constraints. The spec matches the plan's scope — no more, no less. +6. **No under-specification.** Every behavior that WILL occur in production must be defined. If you don't know, put it in Open Questions with a safe default. + +## Output Rules + +- **Do not narrate your research process.** Silently perform Steps 0-3 (reading, analysis, cross-validation). Only output the spec file and the Step 5 summary. +- **If you cannot verify a claim**, flag it explicitly: "UNVERIFIED: [claim]. Plan states X but source file not found/not readable." +- **If the plan is vague or incomplete**, do not guess. Place unresolvable gaps in Section 12 (Open Questions) with a safe default and proceed with what is clear. + +--- + +## Step 0: Read the Plan + Codebase (silent) + +### 0.1 Read the plan completely + +Extract: + +1. Feature name and scope +2. All phases (numbered implementation steps) +3. All files to create/modify (the change surface) +4. All edge cases addressed +5. All deferred/out-of-scope items +6. Audit history (if the plan was audited, note the verdict and key decisions) + +If the plan document doesn't exist, output "Plan document not found at `$1`" and stop. + +### 0.2 Read the actual codebase + +For every **existing** file the plan references: + +1. Read the file +2. Note actual function signatures, tool names, APIs, constants +3. Note behavior that changes vs behavior that stays +4. If the file doesn't exist: flag as plan drift. Search for the functionality elsewhere and note the correct path. + +For every **new** file the plan creates: + +1. Note what module it belongs to, what it exports, what depends on it + +**Stop when you can answer:** What exactly exists today? What exactly changes? What stays the same? + +### 0.3 Check for existing spec + +Search `docs/specs/` for a spec matching this plan. If one exists, read it — you're generating a revision, not starting from scratch. Preserve correct sections and fix issues. + +--- + +## Step 1: Spec Structure + +Every spec MUST contain these sections. No exceptions. + +``` +1. Problem Statement +2. Users and Use Cases +3. Expected Behavior +4. Acceptance Criteria +5. Non-Functional Requirements +6. Constraints +7. Error Taxonomy +8. Data Contracts +9. Out of Scope +10. Risks and Mitigations +11. Success Metrics +12. Open Questions +A. Appendix: Full Inventory +B. Appendix: Glossary +``` + +--- + +## Step 2: Write Each Section + +### Section 1: Problem Statement + +Three subsections, each 2-4 sentences: + +- **What's broken** — current pain, grounded in actual codebase behavior +- **Why it matters** — business/user impact +- **What success looks like** — north star in one sentence + +### Section 2: Users and Use Cases + +**Users table**: Who uses this feature, in what context, what's their current pain. + +**Use cases (UC-N)**: Numbered (5-10), each with: + +- User intent in blockquote +- Numbered steps showing the happy path +- **Key property** callout: the invariant this use case demonstrates +- At least 1 UC must demonstrate failure recovery +- At least 1 UC must demonstrate backward compatibility (if applicable) +- Every UC must be end-to-end and self-contained + +### Section 3: Expected Behavior + +Organize by subsystem or capability. For each behavior: + +- **Input**: What triggers it +- **Output**: What it produces +- **Behavioral requirements table**: Every rule as a row + +For interaction tools/APIs, use pre/postcondition tables: + +| Tool | Preconditions | Postconditions | Side effects | +| ---- | ------------- | -------------- | ------------ | + +Rules: + +- Every behavior must be deterministic given same input and state +- Return types must match Data Contracts (Section 8) exactly +- Reference actual codebase behavior for existing features + +### Section 4: Acceptance Criteria + +Numbered AC-N, grouped by priority: P0 (blocks release), P1 (core experience), P2 (can defer), P3 (nice to have). + +Each AC must pass the three-part test: + + +Well-formed acceptance criterion + +**AC-3 (P0): Screenshot capture returns base64 image data** + +- **Input**: Active browser tab showing `https://example.com`, tab dimensions 1280x720 +- **Action**: Call `browser_screenshot` tool with `{ tabId: activeTabId, format: "png" }` +- **Expected output**: Tool returns `{ success: true, data: "", mimeType: "image/png" }`. Base64 decodes to a valid PNG image with dimensions <= 1280x720. + + + +Well-formed acceptance criterion (failure case) + +**AC-7 (P0): Screenshot of closed tab returns structured error** + +- **Input**: `tabId` referencing a tab that was closed 5 seconds ago +- **Action**: Call `browser_screenshot` tool with `{ tabId: closedTabId }` +- **Expected output**: Tool returns `{ success: false, error: "Tab not found: {tabId}" }`. No crash, no unhandled exception. + + +Rules: + +- Every plan phase must have at least one AC +- Every use case must have at least one AC +- P0 count should be 5-10. More than 15 P0s suggests the scope is too large. +- Include one AC for "all quality gates pass" (typecheck, lint, clippy, build) + +### Section 5: Non-Functional Requirements + +Tables for: Performance (metric | target | measurement method), Reliability, Security, Compatibility. + +Rules: + +- Every performance target must have a feasible measurement method +- Derive targets from plan goals or existing benchmarks — don't invent them +- If measurement requires infrastructure not in scope, note in Open Questions + +### Section 6: Constraints + +| Constraint | Rationale | +| ---------- | --------- | + +Rules: + +- Each must be enforceable (verifiable) +- Cannot contradict ACs or Expected Behavior +- Derive from plan's architectural decisions + +### Section 7: Error Taxonomy + +Every error the system can produce, by category: + +| Error | Message template | Recovery | +| ----- | ---------------- | -------- | + + +Well-formed error taxonomy entry + +| Error | Message template | Recovery | +| ------------------ | --------------------------------------------------- | ------------------------------------------------------------------------------------- | +| Tab not found | `Tab not found: {tabId}. It may have been closed.` | Caller should refresh tab list via `browser_list_tabs` and retry with a valid tab ID. | +| Navigation timeout | `Navigation to {url} timed out after {timeoutMs}ms` | Retry with longer timeout, or verify URL is reachable outside the browser. | + + + +Rules: + +- Every failure path in Section 3 must have a corresponding error +- Every error must have an actionable recovery instruction +- Use concrete message templates with `{variable}` placeholders + +### Section 8: Data Contracts + +TypeScript interfaces for every data structure the feature exposes or consumes. + +Rules: + +- No `any`, no `unknown` without justification +- Every field must be referenced elsewhere in the spec +- Document when optional fields are present vs absent +- Include ASCII state machine diagrams for stateful components + +### Section 9: Out of Scope + +| Item | Why deferred | +| ---- | ------------ | + +Rules: + +- Every deferred item from the plan must appear here +- Nothing in Sections 3-4 should implement an out-of-scope item +- Verify against codebase — don't exclude something that already exists + +### Section 10: Risks and Mitigations + +| Risk | Likelihood | Impact | Mitigation | +| ---- | ---------- | ------ | ---------- | + +Rules: + +- Minimum 5 risks +- At least 1 about the plan's core technical bet +- At least 1 about integration with existing systems +- Mitigations must be actionable + +### Section 11: Success Metrics + +| Metric | Baseline (current) | Target (post-launch) | How to measure | +| ------ | ------------------ | -------------------- | -------------- | + +Rules: + +- Mark estimates as "(estimate)" — don't present guesses as data +- Include at least 1 reliability metric + +### Section 12: Open Questions + +| # | Question | Default if unanswered | Impact | +| --- | -------- | --------------------- | ------ | + +Rules: + +- Genuinely open, not rhetorical +- Each has a safe default so implementation isn't blocked +- Flag questions where the default could be wrong + +### Appendix A: Full Inventory + +Complete enumeration of every tool, command, API, or surface the feature exposes. + +| # | Name | Status (New/Updated/Existing) | Category | +| --- | ---- | ----------------------------- | -------- | + +Rules: + +- Count must be verified against plan's file tables AND codebase +- Every "Existing" item must actually exist (verified in Step 0) +- Every "New" item must have behavior defined in Section 3 + +### Appendix B: Glossary + +| Term | Definition | +| ---- | ---------- | + +For domain-specific terms. A reader unfamiliar with the codebase should understand the spec using only this glossary. + +--- + +## Step 3: Cross-Validate (silent) + +Before writing the file, verify all 8 checks: + +1. **Plan coverage**: Every plan phase -> at least one AC +2. **Use case coverage**: Every UC -> at least one AC +3. **Error coverage**: Every failure path in Section 3 -> error in Section 7 +4. **Contract coverage**: Every return type in Section 3 -> interface in Section 8 +5. **Constraint compliance**: No AC violates a constraint +6. **Scope compliance**: No Expected Behavior implements an out-of-scope item +7. **Inventory accuracy**: Appendix A counts match actual tool/command counts +8. **Naming consistency**: Tool names, schema names, field names identical across all sections + +If any check fails, fix it before writing. + +--- + +## Step 4: Write the Spec + +Write to `docs/specs/{plan-base-name}-spec.md`. + +Example: `docs/plans/agent-browser-integration.md` -> `docs/specs/agent-browser-integration-spec.md` + +**Header format:** + +```markdown +# Spec: [Feature Name] + +> **Companion plan**: `[path to plan]` +> +> **Status**: Draft +> **Date**: [YYYY-MM-DD] +> **Scope**: [1-sentence scope] +``` + +Use H2 (`##`) for sections, H3 (`###`) for subsections. + +If `docs/CLAUDE.md` exists, update it: add the `specs/` folder to the Folder Guide table (if not present) and add the new spec file to the listing. + +--- + +## Step 5: Summary Output + +After writing, output exactly: + +``` +Spec written to docs/specs/{filename} +12 sections, {N} acceptance criteria (P0: {n}, P1: {n}, P2: {n}, P3: {n}) + +Cross-validation: + Plan coverage: {N}/{N} phases covered + Use case coverage: {N}/{N} UCs with ACs + Error coverage: {N}/{N} failure paths with errors + Inventory accuracy: {N} tools verified against codebase + +Run /audit-spec docs/specs/{filename} to validate before implementation. +``` + +Begin by reading the plan document. diff --git a/.codex/skills/audit-spec/SKILL.md b/.codex/skills/audit-spec/SKILL.md new file mode 100644 index 000000000..30025510e --- /dev/null +++ b/.codex/skills/audit-spec/SKILL.md @@ -0,0 +1,474 @@ +--- +name: audit-spec +description: Audit behavioral specification documents for completeness, correctness, internal consistency, plan alignment, codebase drift, and testability before implementation begins. Use when asked to review a feature spec, spec sheet, requirements document, or acceptance criteria document against the companion plan and the existing Tauri 2 + React 19 + Rust codebase. +allowed-tools: + - Read + - Glob + - Grep + - Write + - Bash(git diff:*) + - Bash(git log:*) + - Bash(git branch:*) + - Bash(wc:*) + - Bash(date:*) + - Agent +--- + +# Spec Audit + +Perform a rigorous audit of a behavioral specification document before implementation begins. + +Treat the spec as the source of truth for behavior and the companion plan as the source of truth +for implementation strategy. Read both critically. Find every gap, contradiction, outdated codebase +claim, and untestable acceptance criterion. + +## Input and validation + +Require a spec document path. + +If the path argument is missing, output exactly: + +```text +Usage: /audit-spec +Example: /audit-spec docs/specs/agent-browser-integration-spec.md +``` + +Stop execution. + +If the spec path does not exist, output exactly: + +```text +Spec document not found at `` +``` + +Stop execution. + +Derive the output filename from the spec path. Example: docs/specs/agent-browser-integration-spec.md produces +reviews/audit-agent-browser-integration-spec.md. + +Write the audit report to reviews/audit-{spec-base-name}.md. + +If `reviews/audit-spec.md` already exists, read it first and increment the `Pass` number by 1. +Otherwise set `Pass` to `1`. + +## Context + +Collect these baseline facts before auditing: + +- Current branch: `git branch --show-current` +- Current date: use the local system date +- System assumptions: production Tauri 2 + React 19 desktop app, strict TypeScript, Rust backend, + and user projects at risk if the spec is wrong + +If the spec references a companion plan, read that plan completely before making any judgments about +coverage or contradictions. + +## Critical Rules + +These override everything else. Internalize before proceeding. + +1. Read the actual source code. Do not trust the spec's claims about existing code. Verify every + one. +2. Be adversarial. Your job is to find what's wrong, not to confirm it's right. Assume the spec + has errors until proven otherwise. +3. No rubber-stamping. APPROVED means you verified every claim and found no critical issues. If + you can't verify something, flag it as a gap. +4. Be concrete. Every finding must reference specific sections, line numbers, or file paths. "The + error handling seems incomplete" is not a finding. + +## Severity definitions + +| Severity | Meaning | Action | +| --------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------- | +| `CRITICAL` | Spec is wrong, contradicts plan, contradicts codebase, or leaves likely production behavior undefined | Must fix before implementation begins | +| `INCONSISTENCY` | Spec contradicts itself, or numbers/names do not match between sections | Must fix before implementation | +| `GAP` | Behavior is undefined for a scenario likely to occur in production | Must define before implementation | +| `UNVERIFIABLE` | Acceptance criterion or requirement cannot be tested as written | Must rewrite to be testable | +| `DRIFT` | Spec claims something about the current codebase that is false or outdated | Must correct to match reality | +| `WARNING` | Ambiguity could lead to misinterpretation, but there is a plausible default | Should clarify | +| `NOTE` | Observation worth surfacing, but not a defect | Informational | + +## Audit workflow + +### Step 0: Read the spec completely + +Read the spec document from top to bottom before auditing any single section in isolation. + +Extract and verify: + +1. Companion plan path, if referenced +2. Feature scope and explicitly stated goals +3. Major section list and overall structure +4. Acceptance criteria count, including any priority breakdown +5. Explicit out-of-scope items + +If the spec references a companion plan, read it completely. Use the plan as the implementation +strategy source of truth and the spec as the behavior source of truth. Flag contradictions between +them later in the audit. + +### Step 1: Verify codebase ground truth + +Do not trust any claim in the spec about the current codebase. + +For every claim about existing code: + +1. Locate the actual source file +2. Read the relevant code +3. Compare the code to the spec claim +4. Record any mismatch as `DRIFT` + +Minimum verification scope: + +- Every file path named in the spec +- Every API, function, type, schema, or command the spec says already exists +- Every current-state or baseline claim +- Every tool count, transport detail, timeout, size limit, or capability inventory tied to existing + code + +Stop only when you can answer both questions confidently: + +- Does the spec accurately describe the current codebase state? +- Are all file paths, function names, and API surfaces correct? + +If a referenced file does not exist, flag it as DRIFT. Search for the functionality elsewhere and +note the correct path in the finding. + +If the companion plan does not exist or is not referenced, proceed without plan cross-validation +(Step 2). Note in the audit report that plan alignment was not verified and mark the Plan Alignment +grade as N/A. + +### Step 2: Cross-validate the spec against the plan + +Compare the behavior spec to its companion plan section by section. + +#### 2.1 Coverage check + +- Map every plan phase to at least one acceptance criterion or explicit behavior section +- Verify every file the plan creates or modifies has behavior defined in the spec +- Verify every edge case called out in the plan has expected behavior in the spec + +Flag: + +- `GAP` for any plan phase or edge case with no corresponding spec coverage +- `CRITICAL` for any acceptance criterion or behavior definition that contradicts the plan’s stated + implementation approach + +#### 2.2 Numerical consistency + +Cross-check all counts, limits, and thresholds: + +- Tool inventories +- File counts +- Timeout values +- Retry counts +- Payload caps +- Token or size limits + +Flag any mismatch as `INCONSISTENCY`. + +#### 2.3 Naming consistency + +Cross-check names exactly: + +- Tool names +- Command names +- Type and schema names +- Field names +- Enum values +- State names +- File paths + +Flag any mismatch as `INCONSISTENCY`. + +### Step 3: Audit internal consistency + +Cross-reference every major section against the rest of the document. + +#### 3.1 Acceptance criteria vs expected behavior + +For every acceptance criterion: + +- Verify the behavior it tests is actually defined elsewhere in the spec +- Verify the criterion does not contradict the behavior sections +- Verify a test could be written from the spec alone + +Flag: + +- `UNVERIFIABLE` when the criterion lacks a clear input, action, or expected output +- `INCONSISTENCY` when the criterion contradicts stated behavior + +#### 3.2 Error taxonomy vs expected behavior + +Cross-check error handling in both directions: + +- Every listed error should correspond to a concrete trigger scenario +- Every failure mode in the behavior sections should map to an error or explicit surfaced outcome +- Error names and messages should match between sections + +Flag `GAP` for orphan errors, missing failure outcomes, or missing user-visible error behavior. + +#### 3.3 Data contracts vs usage + +For every data contract, schema, or payload: + +- Verify every field is referenced somewhere meaningful in behavior, validation, error handling, or + acceptance criteria +- Verify field types are consistent everywhere in the spec +- Verify required vs optional semantics are consistent + +Flag `WARNING` for orphan fields, weakly specified fields, or type mismatches that could mislead +implementation. + +#### 3.4 Constraints vs all other sections + +Check that every constraint is enforceable and not violated elsewhere: + +- Performance constraints +- Safety constraints +- Ordering constraints +- Size limits +- Transport constraints +- Persistence or cleanup constraints + +Flag `CRITICAL` if an acceptance criterion or behavior definition violates a stated constraint. + +#### 3.5 Out-of-scope vs in-scope behavior + +Verify the document respects its own scope boundaries: + +- No expected behavior should require an out-of-scope item +- No acceptance criterion should implicitly force an out-of-scope implementation +- No out-of-scope item should be contradicted by an in-scope requirement + +Flag any scope contradiction as `CRITICAL`. + +### Step 4: Audit completeness + +#### 4.1 Happy path coverage + +For every primary use case: + +- Verify the flow is specified from trigger to final state +- Verify intermediate states are defined +- Verify the final state is observable and testable + +Flag `GAP` for incomplete happy paths. + +#### 4.2 Failure path coverage + +For every use case, check: + +- What happens if each major step fails +- What happens on timeout +- What happens on disconnect or process crash +- What happens if the user cancels or interrupts mid-flow +- What happens on partial success + +Flag `GAP` for undefined failure behavior or missing recovery rules. + +#### 4.3 Boundary conditions + +Check at minimum: + +- Empty inputs +- Null and undefined handling +- Zero-length arrays or zero-value counts +- Maximum payload sizes +- Largest expected element counts +- Longest expected strings or prompts +- Concurrent operations +- Unexpected events in each state transition + +Flag `GAP` for undefined boundary behavior. + +#### 4.4 Cross-feature interactions + +Check how the feature interacts with existing systems: + +- Chat or session state +- File system state +- Terminal state +- Browser or canvas state +- Persistence +- Existing tool execution flows + +Flag `GAP` for unaddressed side effects or interactions with adjacent features. + +### Step 5: Audit testability + +For every acceptance criterion, spell out: + +1. Input: what must be provided +2. Action: what exact action occurs +3. Expected output: what exact result must be verified + +If any of the three is ambiguous, flag `UNVERIFIABLE`. + +Also evaluate non-functional requirements: + +- Can performance targets be measured with the tools and instrumentation in scope? +- Can success metrics be collected without inventing extra systems not described in the spec? +- Can non-functional requirements be validated before production rollout? + +Flag any ambiguous or non-measurable requirement as `UNVERIFIABLE`. + +### Step 6: Write the audit report + +Write the report to reviews/audit-{spec-base-name}.md using this structure: + +```markdown +# Spec Audit: [Spec Title] + +**Date**: [current date] +**Spec Document**: [path-to-spec-doc] +**Companion Plan**: [path if present, otherwise "None referenced"] +**Branch**: [branch name] +**Pass**: [incremented pass number] + +## Spec Summary + +[2-3 sentences describing the feature scope, primary behavior being specified, and overall audit +posture] + +## Audit Scope + +| Check | Files read | Claims verified | +| --------------------- | -------------------------- | ----------------------- | +| Codebase ground-truth | [N files] | [N claims] | +| Plan cross-validation | [yes/no] | [N checks] | +| Internal consistency | [N sections cross-checked] | [N cross-references] | +| Completeness | [N use cases, N ACs] | [N boundary conditions] | +| Testability | [N ACs evaluated] | [N verified testable] | + +## Verdict: [APPROVED / APPROVED WITH CHANGES / NEEDS REWORK] + +[1-2 sentence justification] + +## Grading + +| Dimension | Grade | Notes | +| ---------------------------------------- | ----- | ----- | +| Correctness (claims match codebase) | [A-F] | | +| Plan alignment (spec covers plan fully) | [A-F] | | +| Internal consistency (no contradictions) | [A-F] | | +| Completeness (no gaps in behavior) | [A-F] | | +| Testability (all ACs are verifiable) | [A-F] | | +| Error coverage (all failures handled) | [A-F] | | + +## Findings + +### Critical (Must fix before implementation) + +| # | Severity | Section | Finding | Evidence | Fix | +| --- | -------- | ------- | --------------- | -------------------------------------- | ------------ | +| 1 | CRITICAL | 3.2 | [What is wrong] | [File:line or section cross-reference] | [How to fix] | + +### Inconsistencies (Must fix) + +| # | Section A | Section B | Mismatch | Fix | +| --- | --------- | ---------- | --------------------- | -------------------------- | +| 1 | 3.1 | Appendix A | [What does not match] | [Which version is correct] | + +### Gaps (Behavior undefined for production scenario) + +| # | Scenario | What is missing | Suggested behavior | +| --- | ------------------------- | ------------------ | --------------------- | +| 1 | [When X happens during Y] | [Missing behavior] | [Proposed resolution] | + +### Unverifiable ACs (Must rewrite) + +| # | AC | Problem | Suggested rewrite | +| --- | ---- | ------------------------- | ------------------ | +| 1 | AC-N | [Why it cannot be tested] | [Testable version] | + +### Drift (Spec claims do not match codebase) + +| # | Spec claim | Actual from codebase | File | Fix | +| --- | ------------------- | -------------------- | -------------- | ----------- | +| 1 | "13 existing tools" | [Actual count] | `path/to/file` | Update spec | + +### Warnings (Clarify before implementation) + +| # | Section | Ambiguity | Suggested clarification | +| --- | ------- | ----------------- | ----------------------- | +| 1 | 3.3 | [What is unclear] | [How to clarify] | + +### Notes (Informational) + +- [Observation 1] +- [Observation 2] + +## Codebase Verification Log + +| Spec claim | File checked | Verified? | Notes | +| ----------------- | ------------------- | --------- | --------------------- | +| [Claim from spec] | `path/to/file:line` | Yes/No | [Details if mismatch] | + +## Plan-Spec Alignment + +| Plan phase | Spec coverage | Gap? | +| ------------------- | ----------------- | ------------------ | +| Phase 1: Runtime | Section 3.1, AC-1 | No | +| Phase 2: MCP Server | Section 3.4, AC-5 | Partial - [detail] | + +## Required Before Implementation + +- [ ] Fix Critical #1: [description] +- [ ] Fix Inconsistency #1: [description] +- [ ] Define Gap #1: [description] +- [ ] Rewrite AC-N: [description] + +## Verdict Details + +### Correctness: [PASS / CONCERNS] + +[Do the spec's codebase claims match reality?] + +### Plan Alignment: [PASS / CONCERNS] + +[Does the spec fully cover the plan?] + +### Internal Consistency: [PASS / CONCERNS] + +[Does the spec contradict itself?] + +### Completeness: [PASS / CONCERNS] + +[Are all likely production scenarios defined?] + +### Testability: [PASS / CONCERNS] + +[Can every acceptance criterion be verified?] + +### Error Coverage: [PASS / CONCERNS] + +[Is every failure mode handled explicitly?] +``` + +If a section has no findings, keep the section and write `None.` instead of deleting it. + +### Step 7: Final output contract + +After writing the report: + +1. Print Audit written to reviews/audit-{spec-base-name}.md +2. Print the `Verdict`, `Grading`, and `Required Before Implementation` sections +3. Print total finding counts in this format: + +```text +Findings: X critical, X inconsistencies, X gaps, X unverifiable, X drift, X warnings, X notes +``` + +## Operating rules + +- Read actual source code. Never trust the spec's claims about the current implementation without + verifying them. +- Be adversarial. Assume the spec has errors until the evidence says otherwise. +- Be concrete. Reference exact sections, file paths, and line numbers whenever possible. +- Do not rubber-stamp. `APPROVED` requires zero critical issues, zero inconsistencies, and zero + gaps. +- Cross-check exhaustively. Two sections that look reasonable in isolation may still contradict + each other. +- Treat testability as mandatory. If you cannot describe the test, the criterion is not done. +- Prefer multiple audit passes. Each pass should reduce open findings and increase confidence. +- Respect the production bar. A spec gap becomes an implementation bug unless it is fixed now. diff --git a/.codex/skills/audit-spec/agents/openai.yaml b/.codex/skills/audit-spec/agents/openai.yaml new file mode 100644 index 000000000..05d3bab7a --- /dev/null +++ b/.codex/skills/audit-spec/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: 'Spec Audit' + short_description: 'Audit behavior specs for production readiness' + default_prompt: 'Use $audit-spec to audit docs/specs/agent-browser-integration-spec.md and write the report to reviews/audit-spec.md.' diff --git a/.codex/skills/spec-generator/SKILL.md b/.codex/skills/spec-generator/SKILL.md new file mode 100644 index 000000000..cc1624913 --- /dev/null +++ b/.codex/skills/spec-generator/SKILL.md @@ -0,0 +1,415 @@ +--- +name: spec-generator +description: Generate a production-grade behavioral specification from an approved plan document. Use when asked to turn a plan doc into a spec sheet, requirements doc, behavioral contract, acceptance-criteria document, or implementation-ready spec for this Tauri 2 + React 19 + Rust codebase. +allowed-tools: + - Read + - Glob + - Grep + - Write + - Bash(git diff:*) + - Bash(git log:*) + - Bash(git branch:*) + - Bash(date:*) + - Bash(wc:*) + - Agent +--- + +# Spec Generator + +Generate a production-grade behavioral specification from an approved plan document. + +The spec defines what the system must do: behavior, contracts, acceptance criteria, constraints, +error handling, and measurable outcomes. The companion plan defines how to build it: +implementation phases, file changes, and rollout order. Together they form the delivery contract. + +## Input and validation + +Require a plan document path argument. + +If the argument is missing, output exactly: + +```text +Usage: /spec +Example: /spec docs/plans/agent-browser-integration.md +``` + +Stop execution. + +If the path does not exist, output exactly: + +```text +Plan document not found at `` +``` + +Stop execution. + +## Context + +Collect these baseline facts before writing: + +- Current branch: `git branch --show-current` +- Current date: use the local system date +- System assumptions: this is a production Tauri 2 + React 19 desktop application with strict + TypeScript, Rust backend code, and user projects at risk if the specification is wrong + +## Critical Rules + +These override everything else. Internalize before proceeding. + +1. Read the code, not just the plan. The plan is a proposal. The codebase is reality. When they + disagree about existing behavior, the codebase wins. +2. Every claim about existing code must be verified. Do not write "13 existing tools" because the + plan says so. Count them in the actual source file. +3. No AI slop. No filler sentences. No "This ensures a seamless experience." Every sentence must + convey information an implementer needs. +4. Testability is mandatory. Every AC must pass the three-part test (Input, Action, Expected + Output). If you can't test it, don't spec it. + +## Step 0: Read the plan and codebase + +### 0.1 Read the plan completely + +Extract: + +1. Feature name and scope +2. All implementation phases +3. All files to create or modify +4. All edge cases and failure paths already called out +5. All deferred or out-of-scope items +6. Any audit history, including verdicts and key design decisions + +### 0.2 Read the actual codebase + +For every existing file the plan references: + +1. Read the file +2. Note the real function names, APIs, schemas, tool names, constants, and current behavior +3. Distinguish behavior that changes from behavior that remains unchanged +4. If a referenced file does not exist, flag it as plan drift. Search for the functionality + elsewhere and note the correct path. + +For every new file the plan proposes: + +1. Determine which module it belongs to +2. Determine what it is expected to export or expose +3. Determine what depends on it + +Do not trust the plan's memory of the current implementation. The codebase is the source of truth +for existing behavior. + +Stop only when you can answer all three questions confidently: + +- What exists today? +- What changes? +- What stays the same? + +If the plan is vague or incomplete (missing phases, unclear scope, no file tables), do not guess. +Put unresolvable gaps in Section 12 (Open Questions) with a safe default and proceed with what is +clear. + +### 0.3 Check for an existing spec + +Search `docs/specs/` for a matching spec. If one exists, read it before writing. Treat the task as +a revision when appropriate, not automatically as a greenfield document. + +## Step 1: Derive the spec structure + +Every generated spec must contain these sections: + +```text +1. Problem Statement +2. Users and Use Cases +3. Expected Behavior +4. Acceptance Criteria +5. Non-Functional Requirements +6. Constraints +7. Error Taxonomy +8. Data Contracts +9. Out of Scope +10. Risks and Mitigations +11. Success Metrics +12. Open Questions +A. Appendix: Full Inventory +B. Appendix: Glossary +``` + +Do not omit, rename, or merge sections unless the user explicitly overrides the format. + +## Step 2: Write each section + +### Section 1: Problem Statement + +Write three subsections, each 2-4 sentences: + +- What's broken: describe the current pain using real code paths and actual workflows +- Why it matters: describe the concrete impact on users or release quality +- What success looks like: one sentence describing the target outcome + +Do not use generic improvement language. Tie the problem statement to actual code and actual usage. + +### Section 2: Users and Use Cases + +Include: + +- A users table describing who uses the feature, in what context, and what pain they have today +- Between 5 and 10 numbered use cases (`UC-N`) + +Each use case must include: + +- A quoted user intent in a blockquote +- Numbered happy-path steps from request to verifiable outcome +- A `Key property` callout stating the invariant demonstrated by the use case + +Coverage requirements: + +- At least one use case must demonstrate failure recovery +- At least one use case must demonstrate backward compatibility when backward compatibility applies +- Every use case must be understandable without reading other sections first + +### Section 3: Expected Behavior + +Organize by subsystem or capability. + +For every behavior, define: + +- Input: what triggers the behavior +- Output: what the system produces +- A behavioral requirements table listing each rule and expected behavior + +For tools, APIs, or commands, include a pre/postcondition table: + +| Tool | Preconditions | Postconditions | Side effects | +| ---- | ------------- | -------------- | ------------ | + +Rules: + +- Specify deterministic behavior where the same input and state yield the same result +- If external state can vary, define behavior per state +- Keep return types consistent with Section 8 data contracts +- Describe current behavior accurately when it already exists instead of inventing replacements + +### Section 4: Acceptance Criteria + +Group criteria by priority: + +- `P0 — Must ship` +- `P1 — Should ship` +- `P2 — Can defer` +- `P3 — Nice to have` + +Every acceptance criterion must include: + +1. Input +2. Action +3. Expected output + +Rules: + +- Every plan phase must map to at least one acceptance criterion +- Every use case must map to at least one acceptance criterion +- Keep the `P0` count to roughly 5-10 unless the user explicitly wants broader scope +- Include one criterion covering quality gates: lint, typecheck, build, and any required Rust checks +- Include one backward-compatibility criterion when backward compatibility is in scope + +### Section 5: Non-Functional Requirements + +Provide tables for: + +- Performance: metric, target, measurement method +- Reliability: requirement, detail +- Security: requirement, detail +- Compatibility: requirement, detail + +Rules: + +- Every performance target must include a feasible measurement method +- Derive targets from the plan or from current known behavior +- If the target depends on instrumentation not in scope, note that in Open Questions + +### Section 6: Constraints + +Document hard boundaries the implementation must not violate. + +Use this format: + +| Constraint | Rationale | +| ---------- | --------- | + +Every constraint must be enforceable and must not contradict expected behavior or acceptance +criteria. + +### Section 7: Error Taxonomy + +Catalog every error category the feature can surface. + +Use this format: + +| Error | Message template | Recovery | +| ----- | ---------------- | -------- | + +Rules: + +- Every failure path in Expected Behavior must appear here +- Every error must have a concrete recovery instruction +- Message templates must use real placeholder syntax such as `{path}` or `{tool}` + +### Section 8: Data Contracts + +Define precise TypeScript interfaces for every consumed or exposed data structure. + +Rules: + +- Do not use `any` +- Avoid `unknown` unless the ambiguity is intentional and explained +- Document when optional fields are present versus absent +- Include ASCII state diagrams for stateful components +- Ensure every field is referenced somewhere else in the spec + +### Section 9: Out of Scope + +List explicit exclusions and why they are deferred. + +Use this format: + +| Item | Why deferred | +| ---- | ------------ | + +Rules: + +- Include every out-of-scope item named by the plan +- Do not list something as out of scope if it already exists and is working today +- Ensure no in-scope requirement depends on an out-of-scope item + +### Section 10: Risks and Mitigations + +Use this format: + +| Risk | Likelihood | Impact | Mitigation | +| ---- | ---------- | ------ | ---------- | + +Rules: + +- Include at least five risks +- Include at least one risk about the plan's core technical bet +- Include at least one risk about integration with existing systems +- Write actionable mitigations + +### Section 11: Success Metrics + +Use this format: + +| Metric | Baseline (current) | Target (post-launch) | How to measure | +| ------ | ------------------ | -------------------- | -------------- | + +Rules: + +- Use real baselines when available +- Mark estimates clearly when exact data does not exist +- Include at least one reliability metric +- Use measurement methods that are feasible in the current environment + +### Section 12: Open Questions + +Use this format: + +| # | Question | Default if unanswered | Impact | +| --- | -------- | --------------------- | ------ | + +Rules: + +- Only include genuinely unresolved questions +- Every question must have a safe default +- Call out when the default could still be wrong + +### Appendix A: Full Inventory + +Enumerate every tool, command, API, or user-facing surface the feature exposes. + +Use a numbered table with: + +- Name +- Status (`Existing`, `Updated`, or `New`) +- Category + +Verify every entry against both the plan and the actual codebase. + +### Appendix B: Glossary + +Define every domain-specific term needed to read the spec without outside context. + +## Step 3: Cross-validate before writing + +Before writing the final file, verify: + +1. Plan coverage: every plan phase has at least one acceptance criterion +2. Use case coverage: every use case is validated by at least one acceptance criterion +3. Error coverage: every failure path in Expected Behavior maps to Error Taxonomy +4. Contract coverage: every return type or payload in Expected Behavior maps to Data Contracts +5. Constraint compliance: no acceptance criterion violates a stated constraint +6. Scope compliance: no expected behavior implements an out-of-scope item +7. Inventory accuracy: Appendix A matches the verified codebase and plan surface +8. Naming consistency: tools, schemas, fields, commands, and file paths match across sections + +If any check fails, fix the spec before writing the file. + +## Step 4: Write the spec + +Write the spec to `docs/specs/{plan-base-name}-spec.md`. + +Example: + +- Plan: `docs/plans/agent-browser-integration.md` +- Output: `docs/specs/agent-browser-integration-spec.md` + +Use this header format: + +```markdown +# Spec: [Feature Name] + +> **Companion plan**: `[path to plan]` +> +> **Status**: Draft +> **Date**: [current date] +> **Scope**: [1-sentence scope] +``` + +## Step 5: Update the documentation index + +If `docs/CLAUDE.md` exists: + +1. Add the `specs/` folder to the folder guide if it is missing +2. Add the new spec file to the `specs/` listing +3. Follow the file's existing naming and listing conventions + +## Step 6: Report completion + +After writing, print: + +1. `Spec written to docs/specs/{filename}` +2. `12 sections, {N} acceptance criteria (P0: {n}, P1: {n}, P2: {n}, P3: {n})` +3. Cross-validation summary: + +```text +Cross-validation: + Plan coverage: {N}/{N} phases covered + Use case coverage: {N}/{N} UCs with ACs + Error coverage: {N}/{N} failure paths with errors + Inventory accuracy: {N} tools verified against codebase +``` + +4. `Run /audit-spec docs/specs/{filename} to validate before implementation.` + +## Operating rules + +- Read the code, not just the plan. When the plan and the current code disagree about existing + behavior, the codebase wins. +- When the plan and the desired future behavior disagree, the approved plan wins. +- Verify every claim about existing code. Do not trust counts, limits, names, or paths without + reading the source. +- Do not write filler. Every sentence must provide implementation-relevant information. +- Do not over-specify behavior outside the plan's scope. +- Do not under-specify production behavior that will definitely occur. +- If a required behavior cannot be stated confidently, put it in Open Questions with a safe default. +- Write only testable acceptance criteria and measurable requirements. +- Assume the spec will be audited against the plan and the codebase line by line. +- Treat specification gaps as production bugs waiting to happen. diff --git a/.codex/skills/spec-generator/agents/openai.yaml b/.codex/skills/spec-generator/agents/openai.yaml new file mode 100644 index 000000000..c6d376bb8 --- /dev/null +++ b/.codex/skills/spec-generator/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: 'Spec Generator' + short_description: 'Generate behavior specs from plans' + default_prompt: 'Use $spec-generator to turn docs/plans/agent-browser-integration.md into docs/specs/agent-browser-integration-spec.md.' diff --git a/.gitignore b/.gitignore index e9edcfedd..f7c06a387 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,8 @@ dist-ssr # Editor directories and files .vscode/* !.vscode/extensions.json +!.vscode/settings.json +!.vscode/tailwind.css-data.json .idea .DS_Store *.suo diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 000000000..2a8ca2580 --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,3 @@ +{ + "css.customData": [".vscode/tailwind.css-data.json"] +} diff --git a/.vscode/tailwind.css-data.json b/.vscode/tailwind.css-data.json new file mode 100644 index 000000000..063f3c252 --- /dev/null +++ b/.vscode/tailwind.css-data.json @@ -0,0 +1,55 @@ +{ + "version": 1.1, + "atDirectives": [ + { + "name": "@source", + "description": "Tailwind CSS v4: Register additional source paths for class detection.", + "references": [ + { + "name": "Tailwind CSS Documentation", + "url": "https://tailwindcss.com/docs/functions-and-directives" + } + ] + }, + { + "name": "@plugin", + "description": "Tailwind CSS v4: Load a Tailwind CSS plugin.", + "references": [ + { + "name": "Tailwind CSS Documentation", + "url": "https://tailwindcss.com/docs/functions-and-directives" + } + ] + }, + { + "name": "@custom-variant", + "description": "Tailwind CSS v4: Define a custom variant.", + "references": [ + { + "name": "Tailwind CSS Documentation", + "url": "https://tailwindcss.com/docs/functions-and-directives" + } + ] + }, + { + "name": "@theme", + "description": "Tailwind CSS v4: Define design tokens (colors, spacing, fonts, etc.).", + "references": [ + { + "name": "Tailwind CSS Documentation", + "url": "https://tailwindcss.com/docs/functions-and-directives" + } + ] + }, + { + "name": "@apply", + "description": "Tailwind CSS: Inline utility classes into custom CSS.", + "references": [ + { + "name": "Tailwind CSS Documentation", + "url": "https://tailwindcss.com/docs/functions-and-directives" + } + ] + } + ] +} diff --git a/AGENTS.md b/AGENTS.md index 26c018c2f..d11596e3b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -9,19 +9,29 @@ Orbit is a modern AI-powered code editor built with Tauri 2 (Rust backend) and R Frontend: - React 19 + TypeScript + Vite + - Bun workspaces for monorepo management + - Tailwind CSS v4 + - Zustand + Immer for state management + - CodeMirror 6 for code editing + - xterm.js for terminal emulation + - Shiki for code block highlighting + - Zod 4 for runtime validation Backend: - Tauri 2 for the desktop app framework + - Rust workspace with multiple crates + - portable-pty for terminal emulation + - Tree-sitter for syntax parsing (planned) ## Package Manager Policy (Bun Only) @@ -74,22 +84,31 @@ Tests live under `apps/agent/src/__tests__/` and `apps/canvas/src/*.test.ts`. Core: - `bun install` — install workspace dependencies. + - `bunx tauri dev` — full app (Vite + Tauri + Rust). + - `bun run dev` — frontend only (Vite, port 5176). + - `bun run build` — production build (includes sidecar). + - `bun run preview` — preview production build. Quality: - `bun run typecheck` — TypeScript `tsc --noEmit`. + - `bun run lint` / `bun run lint:fix` — ESLint (zero warnings). + - `bun run check` — typecheck + lint + tests. + - `bun run ci` — typecheck + lint + tests + Rust checks. + - `./scripts/lint-all.sh` — comprehensive linting (use `--fix` or `--no-test`). Rust: - `cargo build`, `cargo check`, `cargo test`, `cargo clippy`. + - `bun run rust:fmt` / `bun run rust:lint` / `bun run rust:test` (repo shortcuts). ## Development Workflow @@ -113,20 +132,27 @@ Hot reload behavior: ## Coding Style & Naming Conventions - TypeScript is strict (`tsconfig.json`) with explicit types and no unuseds. + - ESLint enforces import ordering and separate `import type` blocks (`import-x/order`). + - Use Prettier for formatting (`bun run format`). + - React components use `PascalCase`; hooks use `use-` prefixes; stores live in `apps/agent/src/stores/`. ## Testing Guidelines - Unit tests use Vitest APIs and live alongside feature areas. + - Use `bun run check` for the repo baseline; use `bun run rust:test` for Rust. + - Prefer adding tests for store logic and message/handler behavior. ## Commit & Pull Request Guidelines - Commit messages follow a Conventional Commits style (e.g., `feat:`, `fix:`). + - No PR template found; include a concise summary and list tests run. + - For UI changes, add screenshots or short clips when behavior is visual. ## CLAUDE.md (Full) @@ -142,19 +168,29 @@ Orbit is a modern AI-powered code editor built with **Tauri 2** (Rust backend) a ### Frontend - **React 19** + TypeScript + Vite + - **Bun** workspaces for monorepo management + - **Tailwind CSS v4** for styling + - **Zustand + Immer** for state management + - **CodeMirror 6** for code editing with custom themes + - **xterm.js** for terminal emulation + - **Shiki** for code block highlighting in chat + - **Zod 4** for runtime validation ### Backend - **Tauri 2** for desktop app framework + - **Rust** workspace with multiple crates + - **portable-pty** for terminal emulation + - Tree-sitter for syntax parsing (planned) ### Package Manager Policy @@ -175,10 +211,10 @@ Orbit is a modern AI-powered code editor built with **Tauri 2** (Rust backend) a **Rules:** -1. **Never use `npm` or `pnpm`** - Always use `bun` -2. **Use `bun run`** for all scripts -3. **Use `bun install`** for installing dependencies -4. **Run tests with `bun test`** +1. **Never use** **`npm`** **or** **`pnpm`** - Always use `bun` +2. **Use** **`bun run`** for all scripts +3. **Use** **`bun install`** for installing dependencies +4. **Run tests with** **`bun test`** ```bash # [ok] CORRECT @@ -345,7 +381,9 @@ Then restart the Tauri app (`Cmd+C` → `bunx tauri dev`). **Why manual rebuild?** - Tauri watches Rust code, not the agent-bridge TypeScript + - The sidecar is a standalone binary (58MB) with embedded Bun runtime + - Located at `target/debug/agent-bridge` in dev mode **Two build outputs:** @@ -364,7 +402,9 @@ bunx tauri build Creates distributable app in `src-tauri/target/release/bundle/`: - **macOS**: `.dmg` and `.app` + - **Windows**: `.exe` and `.msi` + - **Linux**: `.AppImage` and `.deb` ### Frontend-Only Development @@ -433,10 +473,15 @@ await writeTerminal(id, data); The editor (`apps/agent/src/components/editor/CodeMirrorEditor.tsx`) provides: - Full editing with syntax highlighting + - Custom dark/light themes matching app colors + - Language support: TypeScript, JavaScript, Python, Rust, Go, JSON, HTML, CSS, Markdown + - LSP autocompletion integration + - Cmd-S save functionality + - Theme-aware (syncs with app light/dark mode via MutationObserver) ### Theme Colors @@ -469,27 +514,39 @@ All message types in `apps/agent/src/types/protocol.ts` with Zod schemas: ### Frontend → Backend (WebviewMessage) - `message:send` - Send chat message + - `file:read`, `file:write` - File operations + - `terminal:create`, `terminal:write` - Terminal operations + - `agent:stop` - Stop AI generation ### Backend → Frontend (ExtensionMessage) - `agent:chunk`, `agent:complete` - AI responses + - `tool:start`, `tool:end` - Tool execution + - `file:content`, `file:tree:response` - File data + - `terminal:output`, `terminal:created` - Terminal data ## Code Style ### ESLint Rules (enforced) -- **No `any`** - All unsafe operations are errors +- **No** **`any`** - All unsafe operations are errors + - **Explicit return types** on functions + - **Consistent type imports** - Use `import type { }` separately + - **Import order** - External → Types → Internal, alphabetized. Use `bun run lint --fix` if unsure + - **No console.log** - Use structured logger (see below) + - **Strict boolean expressions** - No implicit truthy checks + - **Exhaustive switches** - All cases must be handled ### Structured Logging @@ -511,20 +568,27 @@ logger.error('Error occurred', new Error('fail')); // Always shown with stack **Benefits:** - **Context prefix** - Easily identify source: `[MyComponent] message` + - **Log levels** - Debug messages hidden in production + - **Structured data** - JSON metadata for log aggregation + - **Error handling** - Proper error serialization with stack traces ### Tailwind + Dynamic Styles - **Never use dynamic Tailwind classes** like `` `w-[${value}px]` `` + - Use inline styles for dynamic dimensions: `style={{ width: value }}` + - Static Tailwind classes work normally: `w-px`, `h-[32px]` ### React Patterns - Functional components with explicit `FC` type + - Ternary for conditional rendering + - Props interfaces marked `readonly` ## Module Organization Patterns @@ -655,27 +719,41 @@ pub use types::{FileStatus, GitStatus}; ### Completed - [x] Tauri 2 project setup with Rust workspace + - [x] Monorepo restructure (apps/, crates/common/, commands/) + - [x] CodeMirror 6 editor with custom themes + - [x] File editing with save (Cmd-S) + - [x] Modified indicator on tabs + - [x] Theme switching (light/dark) + - [x] Frontend-backend communication layer + - [x] Terminal with xterm.js + - [x] Git status & operations + - [x] Bun workspace management + - [x] CI/CD with GitHub Actions ### In Progress - [ ] Canvas app implementation + - [ ] Editor app implementation + - [ ] Shared packages extraction ### TODO - [ ] Tree-sitter syntax highlighting + - [ ] LSP/diagnostics integration + - [ ] Advanced search features ## Troubleshooting @@ -687,7 +765,9 @@ pub use types::{FileStatus, GitStatus}; Common symptoms: - "Invalid credentials" or "No credentials found" errors + - Data parsing silently returning `null` + - "Unrecognized keys" or "Invalid input: expected X, received Y" in logs **Root cause:** Schemas using `.strict()` will reject data with extra fields, and type mismatches (e.g., `number` vs `string`) cause validation failures. @@ -695,12 +775,15 @@ Common symptoms: **Where to look:** - `agent-bridge/src/schemas.ts` - Bridge IPC schemas + - `packages/shared-schemas/` - Shared validation schemas **Quick fix pattern:** - For **external data** (APIs, Keychain, SDK responses): Use `.loose()` instead of `.strict()` + - For **type mismatches**: Use `z.union([z.number(), z.string()])` for flexible types + - For **internal data** (your own code): `.strict()` is fine **Example fix (from KeychainCredentialsSchema):** @@ -737,7 +820,7 @@ Common symptoms: | Conversation Context Format | `apps/agent/src/hooks/use-tauri.ts:formatConversationContext()` | `agent-bridge/src/__tests__/conversation-rewind.test.ts` | | Combined Rewind Flow | `apps/agent/src/hooks/use-tauri.ts:conversation:rewind handler` | `agent-bridge/src/__tests__/combined-rewind.test.ts` | -**When adding new tests:** Always add the [warning] TESTED comment to the source function/handler being tested. +**When adding new tests:** Always add the \[warning] TESTED comment to the source function/handler being tested. ### Mandatory Test Requirements @@ -752,7 +835,7 @@ Common symptoms: #### What We Test -| [bad] DO NOT | [ok] DO | +| \[bad] DO NOT | \[ok] DO | | -------------------------------- | ---------------------------------------- | | Mock the Claude SDK | Use REAL Claude API calls | | Test single files in isolation | Test full module integration | @@ -773,19 +856,27 @@ agent-bridge/src/__tests__/ 1. **When creating a new file:** - Add tests to the relevant E2E test file (e.g., `canvas-e2e.test.ts`) + - Tests must exercise the FULL flow through real systems + - Run the complete test suite to verify integration - - Add the [warning] TESTED comment to the source file + + - Add the \[warning] TESTED comment to the source file 2. **When modifying a file that has existing tests:** - Find the E2E test file for that module + - **ADD NEW TESTS** that cover your new functionality + - Tests must verify the new code integrates with existing systems + - Do NOT just run existing tests - that defeats the purpose + - Run the FULL test suite to verify nothing broke 3. **When modifying a file without existing tests:** - Create E2E tests if the module doesn't have them + - Tests must cover the full integration path #### Real Integration Test Example @@ -826,10 +917,15 @@ it('should route simple requests to fast path via real session', async () => { #### Test Checklist Before PR - [ ] Added tests for ALL new functionality + - [ ] Tests use REAL systems (no mocks for core functionality) + - [ ] Tests verify FULL integration flow + - [ ] Tests cover error cases and cleanup + - [ ] Ran complete test suite: `cd agent-bridge && bun test` + - [ ] All tests pass with real API calls #### Running Tests Locally (IMPORTANT) @@ -852,6 +948,7 @@ bun test Tests use `describe.skipIf(process.env.GITHUB_ACTIONS === 'true')` to: - **Run locally**: OAuth from Claude Code CLI keychain is available + - **Skip in CI**: GitHub Actions has no OAuth access, tests would fail with "No credentials found" **If tests fail locally with "No credentials found":** @@ -953,10 +1050,15 @@ The following CSS properties trigger GPU compositing issues that result in momen #### Quick Checklist for Tauri-Compatible CSS - [ ] No `backdrop-filter: blur()` on interactive elements + - [ ] No `color-mix()` function - use CSS variables or `rgba()` instead + - [ ] No `transition` on elements inside ReactFlow's transformed viewport + - [ ] No `animation` on elements inside ReactFlow's transformed viewport + - [ ] Avoid combining `opacity` transitions with other transforms + - [ ] Test in Tauri app, not just browser (blur won't appear in browser) **Note:** `contain: layout style paint` and `will-change: transform` do NOT fix the blur issue. The only solution for elements inside ReactFlow's viewport is to completely remove transitions and animations. @@ -973,9 +1075,9 @@ The following CSS properties trigger GPU compositing issues that result in momen **Last audited:** January 2025 -| Package | Severity | CVE | Status | Notes | -| --------------------------- | -------- | ------------- | -------------------------- | ----------------------------------------------- | -| `@modelcontextprotocol/sdk` | High | CVE-2026-0621 | [pending] Waiting upstream | ReDoS in UriTemplate class. No patch available. | +| Package | Severity | CVE | Status | Notes | +| --------------------------- | -------- | ------------- | --------------------------- | ----------------------------------------------- | +| `@modelcontextprotocol/sdk` | High | CVE-2026-0621 | \[pending] Waiting upstream | ReDoS in UriTemplate class. No patch available. | #### MCP SDK ReDoS (CVE-2026-0621) @@ -1032,12 +1134,15 @@ All color variables are defined in `apps/agent/src/globals.css`: ### Rules -1. **NEVER define `:root` color variables in canvas globals.css** - They will override agent colors +1. **NEVER define** **`:root`** **color variables in canvas globals.css** - They will override agent colors 2. **Canvas uses agent's variables** - e.g., `var(--background)`, `var(--card)`, `var(--primary)` 3. **Canvas globals.css contains only:** - Tailwind `@theme` mappings (pointing to agent's variables) + - ReactFlow style overrides (`.react-flow__*`) + - Scrollbar styling + - Base layout (html, body, #root) ### Adding New Colors @@ -1072,7 +1177,9 @@ The Canvas app is embedded within the Agent app as a mode/tab (not a separate Ta ### Canvas Libraries - **@xyflow/react** (ReactFlow) - Node-based canvas + - **@codesandbox/sandpack-react** - Code playground integration + - **Zustand** - State management (`apps/canvas/src/stores/`) ### Canvas Directory Structure @@ -1099,10 +1206,16 @@ apps/canvas/src/ - **Migrated from pnpm to Bun** - All package management now uses Bun for faster installs and unified tooling - Removed `pnpm-workspace.yaml` - workspaces defined in `package.json` + - Removed `pnpm-lock.yaml` - replaced by `bun.lockb` + - Updated all scripts, CI workflows, and husky hooks to use `bun` + - Added comprehensive `lint-all.sh` script for running all checks + - **Added comprehensive audit** - Aligned all linting, TypeScript, and CI checks across the monorepo + - **Documented module organization patterns** - Barrel pattern for frontend, explicit paths for Rust backend - Frontend: Every folder with multiple files gets an `index.ts` barrel + - Backend: Tauri commands use explicit paths, shared crates use selective re-exports diff --git a/Cargo.lock b/Cargo.lock index c21a34d50..b2924b670 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2141,6 +2141,12 @@ dependencies = [ "pin-project-lite", ] +[[package]] +name = "http-range" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "21dec9db110f5f872ed9699c3ecf50cf16f423502706ba5c72462e28d3157573" + [[package]] name = "httparse" version = "1.10.1" @@ -3521,6 +3527,7 @@ dependencies = [ "log", "notify", "orbit-ai", + "orbit-app-icons", "orbit-conversations", "orbit-core", "orbit-fs", @@ -3559,6 +3566,19 @@ dependencies = [ "whoami", ] +[[package]] +name = "orbit-app-icons" +version = "0.0.1" +dependencies = [ + "log", + "objc2", + "objc2-app-kit", + "objc2-foundation", + "serde", + "serde_json", + "tempfile", +] + [[package]] name = "orbit-conversations" version = "0.0.1" @@ -3634,6 +3654,7 @@ dependencies = [ name = "orbit-plugin-decorum" version = "0.0.1" dependencies = [ + "block2", "log", "objc2", "objc2-foundation", @@ -5579,6 +5600,7 @@ dependencies = [ "gtk", "heck 0.5.0", "http 1.4.0", + "http-range", "jni", "libc", "log", diff --git a/agent-bridge/bunfig.toml b/agent-bridge/bunfig.toml new file mode 100644 index 000000000..37ff1f948 --- /dev/null +++ b/agent-bridge/bunfig.toml @@ -0,0 +1,2 @@ +[test] +preload = ["./src/__tests__/setup.ts"] diff --git a/agent-bridge/src/__tests__/browser-backward-compat.test.ts b/agent-bridge/src/__tests__/browser-backward-compat.test.ts new file mode 100644 index 000000000..9919012fb --- /dev/null +++ b/agent-bridge/src/__tests__/browser-backward-compat.test.ts @@ -0,0 +1,256 @@ +import { afterEach, beforeEach, describe, expect, it } from 'bun:test'; + +import { createBrowserMcpServer, getBrowserToolNames } from '../browser/browser-mcp-server.js'; +import { BrowserToolBridge } from '../browser/browser-tool-bridge.js'; + +interface McpToolResponse { + content: { type: 'text'; text: string }[]; + isError?: boolean; +} + +interface RegisteredTool { + handler: ( + args: Record, + context: Record + ) => Promise; +} + +interface McpServerInternal { + _registeredTools: Record; +} + +function getRegisteredToolNames(bridge: BrowserToolBridge): string[] { + const server = createBrowserMcpServer(bridge); + const internal = server.instance as unknown as McpServerInternal; + return Object.keys(internal._registeredTools); +} + +function getToolHandler(bridge: BrowserToolBridge, toolName: string): RegisteredTool['handler'] { + const server = createBrowserMcpServer(bridge); + const internal = server.instance as unknown as McpServerInternal; + const tool = internal._registeredTools[toolName]; + if (!tool) { + throw new Error(`Tool not registered: ${toolName}`); + } + return tool.handler; +} + +function getResponseText(result: McpToolResponse): string { + const first = result.content[0]; + if (!first) { + throw new Error('Expected response to have at least one content block'); + } + return first.text; +} + +interface ToolCase { + tool: string; + args: Record; +} + +describe('Browser backward compatibility', () => { + let bridge: BrowserToolBridge; + + beforeEach(() => { + bridge = new BrowserToolBridge(); + }); + + afterEach(() => { + bridge.dispose(); + }); + + it('registers all 38 desktop browser tools', () => { + const toolNames = getRegisteredToolNames(bridge); + expect(toolNames.length).toBe(38); + }); + + it('includes all original 13 tools from the pre-upgrade baseline', () => { + const original13 = [ + 'browser_open', + 'browser_close', + 'browser_navigate', + 'browser_back', + 'browser_forward', + 'browser_reload', + 'browser_click', + 'browser_type', + 'browser_get_text', + 'browser_get_html', + 'browser_screenshot', + 'browser_eval', + 'browser_console_logs', + ]; + + const toolNames = getRegisteredToolNames(bridge); + for (const name of original13) { + expect(toolNames).toContain(name); + } + }); + + it('exports prefixed tool names via getBrowserToolNames()', () => { + const prefixed = getBrowserToolNames(); + expect(prefixed).toContain('mcp__orbit-browser__browser_click'); + expect(prefixed).toContain('mcp__orbit-browser__browser_snapshot'); + expect(prefixed).toContain('mcp__orbit-browser__browser_runtime_info'); + expect(prefixed.length).toBe(38); + }); + + describe('4 element-targeting tools accept selector-only (no ref)', () => { + const elementTools: ToolCase[] = [ + { tool: 'browser_click', args: { selector: '#submit' } }, + { tool: 'browser_fill', args: { selector: '#email', value: 'test@example.com' } }, + { tool: 'browser_type', args: { selector: '#search', text: 'hello' } }, + { tool: 'browser_hover', args: { selector: '.menu-item' } }, + ]; + + for (const { tool, args } of elementTools) { + it(`${tool} accepts selector-only input`, async () => { + bridge.onToolRequest((request) => { + expect(request.toolInput.selector).toBe(args.selector); + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { ok: true }, + }); + }); + + const handler = getToolHandler(bridge, tool); + const result = await handler(args, {}); + expect(result.isError).toBeUndefined(); + }); + } + }); + + describe('element-targeting tools accept ref-only input (new path)', () => { + const elementTools: ToolCase[] = [ + { tool: 'browser_click', args: { ref: 'e1' } }, + { tool: 'browser_fill', args: { ref: 'e3', value: 'hello' } }, + { tool: 'browser_type', args: { ref: 'e5', text: 'world' } }, + { tool: 'browser_hover', args: { ref: 'e7' } }, + ]; + + for (const { tool, args } of elementTools) { + it(`${tool} accepts ref-only input`, async () => { + bridge.onToolRequest((request) => { + expect(request.toolInput.ref).toBe(args.ref); + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { ok: true }, + }); + }); + + const handler = getToolHandler(bridge, tool); + const result = await handler(args, {}); + expect(result.isError).toBeUndefined(); + }); + } + }); + + describe('non-element tools do not require ref or selector', () => { + const nonElementTools = [ + 'browser_navigate', + 'browser_back', + 'browser_forward', + 'browser_reload', + 'browser_close', + 'browser_eval', + 'browser_screenshot', + 'browser_get_url', + 'browser_get_title', + ]; + + for (const toolName of nonElementTools) { + it(`${toolName} works without ref or selector`, async () => { + bridge.onToolRequest((request) => { + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { ok: true }, + }); + }); + + const handler = getToolHandler(bridge, toolName); + + const args: Record = {}; + if (toolName === 'browser_navigate') { + args.url = 'https://example.com'; + } + if (toolName === 'browser_eval') { + args.script = 'return 1'; + } + + const result = await handler(args, {}); + expect(result.isError).toBeUndefined(); + }); + } + }); + + it('browser_type uses text param (append semantics)', async () => { + bridge.onToolRequest((request) => { + expect(request.toolInput).toMatchObject({ + ref: 'e1', + text: 'appended text', + }); + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { typed: true }, + }); + }); + + const handler = getToolHandler(bridge, 'browser_type'); + const result = await handler({ ref: 'e1', text: 'appended text' }, {}); + expect(result.isError).toBeUndefined(); + }); + + it('browser_fill uses value param (replace semantics)', async () => { + bridge.onToolRequest((request) => { + expect(request.toolInput).toMatchObject({ + ref: 'e2', + value: 'replaced text', + }); + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { filled: true }, + }); + }); + + const handler = getToolHandler(bridge, 'browser_fill'); + const result = await handler({ ref: 'e2', value: 'replaced text' }, {}); + expect(result.isError).toBeUndefined(); + }); + + it('browser_get_text works with no target (whole page)', async () => { + bridge.onToolRequest((request) => { + expect(request.toolInput).toEqual({}); + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: 'Full page text content', + }); + }); + + const handler = getToolHandler(bridge, 'browser_get_text'); + const result = await handler({}, {}); + expect(result.isError).toBeUndefined(); + expect(getResponseText(result)).toBe('Full page text content\nexecutionTarget: desktop'); + }); + + it('browser_get_html works with no target (whole page)', async () => { + bridge.onToolRequest((request) => { + expect(request.toolInput).toEqual({}); + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: '
Hello
', + }); + }); + + const handler = getToolHandler(bridge, 'browser_get_html'); + const result = await handler({}, {}); + expect(result.isError).toBeUndefined(); + expect(getResponseText(result)).toBe('
Hello
\nexecutionTarget: desktop'); + }); +}); diff --git a/agent-bridge/src/__tests__/browser-mcp-screenshot.test.ts b/agent-bridge/src/__tests__/browser-mcp-screenshot.test.ts new file mode 100644 index 000000000..128b46c0f --- /dev/null +++ b/agent-bridge/src/__tests__/browser-mcp-screenshot.test.ts @@ -0,0 +1,97 @@ +import { beforeEach, describe, expect, it, mock } from 'bun:test'; + +import { createBrowserMcpServer } from '../browser/browser-mcp-server.js'; + +import type { BrowserToolBridge } from '../browser/browser-tool-bridge.js'; + +interface McpToolResponse { + content: { type: 'text'; text: string }[]; + isError?: boolean; +} + +interface RegisteredTool { + handler: ( + args: Record, + context: Record + ) => Promise; +} + +interface McpServerInternal { + _registeredTools: Record; +} + +describe('Browser MCP screenshot tool', () => { + const sendRequestMock = mock( + (): Promise<{ filePath?: string | null; metadata: Record }> => + Promise.resolve({ + filePath: null, + metadata: {}, + }) + ); + + const bridge = { + sendRequest: sendRequestMock, + } as unknown as BrowserToolBridge; + + beforeEach(() => { + sendRequestMock.mockReset(); + }); + + function getScreenshotHandler(): RegisteredTool['handler'] { + const server = createBrowserMcpServer(bridge); + const internal = server.instance as unknown as McpServerInternal; + const tool = internal._registeredTools.browser_screenshot; + if (!tool) throw new Error('browser_screenshot tool not registered'); + return tool.handler; + } + + it('returns file path + metadata text when image file is available', async () => { + sendRequestMock.mockResolvedValue({ + filePath: '/tmp/orbit-screenshot-12345.jpg', + metadata: { url: 'https://example.com', captureMethod: 'native_wkwebview' }, + }); + + const handler = getScreenshotHandler(); + const result = await handler({}, {}); + + expect(result.isError).toBeUndefined(); + expect(result.content).toHaveLength(1); + const text = result.content[0]?.text ?? ''; + expect(text).toContain('Screenshot saved to: /tmp/orbit-screenshot-12345.jpg'); + expect(text).toContain('Use the Read tool to view this image.'); + expect(text).toContain( + 'Metadata: {"url":"https://example.com","captureMethod":"native_wkwebview"}' + ); + }); + + it('returns metadata-only text when file path is null', async () => { + sendRequestMock.mockResolvedValue({ + filePath: null, + metadata: { captureMethod: 'metadata_fallback', url: 'about:blank' }, + }); + + const handler = getScreenshotHandler(); + const result = await handler({}, {}); + + expect(result.isError).toBeUndefined(); + expect(result.content).toHaveLength(1); + const text = result.content[0]?.text ?? ''; + expect(text).toContain('metadata only'); + expect(text).toContain('Metadata: {"captureMethod":"metadata_fallback","url":"about:blank"}'); + }); + + it('returns error content when bridge call fails', async () => { + sendRequestMock.mockRejectedValue(new Error('bridge failed')); + + const handler = getScreenshotHandler(); + const result = await handler({}, {}); + + expect(result.isError).toBe(true); + expect(result.content).toEqual([ + { + type: 'text', + text: 'Failed: bridge failed', + }, + ]); + }); +}); diff --git a/agent-bridge/src/__tests__/browser-ref-epoch.test.ts b/agent-bridge/src/__tests__/browser-ref-epoch.test.ts new file mode 100644 index 000000000..dd2829b78 --- /dev/null +++ b/agent-bridge/src/__tests__/browser-ref-epoch.test.ts @@ -0,0 +1,205 @@ +import { afterEach, beforeEach, describe, expect, it } from 'bun:test'; + +import { createBrowserMcpServer } from '../browser/browser-mcp-server.js'; +import { BrowserToolBridge } from '../browser/browser-tool-bridge.js'; + +import type { McpToolRequest } from '../browser/types.js'; + +interface McpToolResponse { + content: { type: 'text'; text: string }[]; + isError?: boolean; +} + +interface RegisteredTool { + handler: ( + args: Record, + context: Record + ) => Promise; +} + +interface McpServerInternal { + _registeredTools: Record; +} + +interface SnapshotParsed { + epoch: number; + refCount: number; + snapshot: string; + totalElements: number; + emittedElements: number; + truncated: boolean; + url: string; + title: string; + durationMs: number; +} + +function getToolHandler(bridge: BrowserToolBridge, toolName: string): RegisteredTool['handler'] { + const server = createBrowserMcpServer(bridge); + const internal = server.instance as unknown as McpServerInternal; + const tool = internal._registeredTools[toolName]; + if (!tool) { + throw new Error(`Tool not registered: ${toolName}`); + } + return tool.handler; +} + +function getResponseText(result: McpToolResponse): string { + const first = result.content[0]; + if (!first) { + throw new Error('Expected response to have at least one content block'); + } + return first.text; +} + +describe('Browser ref epoch handling', () => { + let bridge: BrowserToolBridge; + + beforeEach(() => { + bridge = new BrowserToolBridge(); + }); + + afterEach(() => { + bridge.dispose(); + }); + + it('passes snapshot epoch through the MCP tool response', async () => { + bridge.onToolRequest((request) => { + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { + epoch: 7, + snapshot: '- button "Submit" [ref=e1]', + refCount: 1, + totalElements: 5, + emittedElements: 5, + truncated: false, + url: 'https://example.com', + title: 'Example', + durationMs: 12, + }, + }); + }); + + const handler = getToolHandler(bridge, 'browser_snapshot'); + const result = await handler({}, {}); + + expect(result.isError).toBeUndefined(); + const parsed = JSON.parse(getResponseText(result)) as SnapshotParsed; + expect(parsed).toMatchObject({ + epoch: 7, + refCount: 1, + snapshot: '- button "Submit" [ref=e1]', + }); + }); + + it('surfaces stale ref error from the runtime through browser_click', async () => { + bridge.onToolRequest((request) => { + bridge.handleResponse({ + requestId: request.requestId, + success: false, + error: 'Ref is stale (epoch 3, current 4). Call browser_snapshot again.', + }); + }); + + const handler = getToolHandler(bridge, 'browser_click'); + const result = await handler({ ref: 'e2' }, {}); + + expect(result.isError).toBe(true); + expect(getResponseText(result)).toContain('stale'); + expect(getResponseText(result)).toContain('browser_snapshot'); + }); + + it('surfaces stale ref error through browser_fill', async () => { + bridge.onToolRequest((request) => { + bridge.handleResponse({ + requestId: request.requestId, + success: false, + error: 'Ref is stale (epoch 1, current 2). Call browser_snapshot again.', + }); + }); + + const handler = getToolHandler(bridge, 'browser_fill'); + const result = await handler({ ref: 'e5', value: 'test@example.com' }, {}); + + expect(result.isError).toBe(true); + expect(getResponseText(result)).toContain('stale'); + }); + + it('returns incrementing epochs across consecutive snapshots', async () => { + let snapshotCount = 0; + + bridge.onToolRequest((request) => { + snapshotCount += 1; + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { + epoch: snapshotCount, + snapshot: `- document (snapshot ${String(snapshotCount)})`, + refCount: 0, + totalElements: 1, + emittedElements: 1, + truncated: false, + url: 'https://example.com', + title: 'Example', + durationMs: 5, + }, + }); + }); + + const handler = getToolHandler(bridge, 'browser_snapshot'); + + const result1 = await handler({}, {}); + const result2 = await handler({}, {}); + const result3 = await handler({}, {}); + + const parsed1 = JSON.parse(getResponseText(result1)) as SnapshotParsed; + const parsed2 = JSON.parse(getResponseText(result2)) as SnapshotParsed; + const parsed3 = JSON.parse(getResponseText(result3)) as SnapshotParsed; + + expect(parsed1.epoch).toBe(1); + expect(parsed2.epoch).toBe(2); + expect(parsed3.epoch).toBe(3); + }); + + it('epoch resets after navigation because snapshot is called fresh', async () => { + const requests: McpToolRequest[] = []; + + bridge.onToolRequest((request) => { + requests.push(request); + + if (request.toolName === 'browser_navigate') { + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { navigated: true }, + }); + } else if (request.toolName === 'browser_snapshot') { + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { + epoch: 1, + snapshot: '- document', + refCount: 0, + totalElements: 1, + emittedElements: 1, + truncated: false, + url: 'https://other.com', + title: '', + durationMs: 3, + }, + }); + } + }); + + const navHandler = getToolHandler(bridge, 'browser_navigate'); + const snapHandler = getToolHandler(bridge, 'browser_snapshot'); + + await navHandler({ url: 'https://other.com' }, {}); + await snapHandler({}, {}); + + expect(requests.map((r) => r.toolName)).toEqual(['browser_navigate', 'browser_snapshot']); + }); +}); diff --git a/agent-bridge/src/__tests__/browser-ref-flow.test.ts b/agent-bridge/src/__tests__/browser-ref-flow.test.ts new file mode 100644 index 000000000..f34fcd1a8 --- /dev/null +++ b/agent-bridge/src/__tests__/browser-ref-flow.test.ts @@ -0,0 +1,182 @@ +import { afterEach, beforeEach, describe, expect, it } from 'bun:test'; + +import { createBrowserMcpServer } from '../browser/browser-mcp-server.js'; +import { BrowserToolBridge } from '../browser/browser-tool-bridge.js'; + +import type { McpToolRequest } from '../browser/types.js'; + +interface McpToolResponse { + content: { type: 'text'; text: string }[]; + isError?: boolean; +} + +interface RegisteredTool { + handler: ( + args: Record, + context: Record + ) => Promise; +} + +interface McpServerInternal { + _registeredTools: Record; +} + +function getToolHandler(bridge: BrowserToolBridge, toolName: string): RegisteredTool['handler'] { + const server = createBrowserMcpServer(bridge); + const internal = server.instance as unknown as McpServerInternal; + const tool = internal._registeredTools[toolName]; + if (!tool) { + throw new Error(`Tool not registered: ${toolName}`); + } + return tool.handler; +} + +async function flushBridgeQueue(): Promise { + await Promise.resolve(); + await Promise.resolve(); +} + +describe('Browser ref flow', () => { + let bridge: BrowserToolBridge; + + beforeEach(() => { + bridge = new BrowserToolBridge(); + }); + + afterEach(() => { + bridge.dispose(); + }); + + it('serializes stateful requests so snapshot completes before click is emitted', async () => { + const requests: McpToolRequest[] = []; + + bridge.onToolRequest((request) => { + requests.push(request); + }); + + const snapshotPromise = bridge.sendRequest<{ epoch: number }>('browser_snapshot', {}); + const clickPromise = bridge.sendRequest<{ clicked: boolean }>('browser_click', { ref: 'e1' }); + + await flushBridgeQueue(); + expect(requests.map((request) => request.toolName)).toEqual(['browser_snapshot']); + + const firstRequest = requests[0]; + if (!firstRequest) { + throw new Error('Expected the snapshot request to be emitted first'); + } + bridge.handleResponse({ + requestId: firstRequest.requestId, + success: true, + result: { epoch: 1 }, + }); + + const snapshotResult = await snapshotPromise; + await flushBridgeQueue(); + expect(requests.map((request) => request.toolName)).toEqual([ + 'browser_snapshot', + 'browser_click', + ]); + + const secondRequest = requests[1]; + if (!secondRequest) { + throw new Error('Expected the click request to be emitted after snapshot completion'); + } + bridge.handleResponse({ + requestId: secondRequest.requestId, + success: true, + result: { clicked: true }, + }); + + const clickResult = await clickPromise; + expect(snapshotResult).toEqual({ epoch: 1 }); + expect(clickResult).toEqual({ clicked: true }); + }); + + it('allows read-only requests to be emitted concurrently', async () => { + const requests: McpToolRequest[] = []; + + bridge.onToolRequest((request) => { + requests.push(request); + }); + + const urlPromise = bridge.sendRequest<{ url: string }>('browser_get_url', {}); + const titlePromise = bridge.sendRequest<{ title: string }>('browser_get_title', {}); + + await flushBridgeQueue(); + expect(requests.map((request) => request.toolName)).toEqual([ + 'browser_get_url', + 'browser_get_title', + ]); + + const urlRequest = requests[0]; + const titleRequest = requests[1]; + if (!urlRequest || !titleRequest) { + throw new Error('Expected both read-only requests to be emitted immediately'); + } + bridge.handleResponse({ + requestId: urlRequest.requestId, + success: true, + result: { url: 'https://example.com' }, + }); + bridge.handleResponse({ + requestId: titleRequest.requestId, + success: true, + result: { title: 'Example Domain' }, + }); + + const urlResult = await urlPromise; + const titleResult = await titlePromise; + expect(urlResult).toEqual({ url: 'https://example.com' }); + expect(titleResult).toEqual({ title: 'Example Domain' }); + }); + + it('forwards ref and selector inputs through the MCP click tool', async () => { + const requests: McpToolRequest[] = []; + bridge.onToolRequest((request) => { + requests.push(request); + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { clicked: true }, + }); + }); + + const handler = getToolHandler(bridge, 'browser_click'); + const result = await handler({ ref: 'e12', selector: '#submit' }, {}); + + const emittedClickRequest = requests[0]; + if (!emittedClickRequest) { + throw new Error('Expected browser_click to emit a bridge request'); + } + expect(emittedClickRequest.toolInput).toEqual({ ref: 'e12', selector: '#submit' }); + expect(result.isError).toBeUndefined(); + expect(result.content).toEqual([ + { type: 'text', text: '{"clicked":true,"executionTarget":"desktop"}' }, + ]); + }); + + it('keeps browser_get_text target optional for backward compatibility', async () => { + const requests: McpToolRequest[] = []; + bridge.onToolRequest((request) => { + requests.push(request); + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: 'Example body text', + }); + }); + + const handler = getToolHandler(bridge, 'browser_get_text'); + const result = await handler({}, {}); + + const emittedGetTextRequest = requests[0]; + if (!emittedGetTextRequest) { + throw new Error('Expected browser_get_text to emit a bridge request'); + } + expect(emittedGetTextRequest.toolInput).toEqual({}); + expect(result.isError).toBeUndefined(); + expect(result.content).toEqual([ + { type: 'text', text: 'Example body text\nexecutionTarget: desktop' }, + ]); + }); +}); diff --git a/agent-bridge/src/__tests__/browser-snapshot-truncation.test.ts b/agent-bridge/src/__tests__/browser-snapshot-truncation.test.ts new file mode 100644 index 000000000..fa70c0f30 --- /dev/null +++ b/agent-bridge/src/__tests__/browser-snapshot-truncation.test.ts @@ -0,0 +1,210 @@ +import { afterEach, beforeEach, describe, expect, it } from 'bun:test'; + +import { createBrowserMcpServer } from '../browser/browser-mcp-server.js'; +import { BrowserToolBridge } from '../browser/browser-tool-bridge.js'; + +interface McpToolResponse { + content: { type: 'text'; text: string }[]; + isError?: boolean; +} + +interface RegisteredTool { + handler: ( + args: Record, + context: Record + ) => Promise; +} + +interface McpServerInternal { + _registeredTools: Record; +} + +interface SnapshotParsed { + epoch: number; + snapshot: string; + refCount: number; + totalElements: number; + emittedElements: number; + truncated: boolean; + url: string; + title: string; + durationMs: number; + executionTarget?: string; +} + +function getToolHandler(bridge: BrowserToolBridge, toolName: string): RegisteredTool['handler'] { + const server = createBrowserMcpServer(bridge); + const internal = server.instance as unknown as McpServerInternal; + const tool = internal._registeredTools[toolName]; + if (!tool) { + throw new Error(`Tool not registered: ${toolName}`); + } + return tool.handler; +} + +function getResponseText(result: McpToolResponse): string { + const first = result.content[0]; + if (!first) { + throw new Error('Expected response to have at least one content block'); + } + return first.text; +} + +describe('Browser snapshot truncation', () => { + let bridge: BrowserToolBridge; + + beforeEach(() => { + bridge = new BrowserToolBridge(); + }); + + afterEach(() => { + bridge.dispose(); + }); + + it('includes truncation stats when snapshot is truncated', async () => { + bridge.onToolRequest((request) => { + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { + epoch: 1, + snapshot: '- document\n - main\n - ... (truncated)', + refCount: 15, + totalElements: 5000, + emittedElements: 200, + truncated: true, + url: 'https://heavy-page.example.com', + title: 'Heavy Page', + durationMs: 450, + }, + }); + }); + + const handler = getToolHandler(bridge, 'browser_snapshot'); + const result = await handler({}, {}); + + expect(result.isError).toBeUndefined(); + const parsed = JSON.parse(getResponseText(result)) as SnapshotParsed; + + expect(parsed.truncated).toBe(true); + expect(parsed.totalElements).toBe(5000); + expect(parsed.emittedElements).toBe(200); + expect(typeof parsed.totalElements).toBe('number'); + expect(typeof parsed.emittedElements).toBe('number'); + }); + + it('reports truncated=false when snapshot is complete', async () => { + bridge.onToolRequest((request) => { + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { + epoch: 1, + snapshot: '- document\n - button "Submit" [ref=e1]', + refCount: 1, + totalElements: 3, + emittedElements: 3, + truncated: false, + url: 'https://simple.example.com', + title: 'Simple', + durationMs: 8, + }, + }); + }); + + const handler = getToolHandler(bridge, 'browser_snapshot'); + const result = await handler({}, {}); + + const parsed = JSON.parse(getResponseText(result)) as SnapshotParsed; + expect(parsed.truncated).toBe(false); + expect(parsed.totalElements).toBe(3); + expect(parsed.emittedElements).toBe(3); + }); + + it('includes durationMs in snapshot response', async () => { + bridge.onToolRequest((request) => { + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { + epoch: 2, + snapshot: '- document', + refCount: 0, + totalElements: 1, + emittedElements: 1, + truncated: false, + url: 'https://example.com', + title: 'Example', + durationMs: 42, + }, + }); + }); + + const handler = getToolHandler(bridge, 'browser_snapshot'); + const result = await handler({}, {}); + + const parsed = JSON.parse(getResponseText(result)) as SnapshotParsed; + expect(parsed.durationMs).toBe(42); + }); + + it('snapshot response payload stays under 500KB', async () => { + const largeSnapshot = '- node [ref=e1]\n'.repeat(10000); + + bridge.onToolRequest((request) => { + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { + epoch: 1, + snapshot: largeSnapshot, + refCount: 10000, + totalElements: 10000, + emittedElements: 10000, + truncated: false, + url: 'https://example.com', + title: '', + durationMs: 500, + }, + }); + }); + + const handler = getToolHandler(bridge, 'browser_snapshot'); + const result = await handler({}, {}); + + const responseText = getResponseText(result); + const byteLength = new TextEncoder().encode(responseText).length; + expect(byteLength).toBeLessThan(500000); + }); + + it('passes interactive, cursor, and compact flags to the bridge request', async () => { + let capturedInput: Record | null = null; + + bridge.onToolRequest((request) => { + capturedInput = request.toolInput; + bridge.handleResponse({ + requestId: request.requestId, + success: true, + result: { + epoch: 1, + snapshot: '- document', + refCount: 0, + totalElements: 1, + emittedElements: 1, + truncated: false, + url: 'https://example.com', + title: '', + durationMs: 3, + }, + }); + }); + + const handler = getToolHandler(bridge, 'browser_snapshot'); + await handler({ interactive: false, cursor: true, compact: true }, {}); + + expect(capturedInput).toMatchObject({ + interactive: false, + cursor: true, + compact: true, + }); + }); +}); diff --git a/agent-bridge/src/__tests__/browser-timeout.test.ts b/agent-bridge/src/__tests__/browser-timeout.test.ts new file mode 100644 index 000000000..da678cbee --- /dev/null +++ b/agent-bridge/src/__tests__/browser-timeout.test.ts @@ -0,0 +1,72 @@ +import { afterEach, beforeEach, describe, expect, it } from 'bun:test'; + +import { BrowserToolBridge, getBrowserToolTimeoutMs } from '../browser/browser-tool-bridge.js'; + +describe('Browser tool bridge timeouts', () => { + let bridge: BrowserToolBridge; + + beforeEach(() => { + bridge = new BrowserToolBridge(); + }); + + afterEach(() => { + bridge.dispose(); + }); + + async function expectTimeout(request: Promise, message: string): Promise { + try { + await request; + throw new Error(`Expected request to reject with: ${message}`); + } catch (error) { + if (!(error instanceof Error)) { + throw error; + } + expect(error.message).toBe(message); + } + } + + it('returns the 15s timeout budget for browser_snapshot', () => { + expect(getBrowserToolTimeoutMs('browser_snapshot', {})).toBe(15000); + }); + + it('returns the 10s timeout budget for browser_click', () => { + expect(getBrowserToolTimeoutMs('browser_click', { ref: 'e1' })).toBe(10000); + }); + + it('returns the 30s timeout budget for browser_navigate', () => { + expect(getBrowserToolTimeoutMs('browser_navigate', { url: 'https://example.com' })).toBe(30000); + }); + + it('uses the configured timeout for browser_wait_for_selector', () => { + expect( + getBrowserToolTimeoutMs('browser_wait_for_selector', { + selector: '#ready', + state: 'visible', + timeout: 42000, + }) + ).toBe(42000); + }); + + it('clamps browser_wait_for_url timeout at 120s', () => { + expect( + getBrowserToolTimeoutMs('browser_wait_for_url', { + url: '/dashboard/i', + timeout: 999999, + }) + ).toBe(120000); + }); + + it('rejects with the tool name and elapsed time when a short wait request times out', async () => { + bridge.onToolRequest(() => { + // Intentionally never responds. + }); + + await expectTimeout( + bridge.sendRequest('browser_wait_for_selector', { + selector: '#ready', + timeout: 25, + }), + 'browser_wait_for_selector timed out after 25ms' + ); + }); +}); diff --git a/agent-bridge/src/__tests__/ios-e2e.test.ts b/agent-bridge/src/__tests__/ios-e2e.test.ts new file mode 100644 index 000000000..1814e4668 --- /dev/null +++ b/agent-bridge/src/__tests__/ios-e2e.test.ts @@ -0,0 +1,144 @@ +/** + * iOS Simulator end-to-end tests. + * + * These tests require macOS + Xcode + Appium to be installed. + * They auto-skip on non-macOS platforms and when dependencies are unavailable. + * + * Run: cd agent-bridge && bun test src/__tests__/ios-e2e.test.ts + */ + +import { describe, expect, it } from 'bun:test'; + +import { canEnableIOS, IOSService } from '../ios/ios-service.js'; + +import type { IOSManagerLike } from '../ios/types.js'; + +const SKIP_REASON_NON_MACOS = 'iOS E2E tests require macOS (skipped on this platform)'; +const SKIP_REASON_MISSING_DEPS = + 'iOS E2E tests require Xcode and Appium (set ORBIT_ENABLE_IOS_TOOLS=1 and install dependencies)'; + +async function shouldSkip(): Promise { + if (process.platform !== 'darwin') { + return SKIP_REASON_NON_MACOS; + } + + const enabled = await canEnableIOS({ + env: { ...process.env, ORBIT_ENABLE_IOS_TOOLS: '1' }, + }); + + if (!enabled) { + return SKIP_REASON_MISSING_DEPS; + } + + return null; +} + +describe('iOS Simulator E2E', () => { + it('auto-skip gate: checks for macOS + Xcode + Appium', async () => { + const skipReason = await shouldSkip(); + + if (skipReason !== null) { + console.warn(`[ios-e2e] ${skipReason}`); + expect(skipReason).toBeString(); + return; + } + + expect(process.platform).toBe('darwin'); + }); + + it('canEnableIOS returns false on non-darwin platform', async () => { + const result = await canEnableIOS({ platform: 'linux' }); + expect(result).toBe(false); + }); + + it('canEnableIOS returns false when ORBIT_ENABLE_IOS_TOOLS is unset', async () => { + const result = await canEnableIOS({ platform: 'darwin', env: {} }); + expect(result).toBe(false); + }); + + it('canEnableIOS returns false when xcrun is not available', async () => { + const result = await canEnableIOS({ + platform: 'darwin', + env: { ORBIT_ENABLE_IOS_TOOLS: '1' }, + execFile: () => Promise.reject(new Error('not found')), + }); + expect(result).toBe(false); + }); + + it('canEnableIOS returns true when all dependencies are available', async () => { + const result = await canEnableIOS({ + platform: 'darwin', + env: { ORBIT_ENABLE_IOS_TOOLS: '1' }, + execFile: () => Promise.resolve({ stdout: '', stderr: '' }), + }); + expect(result).toBe(true); + }); + + it('full lifecycle: launch -> snapshot -> tap -> screenshot -> close (live)', async () => { + const skipReason = await shouldSkip(); + if (skipReason !== null) { + console.warn(`[ios-e2e] Skipping live test: ${skipReason}`); + return; + } + + const service = new IOSService(); + let manager: IOSManagerLike | undefined; + + try { + manager = await service.acquire('e2e-session'); + expect(service.state).toBe('ready'); + expect(service.hasLease('e2e-session')).toBe(true); + + const launchResult = manager.getLaunchResult(); + if (launchResult === null) { + throw new Error('Expected launch result to be non-null after acquire'); + } + expect(launchResult.udid).toBeString(); + expect(launchResult.name).toBeString(); + + await manager.navigate('https://example.com'); + + const snapshot = await manager.snapshot({ interactive: true }); + expect(snapshot.epoch).toBeGreaterThanOrEqual(1); + expect(snapshot.snapshot).toContain('document'); + expect(typeof snapshot.refCount).toBe('number'); + + const screenshot = await manager.screenshot(); + expect(screenshot.image).toBeString(); + expect(screenshot.image.length).toBeGreaterThan(0); + expect(screenshot.format).toBe('png'); + } finally { + await service.release('e2e-session'); + expect(service.state).toBe('idle'); + expect(service.leaseCount).toBe(0); + } + }); + + it('multi-session lease sharing (live)', async () => { + const skipReason = await shouldSkip(); + if (skipReason !== null) { + console.warn(`[ios-e2e] Skipping live test: ${skipReason}`); + return; + } + + const service = new IOSService(); + + try { + const managerA = await service.acquire('session-a'); + const managerB = await service.acquire('session-b'); + + expect(service.leaseCount).toBe(2); + expect(managerA).toBe(managerB); + + await service.release('session-a'); + expect(service.state).toBe('ready'); + expect(service.leaseCount).toBe(1); + + await service.release('session-b'); + expect(service.state).toBe('idle'); + expect(service.leaseCount).toBe(0); + } finally { + await service.dispose(); + } + }); +}); diff --git a/agent-bridge/src/__tests__/ios-manager-lifecycle.test.ts b/agent-bridge/src/__tests__/ios-manager-lifecycle.test.ts new file mode 100644 index 000000000..34f6ee5ee --- /dev/null +++ b/agent-bridge/src/__tests__/ios-manager-lifecycle.test.ts @@ -0,0 +1,371 @@ +import { describe, expect, it } from 'bun:test'; + +import { IOSManager } from '../ios/ios-manager.js'; + +import type { BrowserConsoleLogEntry, SnapshotResponse } from '../browser/types.js'; +import type { + IOSConsoleLogLevel, + IOSAppiumProcess, + IOSAutomationSession, + IOSDevice, + IOSScreenshotResult, + IOSManagerDependencies, + IOSScrollDirection, + IOSScrollResult, + IOSSnapshotOptions, + IOSSwipeDirection, + IOSTargetInput, + IOSWaitForSelectorState, +} from '../ios/types.js'; + +interface Deferred { + promise: Promise; + resolve: (value: T) => void; + reject: (reason?: unknown) => void; +} + +interface RuntimeHarness { + appium: FakeAppiumProcess; + automationSession: FakeAutomationSession; + dependencies: IOSManagerDependencies; + shutdowns: string[]; +} + +const TEST_DEVICE: IOSDevice = { + name: 'iPhone 16 Pro', + udid: 'TEST-UDID-1', + state: 'Shutdown', + runtime: 'iOS 18.2', + isAvailable: true, +}; + +class FakeAutomationSession implements IOSAutomationSession { + deleteCount = 0; + + deleteSession(): Promise { + this.deleteCount += 1; + return Promise.resolve(); + } + + navigate(): Promise { + return Promise.resolve(); + } + + back(): Promise { + return Promise.resolve(); + } + + forward(): Promise { + return Promise.resolve(); + } + + reload(): Promise { + return Promise.resolve(); + } + + snapshot(options: IOSSnapshotOptions): Promise { + void options; + return Promise.resolve({ + epoch: 1, + snapshot: '- document', + refCount: 0, + totalElements: 0, + emittedElements: 0, + truncated: false, + url: 'about:blank', + title: '', + durationMs: 0, + }); + } + + getText(target: IOSTargetInput): Promise { + void target; + return Promise.resolve(''); + } + + getHtml(target: IOSTargetInput, outer?: boolean): Promise { + void target; + void outer; + return Promise.resolve(''); + } + + screenshot(): Promise { + return Promise.resolve({ image: '', format: 'png' }); + } + + tap(target: IOSTargetInput): Promise { + void target; + return Promise.resolve(); + } + + fill(target: IOSTargetInput, value: string): Promise { + void target; + void value; + return Promise.resolve(); + } + + type(target: IOSTargetInput, text: string): Promise { + void target; + void text; + return Promise.resolve(); + } + + select(target: IOSTargetInput, values: string[]): Promise { + void target; + void values; + return Promise.resolve(); + } + + check(target: IOSTargetInput): Promise { + void target; + return Promise.resolve(); + } + + uncheck(target: IOSTargetInput): Promise { + void target; + return Promise.resolve(); + } + + swipe(direction: IOSSwipeDirection, target?: IOSTargetInput, duration?: number): Promise { + void direction; + void target; + void duration; + return Promise.resolve(); + } + + scroll(direction: IOSScrollDirection, amount?: number): Promise { + void direction; + void amount; + return Promise.resolve({ scrollX: 0, scrollY: 0 }); + } + + evaluate(script: string): Promise { + void script; + return Promise.resolve(null); + } + + waitForSelector( + selector: string, + state?: IOSWaitForSelectorState, + timeout?: number + ): Promise { + void selector; + void state; + void timeout; + return Promise.resolve(); + } + + consoleLogs(level?: IOSConsoleLogLevel): Promise { + void level; + return Promise.resolve([]); + } +} + +class FakeAppiumProcess implements IOSAppiumProcess { + readonly port: number; + readonly pid: number; + + terminateSignals: ('SIGTERM' | 'SIGKILL')[] = []; + waitTimeouts: number[] = []; + + private exitCode: number | null = null; + private readonly listeners = new Set<(code: number | null) => void>(); + private readonly exitOnSigterm: boolean; + + constructor(options: { port?: number; pid?: number; exitOnSigterm?: boolean } = {}) { + this.port = options.port ?? 4723; + this.pid = options.pid ?? 4242; + this.exitOnSigterm = options.exitOnSigterm ?? true; + } + + onExit(callback: (code: number | null) => void): () => void { + this.listeners.add(callback); + return () => { + this.listeners.delete(callback); + }; + } + + terminate(signal: 'SIGTERM' | 'SIGKILL'): Promise { + this.terminateSignals.push(signal); + + if (signal === 'SIGTERM' && this.exitOnSigterm) { + this.simulateExit(0); + } + + if (signal === 'SIGKILL') { + this.simulateExit(9); + } + + return Promise.resolve(); + } + + waitForExit(timeoutMs: number): Promise { + this.waitTimeouts.push(timeoutMs); + return Promise.resolve(this.exitCode); + } + + simulateExit(code: number): void { + this.exitCode = code; + for (const listener of this.listeners) { + listener(code); + } + } +} + +function createRuntimeHarness( + options: { + exitOnSigterm?: boolean; + bootDeferred?: Deferred; + bootRejectOnAbort?: boolean; + } = {} +): RuntimeHarness { + const appium = new FakeAppiumProcess({ + exitOnSigterm: options.exitOnSigterm, + }); + const automationSession = new FakeAutomationSession(); + const shutdowns: string[] = []; + + const dependencies: IOSManagerDependencies = { + listDevices(): Promise { + return Promise.resolve([TEST_DEVICE]); + }, + async bootDevice(_device: IOSDevice, signal: AbortSignal): Promise { + if (options.bootRejectOnAbort) { + await new Promise((_resolve, reject) => { + const abortHandler = (): void => { + reject(new Error('IOSManager launch aborted')); + }; + + if (signal.aborted) { + abortHandler(); + return; + } + + signal.addEventListener('abort', abortHandler, { once: true }); + }); + return; + } + + if (options.bootDeferred !== undefined) { + await new Promise((resolve, reject) => { + const abortHandler = (): void => { + reject(new Error('IOSManager launch aborted')); + }; + + if (signal.aborted) { + abortHandler(); + return; + } + + signal.addEventListener('abort', abortHandler, { once: true }); + void options.bootDeferred?.promise.then( + () => { + signal.removeEventListener('abort', abortHandler); + resolve(); + }, + (error: unknown) => { + signal.removeEventListener('abort', abortHandler); + reject(error instanceof Error ? error : new Error(String(error))); + } + ); + }); + } + }, + shutdownDevice(device: IOSDevice): Promise { + shutdowns.push(device.udid); + return Promise.resolve(); + }, + startAppium(): Promise { + return Promise.resolve(appium); + }, + createAutomationSession(): Promise { + return Promise.resolve(automationSession); + }, + }; + + return { + appium, + automationSession, + dependencies, + shutdowns, + }; +} + +async function expectErrorMessage(promise: Promise, message: string): Promise { + try { + await promise; + throw new Error(`Expected promise to reject with: ${message}`); + } catch (error) { + if (!(error instanceof Error)) { + throw error; + } + expect(error.message).toBe(message); + } +} + +describe('IOSManager lifecycle', () => { + it('launches, closes, and disposes with idempotent cleanup', async () => { + const runtime = createRuntimeHarness(); + const manager = new IOSManager({ dependencies: runtime.dependencies }); + + const launchResult = await manager.launch('iPhone 16 Pro'); + + expect(launchResult).toEqual({ + udid: TEST_DEVICE.udid, + name: TEST_DEVICE.name, + runtime: TEST_DEVICE.runtime, + appiumPort: 4723, + }); + expect(manager.state).toBe('ready'); + + await manager.close(); + expect(manager.state).toBe('idle'); + expect(runtime.automationSession.deleteCount).toBe(1); + expect(runtime.appium.terminateSignals).toEqual(['SIGTERM']); + expect(runtime.shutdowns).toEqual([TEST_DEVICE.udid]); + + await manager.close(); + expect(runtime.automationSession.deleteCount).toBe(1); + + await manager.dispose(); + expect(manager.state).toBe('disposed'); + + await expectErrorMessage(manager.listDevices(), 'IOSManager disposed'); + }); + + it('escalates to SIGKILL if Appium does not exit after SIGTERM', async () => { + const runtime = createRuntimeHarness({ exitOnSigterm: false }); + const manager = new IOSManager({ dependencies: runtime.dependencies }); + + await manager.launch(); + await manager.close(); + + expect(runtime.appium.terminateSignals).toEqual(['SIGTERM', 'SIGKILL']); + }); + + it('aborts an in-flight launch when close is called', async () => { + const runtime = createRuntimeHarness({ bootRejectOnAbort: true }); + const manager = new IOSManager({ dependencies: runtime.dependencies }); + + const launchPromise = manager.launch(); + await Promise.resolve(); + + expect(manager.state).toBe('launching'); + + await manager.close(); + await expectErrorMessage(launchPromise, 'IOSManager launch aborted'); + expect(manager.state).toBe('idle'); + }); + + it('reports the exact Appium crash message after an unexpected exit', async () => { + const runtime = createRuntimeHarness(); + const manager = new IOSManager({ dependencies: runtime.dependencies }); + + await manager.launch(); + runtime.appium.simulateExit(11); + + expect(() => { + manager.assertUsable(); + }).toThrow('Appium process exited unexpectedly (code 11). Call ios_launch to restart.'); + }); +}); diff --git a/agent-bridge/src/__tests__/ios-mcp-server.test.ts b/agent-bridge/src/__tests__/ios-mcp-server.test.ts new file mode 100644 index 000000000..800722df5 --- /dev/null +++ b/agent-bridge/src/__tests__/ios-mcp-server.test.ts @@ -0,0 +1,327 @@ +import { beforeEach, describe, expect, it } from 'bun:test'; + +import { createIOSMcpServer } from '../ios/ios-mcp-server.js'; + +import type { SnapshotResponse } from '../browser/types.js'; +import type { IOSService } from '../ios/ios-service.js'; +import type { + IOSConsoleLogsResult, + IOSDevice, + IOSLaunchResult, + IOSManagerLike, + IOSScrollResult, + IOSScreenshotResult, + IOSTargetInput, +} from '../ios/types.js'; + +interface McpToolResponse { + content: { type: 'text'; text: string }[]; + isError?: boolean; +} + +interface RegisteredTool { + handler: ( + args: Record, + context: Record + ) => Promise; +} + +interface McpServerInternal { + _registeredTools: Record; +} + +const TEST_DEVICE: IOSDevice = { + name: 'iPhone 16 Pro', + udid: 'TEST-UDID-1', + state: 'Booted', + runtime: 'iOS 18.2', + isAvailable: true, +}; + +const TEST_LAUNCH_RESULT: IOSLaunchResult = { + udid: TEST_DEVICE.udid, + name: TEST_DEVICE.name, + runtime: TEST_DEVICE.runtime, + appiumPort: 4723, +}; + +const TEST_SNAPSHOT: SnapshotResponse = { + epoch: 2, + snapshot: '- button "Continue" [ref=e12]', + refCount: 1, + totalElements: 5, + emittedElements: 5, + truncated: false, + url: 'https://example.com', + title: 'Example', + durationMs: 12, +}; + +const TEST_SCREENSHOT: IOSScreenshotResult = { + image: + 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO3ZyXcAAAAASUVORK5CYII=', + format: 'png', +}; + +class FakeIOSManager implements IOSManagerLike { + readonly state = 'ready'; + + lastTapTarget: IOSTargetInput | null = null; + lastSwipe: { + direction: string; + target?: IOSTargetInput; + duration?: number; + } | null = null; + lastSnapshotArgs: Record | null = null; + + listDevices(): Promise { + return Promise.resolve([TEST_DEVICE]); + } + + launch(): Promise { + return Promise.resolve(TEST_LAUNCH_RESULT); + } + + navigate(): Promise<{ success: true }> { + return Promise.resolve({ success: true }); + } + + back(): Promise<{ success: true }> { + return Promise.resolve({ success: true }); + } + + forward(): Promise<{ success: true }> { + return Promise.resolve({ success: true }); + } + + reload(): Promise<{ success: true }> { + return Promise.resolve({ success: true }); + } + + snapshot(options: Record): Promise { + this.lastSnapshotArgs = options; + return Promise.resolve(TEST_SNAPSHOT); + } + + getText(): Promise<{ text: string }> { + return Promise.resolve({ text: 'Continue' }); + } + + getHtml(): Promise<{ html: string }> { + return Promise.resolve({ html: '' }); + } + + screenshot(): Promise { + return Promise.resolve(TEST_SCREENSHOT); + } + + tap(target: IOSTargetInput): Promise<{ tapped: true }> { + this.lastTapTarget = target; + return Promise.resolve({ tapped: true }); + } + + fill(): Promise<{ filled: true }> { + return Promise.resolve({ filled: true }); + } + + type(): Promise<{ typed: true }> { + return Promise.resolve({ typed: true }); + } + + select(): Promise<{ selected: true }> { + return Promise.resolve({ selected: true }); + } + + check(): Promise<{ checked: true }> { + return Promise.resolve({ checked: true }); + } + + uncheck(): Promise<{ unchecked: true }> { + return Promise.resolve({ unchecked: true }); + } + + swipe(direction: string, target?: IOSTargetInput, duration?: number): Promise<{ swiped: true }> { + this.lastSwipe = { direction, target, duration }; + return Promise.resolve({ swiped: true }); + } + + scroll(): Promise { + return Promise.resolve({ scrollX: 0, scrollY: 320 }); + } + + evaluate(): Promise<{ result: unknown }> { + return Promise.resolve({ result: { ok: true } }); + } + + waitForSelector(): Promise<{ matched: true }> { + return Promise.resolve({ matched: true }); + } + + consoleLogs(): Promise { + return Promise.resolve({ logs: [] }); + } + + close(): Promise { + return Promise.resolve(); + } + + dispose(): Promise { + return Promise.resolve(); + } + + assertUsable(): void { + // no-op + } + + getLaunchResult(): IOSLaunchResult | null { + return TEST_LAUNCH_RESULT; + } +} + +class FakeIOSService { + hasLeaseValue = true; + listDevicesCalls = 0; + releaseCalls: string[] = []; + acquireCalls: { sessionId: string; device?: string }[] = []; + manager: FakeIOSManager; + + constructor(manager: FakeIOSManager) { + this.manager = manager; + } + + listDevices(): Promise { + this.listDevicesCalls += 1; + return Promise.resolve([TEST_DEVICE]); + } + + acquire(sessionId: string, device?: string): Promise { + this.acquireCalls.push({ sessionId, device }); + this.hasLeaseValue = true; + return Promise.resolve(this.manager); + } + + getManager(): Promise { + if (!this.hasLeaseValue) { + return Promise.reject(new Error('No iOS lease for session')); + } + return Promise.resolve(this.manager); + } + + hasLease(): boolean { + return this.hasLeaseValue; + } + + release(sessionId: string): Promise { + this.releaseCalls.push(sessionId); + this.hasLeaseValue = false; + return Promise.resolve(); + } +} + +function getToolHandler(service: IOSService, toolName: string): RegisteredTool['handler'] { + const server = createIOSMcpServer(service, 'session-1'); + const internal = server.instance as unknown as McpServerInternal; + const tool = internal._registeredTools[toolName]; + if (!tool) { + throw new Error(`Tool not registered: ${toolName}`); + } + return tool.handler; +} + +function parseJsonContent(result: McpToolResponse): unknown { + const text = result.content[0]?.text; + if (typeof text !== 'string') { + throw new Error('Expected MCP text content'); + } + return JSON.parse(text) as unknown; +} + +describe('iOS MCP server', () => { + let manager: FakeIOSManager; + let service: FakeIOSService; + + beforeEach(() => { + manager = new FakeIOSManager(); + service = new FakeIOSService(manager); + }); + + it('returns available devices without requiring a lease', async () => { + const handler = getToolHandler(service as unknown as IOSService, 'ios_device_list'); + const result = await handler({}, {}); + + expect(result.isError).toBeUndefined(); + expect(parseJsonContent(result)).toEqual({ + devices: [TEST_DEVICE], + executionTarget: 'ios', + }); + expect(service.listDevicesCalls).toBe(1); + }); + + it('launches iOS and returns the simulator metadata', async () => { + const handler = getToolHandler(service as unknown as IOSService, 'ios_launch'); + const result = await handler({ device: 'iPhone 16 Pro' }, {}); + + expect(result.isError).toBeUndefined(); + expect(parseJsonContent(result)).toEqual({ ...TEST_LAUNCH_RESULT, executionTarget: 'ios' }); + expect(service.acquireCalls).toEqual([{ sessionId: 'session-1', device: 'iPhone 16 Pro' }]); + }); + + it('returns the same snapshot shape as desktop snapshot tools', async () => { + const handler = getToolHandler(service as unknown as IOSService, 'ios_snapshot'); + const result = await handler({ interactive: false, compact: true }, {}); + + expect(result.isError).toBeUndefined(); + expect(parseJsonContent(result)).toEqual({ ...TEST_SNAPSHOT, executionTarget: 'ios' }); + expect(manager.lastSnapshotArgs).toEqual({ + interactive: false, + cursor: false, + compact: true, + }); + }); + + it('routes ios_tap through the manager using the provided ref target', async () => { + const handler = getToolHandler(service as unknown as IOSService, 'ios_tap'); + const result = await handler({ ref: 'e12' }, {}); + + expect(result.isError).toBeUndefined(); + expect(parseJsonContent(result)).toEqual({ tapped: true, executionTarget: 'ios' }); + expect(manager.lastTapTarget).toEqual({ ref: 'e12' }); + }); + + it('returns base64 PNG screenshot payloads', async () => { + const handler = getToolHandler(service as unknown as IOSService, 'ios_screenshot'); + const result = await handler({}, {}); + + expect(result.isError).toBeUndefined(); + expect(parseJsonContent(result)).toEqual({ ...TEST_SCREENSHOT, executionTarget: 'ios' }); + }); + + it('passes default swipe duration and optional target through the manager', async () => { + const handler = getToolHandler(service as unknown as IOSService, 'ios_swipe'); + const result = await handler( + { + direction: 'up', + target: { selector: '#feed' }, + }, + {} + ); + + expect(result.isError).toBeUndefined(); + expect(parseJsonContent(result)).toEqual({ swiped: true, executionTarget: 'ios' }); + expect(manager.lastSwipe).toEqual({ + direction: 'up', + target: { selector: '#feed' }, + duration: 300, + }); + }); + + it('returns exact no-lease errors without a wrapper prefix', async () => { + service.hasLeaseValue = false; + + const handler = getToolHandler(service as unknown as IOSService, 'ios_snapshot'); + const result = await handler({}, {}); + + expect(result.isError).toBe(true); + expect(result.content).toEqual([{ type: 'text', text: 'No iOS lease for session' }]); + }); +}); diff --git a/agent-bridge/src/__tests__/ios-platform-gate.test.ts b/agent-bridge/src/__tests__/ios-platform-gate.test.ts new file mode 100644 index 000000000..663c2a137 --- /dev/null +++ b/agent-bridge/src/__tests__/ios-platform-gate.test.ts @@ -0,0 +1,68 @@ +import { describe, expect, it } from 'bun:test'; + +import { canEnableIOS } from '../ios/ios-service.js'; + +describe('canEnableIOS', () => { + it('returns false on non-macOS without running prerequisite checks', async () => { + let calls = 0; + + const enabled = await canEnableIOS({ + platform: 'linux', + env: { ORBIT_ENABLE_IOS_TOOLS: '1' }, + execFile: (): Promise<{ stdout: string; stderr: string }> => { + calls += 1; + return Promise.resolve({ stdout: '', stderr: '' }); + }, + }); + + expect(enabled).toBe(false); + expect(calls).toBe(0); + }); + + it('returns false when the iOS feature flag is disabled', async () => { + let calls = 0; + + const enabled = await canEnableIOS({ + platform: 'darwin', + env: {}, + execFile: (): Promise<{ stdout: string; stderr: string }> => { + calls += 1; + return Promise.resolve({ stdout: '', stderr: '' }); + }, + }); + + expect(enabled).toBe(false); + expect(calls).toBe(0); + }); + + it('returns false when prerequisite checks fail', async () => { + const enabled = await canEnableIOS({ + platform: 'darwin', + env: { ORBIT_ENABLE_IOS_TOOLS: '1' }, + execFile: (): Promise<{ stdout: string; stderr: string }> => { + return Promise.reject(new Error('missing dependency')); + }, + }); + + expect(enabled).toBe(false); + }); + + it('returns true on macOS when the flag is enabled and prerequisites exist', async () => { + const commands: string[] = []; + + const enabled = await canEnableIOS({ + platform: 'darwin', + env: { ORBIT_ENABLE_IOS_TOOLS: '1' }, + execFile: ( + file: string, + args: readonly string[] + ): Promise<{ stdout: string; stderr: string }> => { + commands.push([file, ...args].join(' ')); + return Promise.resolve({ stdout: '', stderr: '' }); + }, + }); + + expect(enabled).toBe(true); + expect(commands).toEqual(['xcrun simctl help', 'which appium']); + }); +}); diff --git a/agent-bridge/src/__tests__/ios-service-lease.test.ts b/agent-bridge/src/__tests__/ios-service-lease.test.ts new file mode 100644 index 000000000..e85cad519 --- /dev/null +++ b/agent-bridge/src/__tests__/ios-service-lease.test.ts @@ -0,0 +1,402 @@ +import { describe, expect, it } from 'bun:test'; + +import { IOSManager } from '../ios/ios-manager.js'; +import { IOSService } from '../ios/ios-service.js'; + +import type { BrowserConsoleLogEntry, SnapshotResponse } from '../browser/types.js'; +import type { + IOSConsoleLogLevel, + IOSAppiumProcess, + IOSAutomationSession, + IOSDevice, + IOSScreenshotResult, + IOSManagerDependencies, + IOSScrollDirection, + IOSScrollResult, + IOSSnapshotOptions, + IOSSwipeDirection, + IOSTargetInput, + IOSWaitForSelectorState, +} from '../ios/types.js'; + +interface Deferred { + promise: Promise; + resolve: (value: T) => void; + reject: (reason?: unknown) => void; +} + +interface RuntimeHarness { + appium: FakeAppiumProcess; + automationSession: FakeAutomationSession; + dependencies: IOSManagerDependencies; + shutdowns: string[]; +} + +const TEST_DEVICE: IOSDevice = { + name: 'iPhone 16 Pro', + udid: 'TEST-UDID-1', + state: 'Shutdown', + runtime: 'iOS 18.2', + isAvailable: true, +}; + +function createDeferred(): Deferred { + let resolve!: (value: T) => void; + let reject!: (reason?: unknown) => void; + + const promise = new Promise((res, rej) => { + resolve = res; + reject = rej; + }); + + return { promise, resolve, reject }; +} + +class FakeAutomationSession implements IOSAutomationSession { + deleteCount = 0; + + deleteSession(): Promise { + this.deleteCount += 1; + return Promise.resolve(); + } + + navigate(): Promise { + return Promise.resolve(); + } + + back(): Promise { + return Promise.resolve(); + } + + forward(): Promise { + return Promise.resolve(); + } + + reload(): Promise { + return Promise.resolve(); + } + + snapshot(options: IOSSnapshotOptions): Promise { + void options; + return Promise.resolve({ + epoch: 1, + snapshot: '- document', + refCount: 0, + totalElements: 0, + emittedElements: 0, + truncated: false, + url: 'about:blank', + title: '', + durationMs: 0, + }); + } + + getText(target: IOSTargetInput): Promise { + void target; + return Promise.resolve(''); + } + + getHtml(target: IOSTargetInput, outer?: boolean): Promise { + void target; + void outer; + return Promise.resolve(''); + } + + screenshot(): Promise { + return Promise.resolve({ image: '', format: 'png' }); + } + + tap(target: IOSTargetInput): Promise { + void target; + return Promise.resolve(); + } + + fill(target: IOSTargetInput, value: string): Promise { + void target; + void value; + return Promise.resolve(); + } + + type(target: IOSTargetInput, text: string): Promise { + void target; + void text; + return Promise.resolve(); + } + + select(target: IOSTargetInput, values: string[]): Promise { + void target; + void values; + return Promise.resolve(); + } + + check(target: IOSTargetInput): Promise { + void target; + return Promise.resolve(); + } + + uncheck(target: IOSTargetInput): Promise { + void target; + return Promise.resolve(); + } + + swipe(direction: IOSSwipeDirection, target?: IOSTargetInput, duration?: number): Promise { + void direction; + void target; + void duration; + return Promise.resolve(); + } + + scroll(direction: IOSScrollDirection, amount?: number): Promise { + void direction; + void amount; + return Promise.resolve({ scrollX: 0, scrollY: 0 }); + } + + evaluate(script: string): Promise { + void script; + return Promise.resolve(null); + } + + waitForSelector( + selector: string, + state?: IOSWaitForSelectorState, + timeout?: number + ): Promise { + void selector; + void state; + void timeout; + return Promise.resolve(); + } + + consoleLogs(level?: IOSConsoleLogLevel): Promise { + void level; + return Promise.resolve([]); + } +} + +class FakeAppiumProcess implements IOSAppiumProcess { + readonly port = 4723; + readonly pid = 4242; + + terminateSignals: ('SIGTERM' | 'SIGKILL')[] = []; + private exitCode: number | null = null; + private readonly listeners = new Set<(code: number | null) => void>(); + + onExit(callback: (code: number | null) => void): () => void { + this.listeners.add(callback); + return () => { + this.listeners.delete(callback); + }; + } + + terminate(signal: 'SIGTERM' | 'SIGKILL'): Promise { + this.terminateSignals.push(signal); + + if (signal === 'SIGTERM') { + this.simulateExit(0); + } + + if (signal === 'SIGKILL') { + this.simulateExit(9); + } + + return Promise.resolve(); + } + + waitForExit(): Promise { + return Promise.resolve(this.exitCode); + } + + simulateExit(code: number): void { + this.exitCode = code; + for (const listener of this.listeners) { + listener(code); + } + } +} + +function createRuntimeHarness( + options: { + bootDeferred?: Deferred; + bootRejectOnAbort?: boolean; + } = {} +): RuntimeHarness { + const appium = new FakeAppiumProcess(); + const automationSession = new FakeAutomationSession(); + const shutdowns: string[] = []; + + const dependencies: IOSManagerDependencies = { + listDevices(): Promise { + return Promise.resolve([TEST_DEVICE]); + }, + async bootDevice(_device: IOSDevice, signal: AbortSignal): Promise { + if (options.bootRejectOnAbort) { + await new Promise((_resolve, reject) => { + const abortHandler = (): void => { + reject(new Error('IOSManager launch aborted')); + }; + + if (signal.aborted) { + abortHandler(); + return; + } + + signal.addEventListener('abort', abortHandler, { once: true }); + }); + return; + } + + if (options.bootDeferred !== undefined) { + await new Promise((resolve, reject) => { + const abortHandler = (): void => { + reject(new Error('IOSManager launch aborted')); + }; + + if (signal.aborted) { + abortHandler(); + return; + } + + signal.addEventListener('abort', abortHandler, { once: true }); + void options.bootDeferred?.promise.then( + () => { + signal.removeEventListener('abort', abortHandler); + resolve(); + }, + (error: unknown) => { + signal.removeEventListener('abort', abortHandler); + reject(error instanceof Error ? error : new Error(String(error))); + } + ); + }); + } + }, + shutdownDevice(device: IOSDevice): Promise { + shutdowns.push(device.udid); + return Promise.resolve(); + }, + startAppium(): Promise { + return Promise.resolve(appium); + }, + createAutomationSession(): Promise { + return Promise.resolve(automationSession); + }, + }; + + return { + appium, + automationSession, + dependencies, + shutdowns, + }; +} + +async function expectErrorMessage(promise: Promise, message: string): Promise { + try { + await promise; + throw new Error(`Expected promise to reject with: ${message}`); + } catch (error) { + if (!(error instanceof Error)) { + throw error; + } + expect(error.message).toBe(message); + } +} + +describe('IOSService lease model', () => { + it('shares a single manager across sessions and only shuts down on last release', async () => { + const runtime = createRuntimeHarness(); + let managerCount = 0; + + const service = new IOSService({ + managerFactory: () => { + managerCount += 1; + return new IOSManager({ dependencies: runtime.dependencies }); + }, + }); + + const managerA = await service.acquire('session-a', 'iPhone 16 Pro'); + const managerB = await service.acquire('session-b'); + + expect(managerA).toBe(managerB); + expect(managerCount).toBe(1); + expect(service.state).toBe('ready'); + expect(service.leaseCount).toBe(2); + + await service.release('session-a'); + expect(service.state).toBe('ready'); + expect(runtime.automationSession.deleteCount).toBe(0); + + await service.release('session-b'); + expect(service.state).toBe('idle'); + expect(runtime.automationSession.deleteCount).toBe(1); + expect(runtime.shutdowns).toEqual([TEST_DEVICE.udid]); + }); + + it('waits on a single in-flight launch for concurrent acquire calls', async () => { + const bootDeferred = createDeferred(); + const runtime = createRuntimeHarness({ bootDeferred }); + let managerCount = 0; + + const service = new IOSService({ + managerFactory: () => { + managerCount += 1; + return new IOSManager({ dependencies: runtime.dependencies }); + }, + }); + + const acquireA = service.acquire('session-a'); + const acquireB = service.acquire('session-b'); + + await Promise.resolve(); + expect(service.state).toBe('launching'); + expect(managerCount).toBe(1); + + bootDeferred.resolve(undefined); + + const [managerA, managerB] = await Promise.all([acquireA, acquireB]); + expect(managerA).toBe(managerB); + expect(service.state).toBe('ready'); + expect(service.leaseCount).toBe(2); + }); + + it('force releases the last lease during launch and aborts the pending startup', async () => { + const runtime = createRuntimeHarness({ bootRejectOnAbort: true }); + const service = new IOSService({ + managerFactory: () => new IOSManager({ dependencies: runtime.dependencies }), + }); + + const acquirePromise = service.acquire('session-a'); + await Promise.resolve(); + + expect(service.state).toBe('launching'); + await service.forceRelease('session-a'); + + await expectErrorMessage(acquirePromise, 'IOSManager launch aborted'); + expect(service.state).toBe('idle'); + expect(service.leaseCount).toBe(0); + + await service.forceRelease('session-a'); + }); + + it('invalidates all leases after an Appium crash on the next manager access', async () => { + const runtime = createRuntimeHarness(); + const service = new IOSService({ + managerFactory: () => new IOSManager({ dependencies: runtime.dependencies }), + }); + + await service.acquire('session-a'); + await service.acquire('session-b'); + + runtime.appium.simulateExit(13); + + await expectErrorMessage( + service.getManager('session-a'), + 'Appium process exited unexpectedly (code 13). Call ios_launch to restart.' + ); + expect(service.state).toBe('idle'); + expect(service.leaseCount).toBe(0); + + await expectErrorMessage(service.getManager('session-b'), 'No iOS lease for session'); + }); +}); diff --git a/agent-bridge/src/__tests__/ios-session-wiring.test.ts b/agent-bridge/src/__tests__/ios-session-wiring.test.ts new file mode 100644 index 000000000..780a1c3eb --- /dev/null +++ b/agent-bridge/src/__tests__/ios-session-wiring.test.ts @@ -0,0 +1,112 @@ +import { afterEach, describe, expect, it, jest } from 'bun:test'; + +import { OrbitAgent } from '../agent/core/agent.js'; +import { SessionManager } from '../agent/session/session-manager.js'; +import { createIOSMcpServer } from '../ios/ios-mcp-server.js'; + +import type { IOSService } from '../ios/ios-service.js'; +import type { Options } from '@anthropic-ai/claude-agent-sdk'; + +interface StartSessionContext { + _mcpServers: Record; +} + +function getSystemPromptAppend(agent: OrbitAgent): string { + const options = (agent as unknown as { _createOptions(): Options })._createOptions(); + const systemPrompt = options.systemPrompt; + if ( + systemPrompt === undefined || + typeof systemPrompt === 'string' || + systemPrompt.type !== 'preset' + ) { + throw new Error('Expected preset system prompt'); + } + return systemPrompt.append ?? ''; +} + +describe('iOS session wiring', () => { + afterEach(() => { + jest.restoreAllMocks(); + }); + + function mockSessionLifecycle(): ReturnType { + return jest + .spyOn( + SessionManager.prototype as unknown as { + _startBackgroundConsumer(sessionId: string, agent: OrbitAgent): void; + }, + '_startBackgroundConsumer' + ) + .mockImplementation((): void => undefined); + } + + it('adds the orbit-ios MCP server to new sessions when an iOS service is available', async () => { + const capturedServerKeys: string[][] = []; + const startSessionSpy = jest + .spyOn(OrbitAgent.prototype, 'startSession') + .mockImplementation(function (this: StartSessionContext): Promise { + capturedServerKeys.push(Object.keys(this._mcpServers)); + return Promise.resolve(); + }); + const backgroundConsumerSpy = mockSessionLifecycle(); + + const manager = new SessionManager({ iosService: {} as IOSService }); + await manager.createSession('session-with-ios'); + + expect(startSessionSpy).toHaveBeenCalledTimes(1); + expect(backgroundConsumerSpy).toHaveBeenCalledTimes(1); + expect(capturedServerKeys).toEqual([['orbit-ios', 'orbit-browser']]); + }); + + it('does not add the orbit-ios MCP server when iOS is unavailable', async () => { + const capturedServerKeys: string[][] = []; + const startSessionSpy = jest + .spyOn(OrbitAgent.prototype, 'startSession') + .mockImplementation(function (this: StartSessionContext): Promise { + capturedServerKeys.push(Object.keys(this._mcpServers)); + return Promise.resolve(); + }); + mockSessionLifecycle(); + + const manager = new SessionManager(); + await manager.createSession('session-without-ios'); + + expect(startSessionSpy).toHaveBeenCalledTimes(1); + expect(capturedServerKeys).toEqual([['orbit-browser']]); + }); + + it('only appends iOS workflow guidance when the orbit-ios MCP server is configured', () => { + const iosAgent = new OrbitAgent({ + mcpServers: { + 'orbit-ios': createIOSMcpServer({} as IOSService, 'ios-session'), + }, + }); + const browserOnlyAgent = new OrbitAgent(); + + const iosPrompt = getSystemPromptAppend(iosAgent); + const browserOnlyPrompt = getSystemPromptAppend(browserOnlyAgent); + + expect(iosPrompt).toContain('mcp__orbit-ios__ios_launch'); + expect(iosPrompt).toContain('mcp__orbit-ios__ios_close'); + expect(browserOnlyPrompt).not.toContain('mcp__orbit-ios__ios_launch'); + }); + + it('force releases the iOS lease when a session is deleted', async () => { + const forceRelease = jest.fn(() => Promise.resolve()); + jest.spyOn(OrbitAgent.prototype, 'startSession').mockImplementation(() => Promise.resolve()); + jest.spyOn(OrbitAgent.prototype, 'stopSession').mockImplementation(() => Promise.resolve()); + mockSessionLifecycle(); + + const manager = new SessionManager({ + iosService: { + forceRelease, + } as unknown as IOSService, + }); + + await manager.createSession('session-to-delete'); + await manager.deleteSession('session-to-delete'); + + expect(forceRelease).toHaveBeenCalledTimes(1); + expect(forceRelease).toHaveBeenCalledWith('session-to-delete'); + }); +}); diff --git a/agent-bridge/src/__tests__/setup.ts b/agent-bridge/src/__tests__/setup.ts new file mode 100644 index 000000000..4319fc8c9 --- /dev/null +++ b/agent-bridge/src/__tests__/setup.ts @@ -0,0 +1,8 @@ +/** + * Bun test preload — runs before any test file. + * + * Clears the CLAUDECODE env var so the SDK can spawn CLI subprocesses + * without hitting the "cannot be launched inside another Claude Code session" guard. + * This only matters when running `bun test` from inside a Claude Code session. + */ +delete process.env.CLAUDECODE; diff --git a/agent-bridge/src/__tests__/text-event-batcher-immediate.test.ts b/agent-bridge/src/__tests__/text-event-batcher-immediate.test.ts new file mode 100644 index 000000000..436057868 --- /dev/null +++ b/agent-bridge/src/__tests__/text-event-batcher-immediate.test.ts @@ -0,0 +1,18 @@ +import { describe, expect, it } from 'bun:test'; + +import { TextEventBatcher } from '../common/batching/text-event-batcher.js'; + +describe('TextEventBatcher immediate mode', () => { + it('emits text immediately when batch interval is non-positive', () => { + const emitted: string[] = []; + const batcher = new TextEventBatcher((event) => { + emitted.push(event.content); + }, 0); + + batcher.add('session-1', 'message-1', 'Hello '); + batcher.add('session-1', 'message-1', 'world'); + + expect(emitted).toEqual(['Hello ', 'world']); + expect(batcher.getAccumulatedLength('session-1', 'message-1')).toBe('Hello world'.length); + }); +}); diff --git a/agent-bridge/src/__tests__/title-generation.e2e.test.ts b/agent-bridge/src/__tests__/title-generation.e2e.test.ts new file mode 100644 index 000000000..8286f6061 --- /dev/null +++ b/agent-bridge/src/__tests__/title-generation.e2e.test.ts @@ -0,0 +1,55 @@ +/** + * TESTED: agent-bridge/src/agent/session/session-manager.ts:generateTitle() + * Run: cd agent-bridge && bun test title-generation + */ + +import { describe, expect, it } from 'bun:test'; + +import { SessionManager } from '../agent/session/session-manager.js'; +import { ClaudeCredentials } from '../common/auth/credentials.js'; +import { GenerateTitleRequestSchema } from '../protocol/schemas.js'; + +const credentials = await ClaudeCredentials.getCredentials(); +const hasCredentials = credentials.hasCredentials; + +describe('title generation (direct SDK)', () => { + it.skipIf(!hasCredentials)( + 'generates a non-empty title through the direct SDK path', + async () => { + const manager = new SessionManager(); + const title = await manager.generateTitle('Help me debug a TypeScript build failure'); + + expect(title.length).toBeGreaterThan(0); + expect(title.length).toBeLessThanOrEqual(50); + } + ); + + it.skipIf(!hasCredentials)('generates a title from the user message alone', async () => { + const manager = new SessionManager(); + const title = await manager.generateTitle('fix this error'); + + expect(title.length).toBeGreaterThan(0); + expect(title.length).toBeLessThanOrEqual(50); + }); + + it('accepts the user-only title request schema', () => { + const parsed = GenerateTitleRequestSchema.parse({ + type: 'generate_title', + userMessage: 'Explain React useEffect cleanup', + }); + + expect(parsed.userMessage).toBe('Explain React useEffect cleanup'); + }); + + it.skipIf(hasCredentials)('throws when credentials are unavailable', async () => { + const manager = new SessionManager(); + + try { + await manager.generateTitle('Explain React useEffect cleanup'); + throw new Error('Expected title generation to fail without credentials'); + } catch (error) { + expect(error).toBeInstanceOf(Error); + expect((error as Error).message).toBe('No credentials available for title generation'); + } + }); +}); diff --git a/agent-bridge/src/agent/core/agent.ts b/agent-bridge/src/agent/core/agent.ts index 2481b6e69..f17c776fa 100644 --- a/agent-bridge/src/agent/core/agent.ts +++ b/agent-bridge/src/agent/core/agent.ts @@ -49,6 +49,7 @@ import type { const logger = createLogger('OrbitAgent'); const BROWSER_MCP_SERVER_KEY = 'orbit-browser'; +const IOS_MCP_SERVER_KEY = 'orbit-ios'; /** * Type guard for tool_result blocks @@ -546,84 +547,7 @@ export class OrbitAgent { systemPrompt: { type: 'preset' as const, preset: 'claude_code' as const, - append: ` -## Browser Automation - -You have access to embedded browser automation tools via MCP. Use mcp__orbit-browser__browser_open to start a browser session. - -### Panel Control -- **mcp__orbit-browser__browser_open**: Open the browser panel and optionally navigate to a URL. -- **mcp__orbit-browser__browser_close**: Close the browser panel when done with automation. - -### Navigation -- **mcp__orbit-browser__browser_navigate**: Go to URL -- **mcp__orbit-browser__browser_back / mcp__orbit-browser__browser_forward / mcp__orbit-browser__browser_reload**: History navigation - -### Interaction -- **mcp__orbit-browser__browser_click**: Click by CSS selector -- **mcp__orbit-browser__browser_type**: Type text into an input field - -### Observation -- **mcp__orbit-browser__browser_get_text**: Get page or element text -- **mcp__orbit-browser__browser_get_html**: Get page or element HTML -- **mcp__orbit-browser__browser_screenshot**: Get page info (URL, title, dimensions) -- **mcp__orbit-browser__browser_console_logs**: Get console logs - -### JavaScript -- **mcp__orbit-browser__browser_eval**: Execute JavaScript in page context - -### Recommended Workflow -1. Use mcp__orbit-browser__browser_open to start a browser session (or browser_navigate if already open) -2. Use browser_get_text/browser_get_html to inspect content -3. Use browser_click/browser_type for interactions -4. Use browser_console_logs to check for JavaScript errors -5. Use browser_close when done - -### Tips -- **Use browser_open first** - it opens the panel and navigates in one step -- **Use specific CSS selectors** for reliable interaction -- **Check console for errors** after page loads or after interactions fail - -## Chrome DevTools (Advanced) - -When browser is open, you also have access to Chrome DevTools Protocol tools via mcp__orbit-devtools__*: - -### Console -- **devtools_console_get**: Get console logs with filtering by type (log/warn/error/info/debug) -- **devtools_console_clear**: Clear console messages -- **devtools_console_eval**: Execute JavaScript in console context - -### Network (Detailed) -- **devtools_network_get**: Get network requests with filtering (url pattern, method, status) -- **devtools_network_detail**: Get full request/response details including headers and body -- **devtools_network_clear**: Clear network logs - -### DOM Inspection -- **devtools_dom_query**: Query DOM with CSS selectors, get element structure -- **devtools_dom_html**: Get outer HTML of elements -- **devtools_dom_styles**: Get computed CSS styles for elements -- **devtools_dom_attributes**: Get all attributes of an element - -### Performance -- **devtools_perf_metrics**: Get performance metrics (memory, DOM stats, rendering times) -- **devtools_perf_trace_start**: Start recording performance trace -- **devtools_perf_trace_stop**: Stop trace and get timeline events - -### Storage -- **devtools_storage_local / devtools_storage_session**: Get localStorage/sessionStorage -- **devtools_storage_cookies**: Get cookies (optionally filter by domain) -- **devtools_storage_set_local / devtools_storage_set_session**: Set storage items -- **devtools_storage_set_cookie**: Set a cookie with full options -- **devtools_storage_clear**: Clear storage (local/session/cookies/all) - -### General -- **devtools_eval**: Execute JavaScript with full page access, can await promises -- **devtools_page_info**: Get current page title and URL - -### When to Use DevTools vs Browser Tools -- **Browser tools (mcp__orbit-browser__)**: Page interaction, navigation, clicking, typing -- **DevTools tools (mcp__orbit-devtools__)**: Deep inspection, debugging, storage, performance analysis -`, + append: this._buildSystemPromptAppend(), }, // Working directory cwd: this.cwd, @@ -1109,6 +1033,134 @@ When browser is open, you also have access to Chrome DevTools Protocol tools via return options; } + private _buildSystemPromptAppend(): string { + const sections = [ + ` +## Browser Automation + +You have access to embedded browser automation tools via MCP. Use mcp__orbit-browser__browser_open to start a browser session. + +### Core Workflow +1. Open or navigate the browser. +2. Call mcp__orbit-browser__browser_snapshot to inspect the page and obtain ref handles like \`e1\`, \`e2\`. +3. Use those refs for interactions and inspection tools whenever possible. +4. After navigation, large DOM changes, or any stale-ref error, call mcp__orbit-browser__browser_snapshot again before continuing. +5. Use CSS selectors only as a fallback when no snapshot ref is available. + +### Panel Control +- **mcp__orbit-browser__browser_open**: Open the browser panel and optionally navigate to a URL. +- **mcp__orbit-browser__browser_close**: Close the browser panel when done with automation. + +### Navigation +- **mcp__orbit-browser__browser_navigate**: Go to URL +- **mcp__orbit-browser__browser_back / mcp__orbit-browser__browser_forward / mcp__orbit-browser__browser_reload**: History navigation +- **mcp__orbit-browser__browser_get_url / mcp__orbit-browser__browser_get_title**: Read the current page location/title after redirects or navigation + +### Interaction +- **mcp__orbit-browser__browser_click**: Click by ref (preferred) or CSS selector +- **mcp__orbit-browser__browser_type**: Append text to an input field by ref or selector +- **mcp__orbit-browser__browser_fill**: Replace the value of an input field by ref or selector +- **mcp__orbit-browser__browser_check / mcp__orbit-browser__browser_uncheck / mcp__orbit-browser__browser_select**: Form controls by ref or selector +- **mcp__orbit-browser__browser_hover / mcp__orbit-browser__browser_focus / mcp__orbit-browser__browser_scroll_into_view**: Prepare targets for interaction +- **mcp__orbit-browser__browser_wait_for_selector / mcp__orbit-browser__browser_wait_for_url**: Wait for async content or redirects before continuing + +### Observation +- **mcp__orbit-browser__browser_snapshot**: Primary inspection tool. Returns a semantic tree with stable refs for later actions. +- **mcp__orbit-browser__browser_get_text / mcp__orbit-browser__browser_get_html**: Read page or element content by ref or selector +- **mcp__orbit-browser__browser_screenshot**: Save viewport screenshot to temp file - use Read to view the image +- **mcp__orbit-browser__browser_console_logs**: Get console logs +- **mcp__orbit-browser__browser_network_requests / mcp__orbit-browser__browser_runtime_info**: Debug network/runtime issues + +### JavaScript +- **mcp__orbit-browser__browser_eval**: Execute JavaScript in page context + +### Recommended Workflow +1. Use mcp__orbit-browser__browser_open to start a browser session (or browser_navigate if already open) +2. Use mcp__orbit-browser__browser_snapshot immediately to inspect the page and get refs +3. Use ref-based browser_click/browser_fill/browser_type/browser_get_text/browser_get_html calls +4. Re-run browser_snapshot after navigation or when a ref becomes stale +5. Use browser_console_logs, browser_network_requests, or browser_runtime_info when automation behaves unexpectedly +6. Use browser_close when done + +### Tips +- **Use browser_open first** - it opens the panel and navigates in one step +- **Prefer refs over selectors** - refs from browser_snapshot are more reliable than CSS selectors +- **Use browser_fill to replace, browser_type to append** +- **If a tool says a ref is stale, call browser_snapshot again** +- **Selectors are fallback-only** - use them when no ref is available or when CSP/runtime limits prevent ref-based actions +- **Check console for errors** after page loads or after interactions fail + +## Chrome DevTools (Advanced) + +When browser is open, you also have access to Chrome DevTools Protocol tools via mcp__orbit-devtools__*: + +### Console +- **devtools_console_get**: Get console logs with filtering by type (log/warn/error/info/debug) +- **devtools_console_clear**: Clear console messages +- **devtools_console_eval**: Execute JavaScript in console context + +### Network (Detailed) +- **devtools_network_get**: Get network requests with filtering (url pattern, method, status) +- **devtools_network_detail**: Get full request/response details including headers and body +- **devtools_network_clear**: Clear network logs + +### DOM Inspection +- **devtools_dom_query**: Query DOM with CSS selectors, get element structure +- **devtools_dom_html**: Get outer HTML of elements +- **devtools_dom_styles**: Get computed CSS styles for elements +- **devtools_dom_attributes**: Get all attributes of an element + +### Performance +- **devtools_perf_metrics**: Get performance metrics (memory, DOM stats, rendering times) +- **devtools_perf_trace_start**: Start recording performance trace +- **devtools_perf_trace_stop**: Stop trace and get timeline events + +### Storage +- **devtools_storage_local / devtools_storage_session**: Get localStorage/sessionStorage +- **devtools_storage_cookies**: Get cookies (optionally filter by domain) +- **devtools_storage_set_local / devtools_storage_set_session**: Set storage items +- **devtools_storage_set_cookie**: Set a cookie with full options +- **devtools_storage_clear**: Clear storage (local/session/cookies/all) + +### General +- **devtools_eval**: Execute JavaScript with full page access, can await promises +- **devtools_page_info**: Get current page title and URL + +### When to Use DevTools vs Browser Tools +- **Browser tools (mcp__orbit-browser__)**: Page interaction, navigation, clicking, typing +- **DevTools tools (mcp__orbit-devtools__)**: Deep inspection, debugging, storage, performance analysis +`, + ]; + + if (Object.hasOwn(this._mcpServers, IOS_MCP_SERVER_KEY)) { + sections.push(` +## iOS Simulator Automation + +You may also have access to iOS simulator automation tools via MCP when the \`mcp__orbit-ios__*\` tool namespace is present. + +### Core Workflow +1. Call \`mcp__orbit-ios__ios_launch\` first to boot the simulator, start Safari, and acquire the session lease. +2. Use \`mcp__orbit-ios__ios_navigate\` to open the target page. +3. Call \`mcp__orbit-ios__ios_snapshot\` to inspect the mobile page and obtain refs like \`e1\`, \`e2\`. +4. Use those refs for \`ios_tap\`, \`ios_fill\`, \`ios_type\`, \`ios_get_text\`, and \`ios_get_html\` whenever possible. +5. Re-run \`ios_snapshot\` after navigation, major DOM changes, or stale-ref errors. + +### Mobile-Specific Actions +- **\`mcp__orbit-ios__ios_swipe\`**: Native touch gesture for mobile carousels and scroll regions +- **\`mcp__orbit-ios__ios_scroll\`**: Scroll the viewport and inspect updated positions +- **\`mcp__orbit-ios__ios_screenshot\`**: Capture a PNG screenshot of the current mobile viewport +- **\`mcp__orbit-ios__ios_console_logs\` / \`mcp__orbit-ios__ios_eval\`**: Debug mobile Safari behavior + +### Recovery Rules +- **If a tool says \`No iOS lease for session\`**: call \`mcp__orbit-ios__ios_launch\` first +- **If Appium exited unexpectedly**: call \`mcp__orbit-ios__ios_launch\` to restart; the system does not auto-recover +- **Use \`mcp__orbit-ios__ios_close\` when done** so the simulator lease is released cleanly +`); + } + + return sections.join('\n'); + } + async startSession(): Promise { /** * Start a persistent streaming session with Claude. diff --git a/agent-bridge/src/agent/definitions/skill-definitions.ts b/agent-bridge/src/agent/definitions/skill-definitions.ts index 1ab3a05d9..4c067043a 100644 --- a/agent-bridge/src/agent/definitions/skill-definitions.ts +++ b/agent-bridge/src/agent/definitions/skill-definitions.ts @@ -86,10 +86,26 @@ function parseSkillFile( }; } +/** + * Resolve whether a dirent represents a directory, following symlinks. + * Dirent.isDirectory() returns false for symlinks even when the target + * is a directory. Marketplace skills (bunx skills add) are installed as + * symlinks, so we must resolve them via fs.statSync. + */ +function isDirectoryEntry(entry: fs.Dirent, entryPath: string): boolean { + if (entry.isDirectory()) return true; + if (!entry.isSymbolicLink()) return false; + try { + return fs.statSync(entryPath).isDirectory(); + } catch { + return false; // broken symlink + } +} + /** * Scan a single skills directory using dual-pattern matching: - * 1. Subdirectories containing SKILL.md → parse as skill - * 2. Root-level *.md files → parse as skill (catches standalone skill files) + * 1. Subdirectories (or symlinks to directories) containing SKILL.md → parse as skill + * 2. Root-level *.md files (or symlinks to .md files) → parse as skill */ function scanSkillsDirectory(skillsDir: string, source: 'project' | 'user'): SkillDefinition[] { if (!fs.existsSync(skillsDir)) { @@ -102,9 +118,11 @@ function scanSkillsDirectory(skillsDir: string, source: 'project' | 'user'): Ski const entries = fs.readdirSync(skillsDir, { withFileTypes: true }); for (const entry of entries) { - if (entry.isDirectory()) { - // Pattern 1: Subdirectory with SKILL.md - const skillFilePath = path.join(skillsDir, entry.name, 'SKILL.md'); + const entryPath = path.join(skillsDir, entry.name); + + if (isDirectoryEntry(entry, entryPath)) { + // Pattern 1: Subdirectory (or symlink to directory) with SKILL.md + const skillFilePath = path.join(entryPath, 'SKILL.md'); if (fs.existsSync(skillFilePath)) { try { const content = fs.readFileSync(skillFilePath, 'utf-8'); @@ -118,15 +136,14 @@ function scanSkillsDirectory(skillsDir: string, source: 'project' | 'user'): Ski logger.warn({ file: skillFilePath, error }, 'Failed to parse skill file'); } } - } else if (entry.isFile() && entry.name.endsWith('.md')) { - // Pattern 2: Root-level .md files - const filePath = path.join(skillsDir, entry.name); + } else if ((entry.isFile() || entry.isSymbolicLink()) && entry.name.endsWith('.md')) { + // Pattern 2: Root-level .md files (or symlinks to .md files) try { - const content = fs.readFileSync(filePath, 'utf-8'); - const skill = parseSkillFile(content, entry.name, source, filePath); + const content = fs.readFileSync(entryPath, 'utf-8'); + const skill = parseSkillFile(content, entry.name, source, entryPath); skills.push(skill); } catch (error) { - logger.warn({ file: filePath, error }, 'Failed to parse skill file'); + logger.warn({ file: entryPath, error }, 'Failed to parse skill file'); } } } diff --git a/agent-bridge/src/agent/session/session-manager.ts b/agent-bridge/src/agent/session/session-manager.ts index 9843dbe1d..fcfec8fcf 100644 --- a/agent-bridge/src/agent/session/session-manager.ts +++ b/agent-bridge/src/agent/session/session-manager.ts @@ -9,18 +9,22 @@ import * as os from 'node:os'; import * as path from 'node:path'; import { query } from '@anthropic-ai/claude-agent-sdk'; +import Anthropic from '@anthropic-ai/sdk'; import { formatZodError } from '@orbit/shared-schemas'; import { z } from 'zod'; -import { ClaudeCredentials } from '../../common/auth/credentials.js'; // [oauth-401-recovery] import — revert: remove this line +import { ClaudeCredentials } from '../../common/auth/credentials.js'; import { TextEventBatcher } from '../../common/batching/index.js'; +import { getShellEnvironment } from '../../common/env/shell-env.js'; import { Disposable, Emitter } from '../../common/events/events.js'; import { createLogger } from '../../common/logging/logger.js'; +import { createIOSMcpServer } from '../../ios/index.js'; import { OrbitAgent } from '../core/agent.js'; import { getStorageDirPath, getStoredSessionsSnapshot } from './session-storage.js'; import type { McpToolRequest, McpToolResponse } from '../../browser/index.js'; +import type { IOSService } from '../../ios/ios-service.js'; import type { OrbitAgentConfig } from '../core/agent.js'; import type { AttachmentContentBlock } from '../types/messages.js'; @@ -86,6 +90,35 @@ const GeneratedCommandSchema = z .strict(); const logger = createLogger('SessionManager'); +const TITLE_MODEL = 'claude-haiku-4-5-20251001'; +const TITLE_QUERY_MODEL = 'haiku'; + +function buildTitlePrompt(userMessage: string): string { + const maxTitleChars = 50; + + return `Generate a short title (max ${String(maxTitleChars)} characters) for this conversation. +The title should capture the main topic or intent. +Do NOT use quotes, periods, or prefixes like "Title:". +Do NOT exceed ${String(maxTitleChars)} characters. Just output the title text and nothing else. + +User: "${userMessage.slice(0, 300)}"`; +} + +function cleanupGeneratedTitle(text: string, maxChars = 50): string { + let title = text + .replace(/^["']|["']$/g, '') + .replace(/^Title:\s*/i, '') + .replace(/\.$/, '') + .trim(); + + if (title.length > maxChars) { + const truncated = title.slice(0, maxChars); + const lastSpace = truncated.lastIndexOf(' '); + title = lastSpace > maxChars * 0.4 ? truncated.slice(0, lastSpace) : truncated; + } + + return title; +} /** * Agent message types sent to the frontend @@ -197,6 +230,10 @@ export interface SessionConfig { forkSession?: boolean; } +interface SessionManagerOptions { + iosService?: IOSService; +} + /** * Permission resolver - resolves when user responds to permission request */ @@ -786,8 +823,10 @@ async function hardLinkFileBackups( * Session Manager - orchestrates Claude Agent SDK sessions */ export class SessionManager extends Disposable { - // Text event batcher - reduces ~200 events per response to ~4-8 batched events - // Batches at 50ms intervals to align with screen refresh and reduce IPC overhead + // Text event batcher - configured for immediate emission in production so + // short assistant replies do not collapse into whole-clause updates. + // The batcher still tracks accumulated lengths for contentOffset math and + // retains its drain/flush utilities for non-production intervals. private readonly textBatcher: TextEventBatcher; // Content loss tracking - monitors data loss due to SDK integration issues @@ -898,6 +937,7 @@ export class SessionManager extends Disposable { private sessionInitFired = new Set(); private pendingDisplayNames = new Map(); private browserToolUnsubscribers = new Map void>(); + private readonly iosService?: IOSService; // Track the current turn ID per session (our OWN stable ID, not SDK's uuid) // The SDK sends different UUIDs for each message (stream_event, assistant, etc.) @@ -906,12 +946,13 @@ export class SessionManager extends Disposable { // and results all share the same messageId for proper UI interleaving. private currentTurnId = new Map(); - constructor() { + constructor(options: SessionManagerOptions = {}) { super(); + this.iosService = options.iosService; - // Initialize text batcher - fires batched text events every 16ms (~1 frame) - // Keep backend batching minimal; frontend RAF batching handles render smoothness. - // This gives fast feedback while frontend coalesces into 60fps renders. + // Initialize text batcher - fires batched text events every 16ms (~1 frame). + // Keep backend batching minimal; frontend CSS animation-delay handles render smoothness. + // This gives fast feedback while reducing IPC overhead vs per-token emission. this.textBatcher = new TextEventBatcher((event) => { this._onAgentMessage.fire({ sessionId: event.sessionId, @@ -969,6 +1010,10 @@ export class SessionManager extends Disposable { /** * Create a new agent session + * + * [warning] TESTED: This function is covered by integration-style session wiring tests. + * If you modify this, run: cd agent-bridge && bun test + * Test file: src/__tests__/ios-session-wiring.test.ts */ async createSession(sessionId: string, config?: SessionConfig): Promise { if (this.activeSessions.has(sessionId)) { @@ -1091,6 +1136,13 @@ export class SessionManager extends Disposable { }, }; + if (this.iosService) { + finalConfig.mcpServers = { + ...(finalConfig.mcpServers ?? {}), + 'orbit-ios': createIOSMcpServer(this.iosService, sessionId), + }; + } + logger.info( { sessionId, @@ -1329,8 +1381,7 @@ export class SessionManager extends Disposable { // Get our stable turn ID for this session const streamMessageId = this.currentTurnId.get(sessionId); - // Handle text deltas - batch to reduce event flooding - // SDK emits ~200 events per response, we batch at 50ms intervals + // Handle text deltas - forward immediately so streaming cadence stays granular. if (event.type === 'content_block_delta' && event.delta?.type === 'text_delta') { const textDelta = event.delta.text; if (textDelta !== undefined) { @@ -1468,11 +1519,8 @@ export class SessionManager extends Disposable { const initialStatus = wasAlreadyApproved ? 'running' : 'awaiting-permission'; // CRITICAL: Flush buffered text BEFORE calculating contentOffset. - // The text batcher accumulates text at 50ms intervals. If we calculate - // contentOffset without flushing, it includes buffered text that hasn't - // been emitted yet. The frontend would receive tool:start BEFORE the - // text chunks, causing contentOffset > displayedContent.length → text - // split mid-word. Flushing ensures frontend has all text before tool arrives. + // In production the live path emits immediately, but keeping this flush + // makes tool ordering correct for any future non-zero batch interval. this.textBatcher.flushSession(sessionId); // Get the current accumulated text length - this is where the tool @@ -1777,17 +1825,35 @@ export class SessionManager extends Disposable { } const agent = this.activeSessions.get(sessionId); + let stopError: unknown; + if (agent) { - await agent.stopSession(); - this.activeSessions.delete(sessionId); + try { + await agent.stopSession(); + } catch (error) { + stopError = error; + } } + this.activeSessions.delete(sessionId); + + await this.iosService?.forceRelease(sessionId).catch((error: unknown) => { + logger.warn({ sessionId, error }, 'iOS lease release failed during session deletion'); + }); + this.pendingTools.delete(sessionId); this.approvedToolNames.delete(sessionId); this.sessionResumeState.delete(sessionId); this.sessionInitFired.delete(sessionId); this.pendingDisplayNames.delete(sessionId); this.currentTurnId.delete(sessionId); + + if (stopError !== undefined) { + if (stopError instanceof Error) { + throw stopError; + } + throw new Error('Session stop failed during deleteSession()', { cause: stopError }); + } } /** @@ -2525,63 +2591,77 @@ Example format: /** * Generate a concise AI title for a conversation. * - * Uses query() directly with minimal options — no tools, no system prompt, - * no MCP, maxTurns=1. This avoids the heavy OrbitAgent initialization - * (system prompt, MCP servers, file checkpointing, permissions) that caused - * ~10s latency. Direct query() completes in ~1-2s. + * [warning] TESTED: This function is covered by integration tests. + * If you modify this, run: cd agent-bridge && bun test title-generation + * Test file: src/__tests__/title-generation.e2e.test.ts */ - async generateTitle(userMessage: string, assistantResponse: string): Promise { - const maxTitleChars = 50; + async generateTitle(userMessage: string): Promise { + const credentials = await ClaudeCredentials.getCredentials(); + if (!credentials.hasCredentials || credentials.token === undefined) { + throw new Error('No credentials available for title generation'); + } - const prompt = `Generate a short title (max ${String(maxTitleChars)} characters) for this conversation. -The title should capture the main topic or intent. -Do NOT use quotes, periods, or prefixes like "Title:". -Do NOT exceed ${String(maxTitleChars)} characters. Just output the title text and nothing else. + let resultText = ''; -User: "${userMessage.slice(0, 300)}" + if (credentials.type === 'oauth') { + const shellEnv = getShellEnvironment(); + if (shellEnv.PATH) { + process.env.PATH = shellEnv.PATH; + } -Assistant (truncated): "${assistantResponse.slice(0, 300)}"`; + process.env.CLAUDE_CODE_OAUTH_TOKEN = credentials.token; + delete process.env.ANTHROPIC_API_KEY; + delete process.env.ANTHROPIC_AUTH_TOKEN; - const envClaudePath = process.env.CLAUDE_CLI_PATH; - const claudePath = - envClaudePath !== undefined && envClaudePath !== '' ? envClaudePath : undefined; + const envClaudePath = process.env.CLAUDE_CLI_PATH; + const claudePath = + envClaudePath !== undefined && envClaudePath !== '' ? envClaudePath : undefined; - const q = query({ - prompt, - options: { - pathToClaudeCodeExecutable: claudePath, - model: 'haiku', - maxTurns: 1, - }, - }); + const q = query({ + prompt: buildTitlePrompt(userMessage), + options: { + pathToClaudeCodeExecutable: claudePath, + model: TITLE_QUERY_MODEL, + maxTurns: 1, + }, + }); - let resultText = ''; + for await (const message of q) { + const msg = message as Record; + if (msg.type === 'result') { + if (typeof msg.result === 'string') { + resultText = msg.result; + } + break; + } + } + } else { + const client = new Anthropic({ apiKey: credentials.token, timeout: 15_000 }); - for await (const message of q) { - const msg = message as Record; - if (msg.type === 'result') { - if (typeof msg.result === 'string') { - resultText = msg.result; + const response = await client.messages.create({ + model: TITLE_MODEL, + max_tokens: 80, + messages: [ + { + role: 'user', + content: buildTitlePrompt(userMessage), + }, + ], + }); + + for (const block of response.content) { + if (block.type === 'text') { + resultText += block.text; } - break; } } - // Clean up: remove quotes, "Title:" prefix, trailing period - let title = resultText - .replace(/^["']|["']$/g, '') - .replace(/^Title:\s*/i, '') - .replace(/\.$/, '') - .trim(); - - // Hard cap: truncate at word boundary if Haiku exceeded the limit - if (title.length > maxTitleChars) { - const truncated = title.slice(0, maxTitleChars); - const lastSpace = truncated.lastIndexOf(' '); - title = lastSpace > maxTitleChars * 0.4 ? truncated.slice(0, lastSpace) : truncated; + const title = cleanupGeneratedTitle(resultText); + if (title.length === 0) { + throw new Error('Title generation returned no text'); } - return title || 'Untitled'; + return title; } /** @@ -2605,47 +2685,89 @@ Assistant (truncated): "${assistantResponse.slice(0, 300)}"`; ); } + async shutdownAsync(): Promise { + if (this.isDisposed) { + return; + } + + this.logContentLossIfNeeded(); + this.textBatcher.destroy(); + this.denyPendingPermissionRequests(); + this.cancelSessionConsumers(); + this.disposeBrowserToolSubscriptions(); + + for (const [sessionId, agent] of this.activeSessions.entries()) { + try { + await agent.stopSession(); + } catch (error) { + logger.error({ sessionId, error }, 'Error stopping session'); + } + + await this.iosService?.release(sessionId).catch((error: unknown) => { + logger.warn({ sessionId, error }, 'iOS lease release failed during async shutdown'); + }); + } + + this.activeSessions.clear(); + super.dispose(); + } + /** * Dispose the session manager */ override dispose(): void { - // Log content loss metrics on shutdown if any loss occurred + if (this.isDisposed) { + return; + } + + this.logContentLossIfNeeded(); + this.textBatcher.destroy(); + this.denyPendingPermissionRequests(); + this.cancelSessionConsumers(); + this.disposeBrowserToolSubscriptions(); + + // Stop all active sessions + for (const [sessionId, agent] of this.activeSessions.entries()) { + void agent.stopSession().catch((err: unknown) => { + logger.error({ sessionId, error: err }, 'Error stopping session'); + }); + void this.iosService?.forceRelease(sessionId).catch((error: unknown) => { + logger.warn({ sessionId, error }, 'iOS lease release failed during dispose'); + }); + } + this.activeSessions.clear(); + + super.dispose(); + } + + private logContentLossIfNeeded(): void { if (this.hasContentLoss()) { logger.warn( { metrics: this.contentLossMetrics }, 'Content loss detected during session manager lifetime - this indicates SDK integration issues' ); } + } - // Flush and destroy text batcher - this.textBatcher.destroy(); - - // Deny all pending permission requests + private denyPendingPermissionRequests(): void { for (const [, resolver] of this.permissionResolvers.entries()) { resolver({ decision: 'deny', always: false }); } this.permissionResolvers.clear(); + } - // Cancel all background consumers + private cancelSessionConsumers(): void { for (const [, consumer] of this.sessionConsumers.entries()) { consumer.cancel(); } this.sessionConsumers.clear(); + } + private disposeBrowserToolSubscriptions(): void { for (const [, unsubscribe] of this.browserToolUnsubscribers.entries()) { unsubscribe(); } this.browserToolUnsubscribers.clear(); - - // Stop all active sessions - for (const [sessionId, agent] of this.activeSessions.entries()) { - void agent.stopSession().catch((err: unknown) => { - logger.error({ sessionId, error: err }, 'Error stopping session'); - }); - } - this.activeSessions.clear(); - - super.dispose(); } } diff --git a/agent-bridge/src/browser/browser-mcp-server.ts b/agent-bridge/src/browser/browser-mcp-server.ts index 6f5ad22d2..bd21a70c9 100644 --- a/agent-bridge/src/browser/browser-mcp-server.ts +++ b/agent-bridge/src/browser/browser-mcp-server.ts @@ -10,9 +10,66 @@ import { z } from 'zod'; import { createLogger } from '../common/logging/logger.js'; +import { + BrowserConsoleLogLevels, + BrowserScrollDirections, + BrowserStorageStores, + BrowserTargetShape, + BrowserWaitForSelectorStates, + validateTargetInput, +} from './types.js'; + import type { BrowserToolBridge } from './browser-tool-bridge.js'; const logger = createLogger('BrowserMcpServer'); +const MCP_SERVER_NAME = 'orbit-browser'; +const MCP_SERVER_VERSION = '1.0.0'; +const SNAPSHOT_WORKFLOW_GUIDANCE = + 'WORKFLOW: Call browser_snapshot first -> inspect the returned tree -> use ref handles for click/fill/type/check/select. If a ref is stale or the page changed, call browser_snapshot again. CSS selectors remain supported as a fallback.'; +const BROWSER_TOOL_NAMES = [ + 'browser_open', + 'browser_close', + 'browser_navigate', + 'browser_back', + 'browser_forward', + 'browser_reload', + 'browser_get_url', + 'browser_get_title', + 'browser_snapshot', + 'browser_get_text', + 'browser_get_html', + 'browser_screenshot', + 'browser_is_visible', + 'browser_is_enabled', + 'browser_get_attribute', + 'browser_bounding_box', + 'browser_count', + 'browser_click', + 'browser_type', + 'browser_fill', + 'browser_select', + 'browser_check', + 'browser_uncheck', + 'browser_hover', + 'browser_focus', + 'browser_scroll', + 'browser_scroll_into_view', + 'browser_wait_for_selector', + 'browser_wait_for_url', + 'browser_eval', + 'browser_cookies_get', + 'browser_cookies_clear', + 'browser_storage_get', + 'browser_storage_set', + 'browser_storage_clear', + 'browser_network_requests', + 'browser_console_logs', + 'browser_runtime_info', +] as const; + +type BrowserToolName = (typeof BROWSER_TOOL_NAMES)[number]; +type ToolSchemaShape = Record; +type ToolArgs = Record; /** * Safely stringify a result for MCP text content. @@ -28,345 +85,674 @@ function safeStringify(value: unknown): string { try { return JSON.stringify(value); } catch { - // Handle circular references, BigInt, or other non-serializable values - // For primitives, convert directly. For objects, provide a meaningful fallback. if (typeof value === 'bigint') { return value.toString(); } if (typeof value === 'object' && value !== null) { return '[Unserializable Object]'; } - // eslint-disable-next-line @typescript-eslint/no-base-to-string -- primitives are safe to stringify - return String(value); + if (typeof value === 'string') { + return value; + } + if (typeof value === 'number' || typeof value === 'boolean') { + return JSON.stringify(value); + } + if (typeof value === 'symbol') { + return value.description ?? 'Symbol()'; + } + if (typeof value === 'function') { + return `[Function ${value.name || 'anonymous'}]`; + } + return 'null'; + } +} + +function addExecutionTarget(text: string, target: 'desktop' | 'ios'): string { + try { + const parsed: unknown = JSON.parse(text); + if (typeof parsed === 'object' && parsed !== null && !Array.isArray(parsed)) { + return JSON.stringify({ ...(parsed as Record), executionTarget: target }); + } + } catch { + // Not JSON — append as text line + } + return `${text}\nexecutionTarget: ${target}`; +} + +function createSuccessResponse(text: string): { + content: [{ type: 'text'; text: string }]; +} { + return { + content: [{ type: 'text' as const, text }], + }; +} + +function createErrorResponse( + prefix: string, + error: unknown +): { + content: [{ type: 'text'; text: string }]; + isError: true; +} { + return { + content: [ + { + type: 'text' as const, + text: `${prefix}: ${error instanceof Error ? error.message : String(error)}`, + }, + ], + isError: true, + }; +} + +async function runBridgeRequest( + bridge: BrowserToolBridge, + toolName: BrowserToolName, + args: ToolArgs, + options: { + errorPrefix: string; + formatResult?: (result: unknown) => string; + validateArgs?: (args: ToolArgs) => void; + } +): Promise<{ + content: [{ type: 'text'; text: string }]; + isError?: true; +}> { + try { + options.validateArgs?.(args); + const result = await bridge.sendRequest(toolName, args); + const formatted = options.formatResult ? options.formatResult(result) : safeStringify(result); + return createSuccessResponse(addExecutionTarget(formatted, 'desktop')); + } catch (error: unknown) { + return createErrorResponse(options.errorPrefix, error); + } +} + +function validateRequiredTarget(args: ToolArgs): void { + validateTargetInput(args); +} + +function stringResult(result: unknown): string { + if (typeof result === 'string') { + return result; + } + if (result === null || result === undefined) { + return ''; + } + return safeStringify(result); +} + +function createBridgeTool( + bridge: BrowserToolBridge, + toolName: BrowserToolName, + description: string, + shape: ToolSchemaShape, + options: { + errorPrefix: string; + formatResult?: (result: unknown) => string; + validateArgs?: (args: ToolArgs) => void; } +): ReturnType { + return tool(toolName, description, shape, async (args) => { + logger.debug({ toolName }, 'Running browser MCP tool'); + return runBridgeRequest(bridge, toolName, args, options); + }); } /** * Create the Browser MCP server with all browser tools. + * + * [warning] TESTED: This server is covered by integration tests. + * If you modify this, run: cd agent-bridge && bun test + * Test files: src/__tests__/browser-mcp-screenshot.test.ts, src/__tests__/browser-ref-flow.test.ts */ export function createBrowserMcpServer( bridge: BrowserToolBridge ): ReturnType { const tools = [ - tool( + createBridgeTool( + bridge, 'browser_open', - 'Open the embedded browser panel. Optionally navigate to a URL.', + `Open the embedded browser panel and optionally navigate to a URL. ${SNAPSHOT_WORKFLOW_GUIDANCE}`, { url: z.string().optional().describe('URL to open'), }, - async (args) => { - logger.info({ url: args.url }, 'Opening browser'); - try { - const result = await bridge.sendRequest('browser_open', args); - return { - content: [{ type: 'text' as const, text: safeStringify(result) }], - }; - } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to open browser: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; - } + { + errorPrefix: 'Failed to open browser', } ), - tool( + createBridgeTool( + bridge, 'browser_navigate', - 'Navigate the browser to a URL.', + 'Navigate the browser to a URL. After navigation, call browser_snapshot again before using old refs.', { url: z.string().describe('The URL to navigate to'), }, - async (args) => { - logger.info({ url: args.url }, 'Navigating browser'); - try { - await bridge.sendRequest('browser_navigate', args); - return { - content: [{ type: 'text' as const, text: `Navigated to ${args.url}` }], - }; - } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to navigate: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; - } + { + errorPrefix: 'Failed to navigate', + formatResult: () => 'Navigated successfully', } ), - tool( + createBridgeTool( + bridge, 'browser_click', - 'Click an element on the page by CSS selector.', + 'Click an element by snapshot ref (preferred) or CSS selector (legacy fallback). Call browser_snapshot first to obtain refs.', { - selector: z.string().describe('CSS selector of element to click'), + ...BrowserTargetShape, }, - async (args) => { - logger.debug({ selector: args.selector }, 'Clicking element'); - try { - const result = await bridge.sendRequest('browser_click', args); - return { - content: [{ type: 'text' as const, text: safeStringify(result) }], - }; - } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to click: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; - } + { + errorPrefix: 'Failed to click', + validateArgs: validateRequiredTarget, } ), - tool( + createBridgeTool( + bridge, 'browser_type', - 'Type text into an input field.', + 'Type text into a focusable element by snapshot ref (preferred) or CSS selector. This appends to the existing value character-by-character. Use browser_fill to replace the value.', { - selector: z.string().describe('CSS selector of the input'), + ...BrowserTargetShape, text: z.string().describe('Text to type'), }, - async (args) => { - logger.debug({ selector: args.selector }, 'Typing into element'); - try { - const result = await bridge.sendRequest('browser_type', args); - return { - content: [{ type: 'text' as const, text: safeStringify(result) }], - }; - } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to type: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; - } + { + errorPrefix: 'Failed to type', + validateArgs: validateRequiredTarget, } ), - tool( + createBridgeTool( + bridge, + 'browser_fill', + 'Fill a text input, textarea, or contenteditable element by snapshot ref (preferred) or CSS selector. This replaces the existing value.', + { + ...BrowserTargetShape, + value: z.string().describe('Replacement value'), + }, + { + errorPrefix: 'Failed to fill', + validateArgs: validateRequiredTarget, + } + ), + + createBridgeTool( + bridge, 'browser_get_text', - 'Get the text content of the page or a specific element.', + 'Get the trimmed text content of the page or a specific element. Supports ref (preferred) or selector.', { - selector: z.string().optional().describe('CSS selector (default: body)'), + ...BrowserTargetShape, }, - async (args) => { - logger.debug({ selector: args.selector }, 'Getting page text'); - try { - const result = await bridge.sendRequest('browser_get_text', args); - return { - content: [{ type: 'text' as const, text: result ?? '' }], - }; - } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to get text: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; - } + { + errorPrefix: 'Failed to get text', + formatResult: stringResult, } ), - tool( + createBridgeTool( + bridge, 'browser_get_html', - 'Get the HTML content of the page or a specific element.', + 'Get the HTML content of the page or a specific element. Supports ref (preferred) or selector.', { - selector: z.string().optional().describe('CSS selector (default: body)'), + ...BrowserTargetShape, + outer: z.boolean().optional().describe('Return outerHTML instead of innerHTML'), }, - async (args) => { - logger.debug({ selector: args.selector }, 'Getting page HTML'); - try { - const result = await bridge.sendRequest('browser_get_html', args); - return { - content: [{ type: 'text' as const, text: result ?? '' }], - }; - } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to get HTML: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; - } + { + errorPrefix: 'Failed to get HTML', + formatResult: stringResult, } ), tool( 'browser_screenshot', - 'Get information about the current page (URL, title, dimensions).', + 'Take a screenshot of the current browser page. Saves a JPEG to a temp file and returns the path. Use the Read tool on the returned file path to view the image.', {}, async () => { - logger.debug('Getting browser screenshot info'); + logger.debug({ toolName: 'browser_screenshot' }, 'Running browser MCP tool'); try { - const result = await bridge.sendRequest('browser_screenshot', {}); - return { - content: [{ type: 'text' as const, text: safeStringify(result) }], - }; + const result = await bridge.sendRequest<{ + filePath?: string | null; + metadata: Record; + }>('browser_screenshot', {}); + + const parts: string[] = []; + if (typeof result.filePath === 'string' && result.filePath.length > 0) { + parts.push(`Screenshot saved to: ${result.filePath}`); + parts.push( + 'Use the Read tool to view this image. Read the file BEFORE closing or recreating the browser - the file is cleaned up on close.' + ); + } else { + parts.push('Screenshot capture returned metadata only (no image file).'); + } + + parts.push(`Metadata: ${safeStringify(result.metadata)}`); + return createSuccessResponse(addExecutionTarget(parts.join('\n'), 'desktop')); } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to get screenshot info: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; + return createErrorResponse('Failed', error); } } ), - tool('browser_console_logs', 'Get console logs from the browser.', {}, async () => { - logger.debug('Getting browser console logs'); - try { - const result = await bridge.sendRequest('browser_console_logs', {}); - return { - content: [{ type: 'text' as const, text: safeStringify(result) }], - }; - } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to get console logs: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; - } - }), - - tool('browser_back', 'Go back in browser history.', {}, async () => { - logger.debug('Navigating back'); - try { - await bridge.sendRequest('browser_back', {}); - return { - content: [{ type: 'text' as const, text: 'Navigated back' }], - }; - } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to navigate back: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; - } - }), - - tool('browser_forward', 'Go forward in browser history.', {}, async () => { - logger.debug('Navigating forward'); - try { - await bridge.sendRequest('browser_forward', {}); - return { - content: [{ type: 'text' as const, text: 'Navigated forward' }], - }; - } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to navigate forward: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; - } - }), - - tool('browser_reload', 'Reload the current page.', {}, async () => { - logger.debug('Reloading page'); - try { - await bridge.sendRequest('browser_reload', {}); - return { - content: [{ type: 'text' as const, text: 'Page reloaded' }], - }; - } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to reload: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; - } - }), - - tool('browser_close', 'Close the browser panel.', {}, async () => { - logger.debug('Closing browser'); - try { - await bridge.sendRequest('browser_close', {}); - return { - content: [{ type: 'text' as const, text: 'Browser closed' }], - }; - } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to close browser: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; - } - }), + createBridgeTool( + bridge, + 'browser_console_logs', + 'Get console logs captured from the current browser page. Useful after interactions fail or a page behaves unexpectedly.', + { + level: z.enum(BrowserConsoleLogLevels).optional().describe('Optional log level filter'), + }, + { + errorPrefix: 'Failed to get console logs', + } + ), - tool( + createBridgeTool( + bridge, + 'browser_back', + 'Go back in browser history.', + {}, + { + errorPrefix: 'Failed to navigate back', + formatResult: () => 'Navigated back', + } + ), + + createBridgeTool( + bridge, + 'browser_forward', + 'Go forward in browser history.', + {}, + { + errorPrefix: 'Failed to navigate forward', + formatResult: () => 'Navigated forward', + } + ), + + createBridgeTool( + bridge, + 'browser_reload', + 'Reload the current page.', + {}, + { + errorPrefix: 'Failed to reload', + formatResult: () => 'Page reloaded', + } + ), + + createBridgeTool( + bridge, + 'browser_close', + 'Close the browser panel.', + {}, + { + errorPrefix: 'Failed to close browser', + formatResult: () => 'Browser closed', + } + ), + + createBridgeTool( + bridge, 'browser_eval', 'Execute JavaScript in the browser and return the result. Use `return` to output a value.', { script: z.string().describe('JavaScript code to execute'), }, - async (args) => { - logger.debug('Evaluating script'); - try { - const result = await bridge.sendRequest('browser_eval', args); - return { - content: [{ type: 'text' as const, text: safeStringify(result) }], - }; - } catch (error: unknown) { - return { - content: [ - { - type: 'text' as const, - text: `Failed to evaluate script: ${error instanceof Error ? error.message : String(error)}`, - }, - ], - isError: true, - }; - } + { + errorPrefix: 'Failed to evaluate script', + } + ), + + createBridgeTool( + bridge, + 'browser_snapshot', + 'Capture an accessibility snapshot of the current page. This is the primary inspection tool: it returns a semantic tree with stable ref handles for later actions.', + { + interactive: z + .boolean() + .optional() + .describe('When false, include refs on all elements, not only interactive ones'), + cursor: z + .boolean() + .optional() + .describe('Include cursor:pointer or tabindex elements as ref targets'), + compact: z + .boolean() + .optional() + .describe('Omit non-ref-bearing nodes from the snapshot output'), + }, + { + errorPrefix: 'Failed to capture snapshot', + } + ), + + createBridgeTool( + bridge, + 'browser_get_url', + 'Get the current page URL. Useful after navigation or redirects.', + {}, + { + errorPrefix: 'Failed to get URL', + } + ), + + createBridgeTool( + bridge, + 'browser_get_title', + 'Get the current page title.', + {}, + { + errorPrefix: 'Failed to get title', + } + ), + + createBridgeTool( + bridge, + 'browser_check', + 'Check a checkbox or switch by snapshot ref (preferred) or CSS selector.', + { + ...BrowserTargetShape, + }, + { + errorPrefix: 'Failed to check element', + validateArgs: validateRequiredTarget, + } + ), + + createBridgeTool( + bridge, + 'browser_uncheck', + 'Uncheck a checkbox or switch by snapshot ref (preferred) or CSS selector.', + { + ...BrowserTargetShape, + }, + { + errorPrefix: 'Failed to uncheck element', + validateArgs: validateRequiredTarget, + } + ), + + createBridgeTool( + bridge, + 'browser_select', + 'Select option values on a dropdown by snapshot ref (preferred) or CSS selector.', + { + ...BrowserTargetShape, + values: z.array(z.string()).min(1).describe('Option values to select'), + }, + { + errorPrefix: 'Failed to select option', + validateArgs: validateRequiredTarget, + } + ), + + createBridgeTool( + bridge, + 'browser_hover', + 'Hover over an element by snapshot ref (preferred) or CSS selector.', + { + ...BrowserTargetShape, + }, + { + errorPrefix: 'Failed to hover element', + validateArgs: validateRequiredTarget, + } + ), + + createBridgeTool( + bridge, + 'browser_focus', + 'Focus an element by snapshot ref (preferred) or CSS selector.', + { + ...BrowserTargetShape, + }, + { + errorPrefix: 'Failed to focus element', + validateArgs: validateRequiredTarget, + } + ), + + createBridgeTool( + bridge, + 'browser_scroll', + 'Scroll the page or a specific element. Optional ref targets a specific scroll container; otherwise the page is scrolled.', + { + ...BrowserTargetShape, + direction: z + .enum(BrowserScrollDirections) + .describe('Scroll direction: up, down, left, or right'), + amount: z.number().positive().optional().describe('Optional scroll amount in pixels'), + }, + { + errorPrefix: 'Failed to scroll', + } + ), + + createBridgeTool( + bridge, + 'browser_scroll_into_view', + 'Scroll an element into view by snapshot ref (preferred) or CSS selector.', + { + ...BrowserTargetShape, + }, + { + errorPrefix: 'Failed to scroll element into view', + validateArgs: validateRequiredTarget, + } + ), + + createBridgeTool( + bridge, + 'browser_wait_for_selector', + 'Wait for a CSS selector to reach the requested state. Use this when content appears asynchronously and no ref exists yet.', + { + selector: z.string().describe('CSS selector to wait for'), + state: z + .enum(BrowserWaitForSelectorStates) + .optional() + .describe('Desired selector state (default: visible)'), + timeout: z + .number() + .int() + .positive() + .max(120000) + .optional() + .describe('Timeout in milliseconds (max 120000)'), + }, + { + errorPrefix: 'Failed to wait for selector', + } + ), + + createBridgeTool( + bridge, + 'browser_wait_for_url', + 'Wait for the current page URL to match an exact string or a regex pattern written as /pattern/flags.', + { + url: z.string().describe('Exact URL or regex string in /pattern/flags form'), + timeout: z + .number() + .int() + .positive() + .max(120000) + .optional() + .describe('Timeout in milliseconds (max 120000)'), + }, + { + errorPrefix: 'Failed to wait for URL', + } + ), + + createBridgeTool( + bridge, + 'browser_is_visible', + 'Check whether an element is visible. Supports ref (preferred) or selector.', + { + ...BrowserTargetShape, + }, + { + errorPrefix: 'Failed to inspect visibility', + validateArgs: validateRequiredTarget, + } + ), + + createBridgeTool( + bridge, + 'browser_is_enabled', + 'Check whether an element is enabled. Supports ref (preferred) or selector.', + { + ...BrowserTargetShape, + }, + { + errorPrefix: 'Failed to inspect enabled state', + validateArgs: validateRequiredTarget, + } + ), + + createBridgeTool( + bridge, + 'browser_get_attribute', + 'Get an attribute from an element. Supports ref (preferred) or selector.', + { + ...BrowserTargetShape, + name: z.string().describe('Attribute name'), + }, + { + errorPrefix: 'Failed to get attribute', + validateArgs: validateRequiredTarget, + } + ), + + createBridgeTool( + bridge, + 'browser_bounding_box', + 'Get the bounding box of an element. Supports ref (preferred) or selector.', + { + ...BrowserTargetShape, + }, + { + errorPrefix: 'Failed to get bounding box', + validateArgs: validateRequiredTarget, + } + ), + + createBridgeTool( + bridge, + 'browser_count', + 'Count the number of elements matching a CSS selector.', + { + selector: z.string().describe('CSS selector to count'), + }, + { + errorPrefix: 'Failed to count elements', + } + ), + + createBridgeTool( + bridge, + 'browser_cookies_get', + 'Get cookies for the current page, optionally filtered by name or domain.', + { + name: z.string().optional().describe('Optional cookie name filter'), + domain: z.string().optional().describe('Optional cookie domain filter'), + }, + { + errorPrefix: 'Failed to get cookies', + } + ), + + createBridgeTool( + bridge, + 'browser_cookies_clear', + 'Clear cookies for the current page, optionally filtered by name or domain.', + { + name: z.string().optional().describe('Optional cookie name filter'), + domain: z.string().optional().describe('Optional cookie domain filter'), + }, + { + errorPrefix: 'Failed to clear cookies', + } + ), + + createBridgeTool( + bridge, + 'browser_storage_get', + 'Get a value from localStorage or sessionStorage.', + { + key: z.string().describe('Storage key'), + store: z + .enum(BrowserStorageStores) + .optional() + .describe('Storage type: local (default) or session'), + }, + { + errorPrefix: 'Failed to get storage value', + } + ), + + createBridgeTool( + bridge, + 'browser_storage_set', + 'Set a value in localStorage or sessionStorage.', + { + key: z.string().describe('Storage key'), + value: z.string().describe('Storage value'), + store: z + .enum(BrowserStorageStores) + .optional() + .describe('Storage type: local (default) or session'), + }, + { + errorPrefix: 'Failed to set storage value', + } + ), + + createBridgeTool( + bridge, + 'browser_storage_clear', + 'Clear localStorage or sessionStorage.', + { + store: z + .enum(BrowserStorageStores) + .optional() + .describe('Storage type: local (default) or session'), + }, + { + errorPrefix: 'Failed to clear storage', + } + ), + + createBridgeTool( + bridge, + 'browser_network_requests', + 'Get captured network requests since the last read or page load. Useful for debugging failed page loads, APIs, or redirects.', + { + filter: z + .object({ + url: z.string().optional(), + method: z.string().optional(), + status: z.number().int().optional(), + }) + .optional() + .describe('Optional request filter'), + }, + { + errorPrefix: 'Failed to get network requests', + } + ), + + createBridgeTool( + bridge, + 'browser_runtime_info', + 'Get the injected browser runtime version, epoch, and capability state. Use this to debug runtime injection issues such as CSP blocking.', + {}, + { + errorPrefix: 'Failed to get runtime info', } ), ]; return createSdkMcpServer({ - name: 'orbit-browser', - version: '1.0.0', + name: MCP_SERVER_NAME, + version: MCP_SERVER_VERSION, tools, }); } @@ -375,19 +761,5 @@ export function createBrowserMcpServer( * Get full tool names with MCP prefix. */ export function getBrowserToolNames(): string[] { - return [ - 'mcp__orbit-browser__browser_open', - 'mcp__orbit-browser__browser_navigate', - 'mcp__orbit-browser__browser_click', - 'mcp__orbit-browser__browser_type', - 'mcp__orbit-browser__browser_get_text', - 'mcp__orbit-browser__browser_get_html', - 'mcp__orbit-browser__browser_screenshot', - 'mcp__orbit-browser__browser_console_logs', - 'mcp__orbit-browser__browser_back', - 'mcp__orbit-browser__browser_forward', - 'mcp__orbit-browser__browser_reload', - 'mcp__orbit-browser__browser_close', - 'mcp__orbit-browser__browser_eval', - ]; + return BROWSER_TOOL_NAMES.map((toolName) => `mcp__${MCP_SERVER_NAME}__${toolName}`); } diff --git a/agent-bridge/src/browser/browser-tool-bridge.ts b/agent-bridge/src/browser/browser-tool-bridge.ts index 0f67cf43f..97d1d465f 100644 --- a/agent-bridge/src/browser/browser-tool-bridge.ts +++ b/agent-bridge/src/browser/browser-tool-bridge.ts @@ -16,6 +16,31 @@ import { createLogger } from '../common/logging/logger.js'; import type { McpToolRequest, McpToolResponse } from './types.js'; const logger = createLogger('BrowserToolBridge'); +const DEFAULT_TIMEOUT_MS = 30000; +const ACTION_TIMEOUT_MS = 10000; +const SNAPSHOT_TIMEOUT_MS = 15000; +const MAX_WAIT_TIMEOUT_MS = 120000; +const STATEFUL_BROWSER_TOOLS = new Set([ + 'browser_open', + 'browser_close', + 'browser_navigate', + 'browser_back', + 'browser_forward', + 'browser_reload', + 'browser_snapshot', + 'browser_click', + 'browser_type', + 'browser_fill', + 'browser_select', + 'browser_check', + 'browser_uncheck', + 'browser_hover', + 'browser_focus', + 'browser_scroll', + 'browser_scroll_into_view', + 'browser_wait_for_selector', + 'browser_wait_for_url', +]); /** * Pending tool request with resolver @@ -27,6 +52,64 @@ interface PendingRequest { toolName: string; } +class AsyncQueue { + private tail: Promise = Promise.resolve(); + + enqueue(task: () => Promise): Promise { + const run = this.tail.catch(() => undefined).then(task); + this.tail = run.then( + () => undefined, + () => undefined + ); + return run; + } +} + +function clampWaitTimeout(toolInput: Record): number { + const timeout = toolInput.timeout; + if (typeof timeout !== 'number' || !Number.isFinite(timeout) || timeout <= 0) { + return DEFAULT_TIMEOUT_MS; + } + return Math.min(Math.trunc(timeout), MAX_WAIT_TIMEOUT_MS); +} + +export function getBrowserToolTimeoutMs( + toolName: string, + toolInput: Record +): number { + switch (toolName) { + case 'browser_snapshot': + return SNAPSHOT_TIMEOUT_MS; + case 'browser_click': + case 'browser_type': + case 'browser_fill': + case 'browser_select': + case 'browser_check': + case 'browser_uncheck': + case 'browser_hover': + case 'browser_focus': + case 'browser_scroll': + case 'browser_scroll_into_view': + return ACTION_TIMEOUT_MS; + case 'browser_wait_for_selector': + case 'browser_wait_for_url': + return clampWaitTimeout(toolInput); + case 'browser_open': + case 'browser_close': + case 'browser_navigate': + case 'browser_back': + case 'browser_forward': + case 'browser_reload': + return DEFAULT_TIMEOUT_MS; + default: + return DEFAULT_TIMEOUT_MS; + } +} + +export function isStatefulBrowserTool(toolName: string): boolean { + return STATEFUL_BROWSER_TOOLS.has(toolName); +} + /** * Callback type for tool request events */ @@ -39,11 +122,11 @@ export type ToolRequestCallback = (request: McpToolRequest) => void; * webview/frontend communication. */ export class BrowserToolBridge { - private static readonly TIMEOUT_MS = 30000; // 30 second timeout - // Pending requests waiting for response private readonly pendingRequests = new Map(); + private readonly statefulQueue = new AsyncQueue(); + // Event emitter for tool requests private readonly emitter = new EventEmitter(); @@ -107,29 +190,36 @@ export class BrowserToolBridge { * @returns Promise that resolves with the tool result */ async sendRequest(toolName: string, toolInput: Record): Promise { + if (isStatefulBrowserTool(toolName)) { + return this.statefulQueue.enqueue(() => this.sendRequestInternal(toolName, toolInput)); + } + + return this.sendRequestInternal(toolName, toolInput); + } + + /** + * [warning] TESTED: This bridge is covered by integration tests. + * If you modify this, run: cd agent-bridge && bun test + * Test files: src/__tests__/browser-ref-flow.test.ts, src/__tests__/browser-timeout.test.ts + */ + private async sendRequestInternal( + toolName: string, + toolInput: Record + ): Promise { if (this.disposed) { throw new Error('BrowserToolBridge has been disposed'); } - // Generate unique request ID with timestamp, random string, and counter to prevent collisions const requestId = `browser-${String(Date.now())}-${Math.random().toString(36).slice(2, 11)}-${String(this.pendingRequests.size)}`; + const timeoutMs = getBrowserToolTimeoutMs(toolName, toolInput); return new Promise((resolve, reject) => { - // Set up timeout const timeout = setTimeout(() => { this.pendingRequests.delete(requestId); - logger.warn( - { requestId, toolName, timeoutMs: BrowserToolBridge.TIMEOUT_MS }, - 'Tool request timed out' - ); - reject( - new Error( - `Tool request '${toolName}' timed out after ${String(BrowserToolBridge.TIMEOUT_MS)}ms` - ) - ); - }, BrowserToolBridge.TIMEOUT_MS); - - // Store pending request + logger.warn({ requestId, toolName, timeoutMs }, 'Tool request timed out'); + reject(new Error(`${toolName} timed out after ${String(timeoutMs)}ms`)); + }, timeoutMs); + this.pendingRequests.set(requestId, { resolve: resolve as (result: unknown) => void, reject, @@ -137,7 +227,6 @@ export class BrowserToolBridge { toolName, }); - // Emit request to registered callbacks const request: McpToolRequest = { requestId, toolName, diff --git a/agent-bridge/src/browser/index.ts b/agent-bridge/src/browser/index.ts index 944058a07..0e35aef3b 100644 --- a/agent-bridge/src/browser/index.ts +++ b/agent-bridge/src/browser/index.ts @@ -15,27 +15,75 @@ export { createBrowserMcpServer, getBrowserToolNames } from './browser-mcp-serve // Types export type { + BrowserTargetInput, + BrowserOptionalTargetInput, + BrowserScrollDirection, + BrowserStorageStore, + BrowserConsoleLogLevel, + BrowserWaitForSelectorState, BrowserToolResult, BrowserToolResultTyped, BrowserToolError, + BrowserCookie, + BrowserNetworkRequest, + BrowserBoundingBox, + BrowserRuntimeInfo, + SnapshotResponse, BrowserCommand, BrowserCommandType, BrowserOpenCommand, BrowserNavigateCommand, + BrowserSnapshotCommand, + BrowserGetUrlCommand, + BrowserGetTitleCommand, BrowserClickCommand, BrowserTypeCommand, + BrowserFillCommand, + BrowserSelectCommand, + BrowserCheckCommand, + BrowserUncheckCommand, + BrowserHoverCommand, + BrowserFocusCommand, + BrowserScrollCommand, + BrowserScrollIntoViewCommand, + BrowserWaitForSelectorCommand, + BrowserWaitForUrlCommand, BrowserEvalCommand, BrowserScreenshotCommand, BrowserGetHtmlCommand, BrowserGetTextCommand, + BrowserIsVisibleCommand, + BrowserIsEnabledCommand, + BrowserGetAttributeCommand, + BrowserBoundingBoxCommand, + BrowserCountCommand, + BrowserCookiesGetCommand, + BrowserCookiesClearCommand, + BrowserStorageGetCommand, + BrowserStorageSetCommand, + BrowserStorageClearCommand, + BrowserNetworkRequestsCommand, BrowserBackCommand, BrowserForwardCommand, BrowserReloadCommand, BrowserCloseCommand, BrowserConsoleLogsCommand, + BrowserRuntimeInfoCommand, BrowserScreenshotData, BrowserConsoleLogEntry, } from './types.js'; // Type guards -export { isSuccess, isError } from './types.js'; +export { + BrowserTargetShape, + TargetSchema, + OptionalTargetSchema, + BrowserScrollDirections, + BrowserStorageStores, + BrowserConsoleLogLevels, + BrowserWaitForSelectorStates, + validateTargetInput, + validateOptionalTargetInput, + isSuccess, + isError, +} from './types.js'; diff --git a/agent-bridge/src/browser/types.ts b/agent-bridge/src/browser/types.ts index 7f56d13d4..63b5cbc72 100644 --- a/agent-bridge/src/browser/types.ts +++ b/agent-bridge/src/browser/types.ts @@ -1,59 +1,86 @@ /** * Browser MCP Types * - * Type definitions for browser automation commands used by AI agents. - * These types define the contract between the agent and the browser - * automation layer. + * Shared browser-tool input contracts and lightweight result types used by the + * embedded browser MCP server and its frontend bridge. */ +import { z } from 'zod'; + +// ============================================ +// Shared Schemas +// ============================================ + +export const BrowserTargetShape = { + ref: z.string().optional().describe('Element ref from browser_snapshot (preferred)'), + selector: z.string().optional().describe('CSS selector (legacy, still supported)'), +} satisfies Record; + +export const OptionalTargetSchema = z.object(BrowserTargetShape); + +export const TargetSchema = OptionalTargetSchema.refine( + (value) => value.ref !== undefined || value.selector !== undefined, + 'Provide ref (from snapshot) or selector (CSS)' +); + +export const BrowserScrollDirections = ['up', 'down', 'left', 'right'] as const; +export const BrowserStorageStores = ['local', 'session'] as const; +export const BrowserConsoleLogLevels = ['error', 'warn', 'info', 'log'] as const; +export const BrowserWaitForSelectorStates = ['attached', 'detached', 'hidden', 'visible'] as const; + +export type BrowserTargetInput = z.infer; +export type BrowserOptionalTargetInput = z.infer; +export type BrowserScrollDirection = (typeof BrowserScrollDirections)[number]; +export type BrowserStorageStore = (typeof BrowserStorageStores)[number]; +export type BrowserConsoleLogLevel = (typeof BrowserConsoleLogLevels)[number]; +export type BrowserWaitForSelectorState = (typeof BrowserWaitForSelectorStates)[number]; + +function getValidationErrorMessage(error: z.ZodError): string { + return error.issues[0]?.message ?? 'Invalid browser tool input'; +} + +export function validateTargetInput(input: unknown): BrowserTargetInput { + const result = TargetSchema.safeParse(input); + if (!result.success) { + throw new Error(getValidationErrorMessage(result.error)); + } + return result.data; +} + +export function validateOptionalTargetInput(input: unknown): BrowserOptionalTargetInput { + const result = OptionalTargetSchema.safeParse(input); + if (!result.success) { + throw new Error(getValidationErrorMessage(result.error)); + } + return result.data; +} + // ============================================ // Result Types // ============================================ -/** - * Result of a browser tool execution. - */ export interface BrowserToolResult { - /** Whether the command succeeded */ success: boolean; - /** Result data (type depends on the command) */ data?: unknown; - /** Error message if success is false */ error?: string; } -/** - * Typed result with known data shape. - */ export interface BrowserToolResultTyped extends BrowserToolResult { success: true; data: T; } -/** - * Error result type. - */ export interface BrowserToolError extends BrowserToolResult { success: false; error: string; } -// ============================================ -// MCP Tool Types -// ============================================ - -/** - * MCP tool request sent to the frontend. - */ export interface McpToolRequest { requestId: string; toolName: string; toolInput: Record; } -/** - * MCP tool response from the frontend. - */ export interface McpToolResponse { requestId: string; success: boolean; @@ -61,180 +88,312 @@ export interface McpToolResponse { error?: string; } +export interface BrowserCookie { + name: string; + value: string; + domain: string | null; + path: string | null; + expires: number | null; +} + +export interface BrowserNetworkRequest { + url: string; + method: string; + status: number; + duration: number; + size: number; +} + +export interface BrowserBoundingBox { + x: number; + y: number; + width: number; + height: number; +} + +export interface BrowserScreenshotData { + url: string; + title: string; + width: number; + height: number; + scrollX: number; + scrollY: number; + devicePixelRatio: number; +} + +export interface BrowserConsoleLogEntry { + level: BrowserConsoleLogLevel; + message: string; + timestamp: number; +} + +export interface BrowserRuntimeInfo { + available: boolean; + reason?: string; + version?: string; + epoch?: number; + capabilities?: { + snapshot: boolean; + refResolution: boolean; + consoleCapture: boolean; + networkCapture: boolean; + storageAccess: boolean; + }; +} + +export interface SnapshotResponse { + epoch: number; + snapshot: string; + refCount: number; + totalElements: number; + emittedElements: number; + truncated: boolean; + url: string; + title: string; + durationMs: number; +} + // ============================================ // Command Types // ============================================ -/** - * Open a browser or navigate to a URL. - */ export interface BrowserOpenCommand { type: 'browser:open'; - /** URL to open (optional, opens blank if not provided) */ url?: string; } -/** - * Navigate to a specific URL. - */ export interface BrowserNavigateCommand { type: 'browser:navigate'; - /** URL to navigate to */ url: string; } -/** - * Click an element by CSS selector. - */ -export interface BrowserClickCommand { +export interface BrowserSnapshotCommand { + type: 'browser:snapshot'; + interactive?: boolean; + cursor?: boolean; + compact?: boolean; +} + +export interface BrowserGetUrlCommand { + type: 'browser:get_url'; +} + +export interface BrowserGetTitleCommand { + type: 'browser:get_title'; +} + +export interface BrowserClickCommand extends BrowserTargetInput { type: 'browser:click'; - /** CSS selector for the element to click */ - selector: string; } -/** - * Type text into an input element. - */ -export interface BrowserTypeCommand { +export interface BrowserTypeCommand extends BrowserTargetInput { type: 'browser:type'; - /** CSS selector for the input element */ - selector: string; - /** Text to type */ text: string; } -/** - * Execute JavaScript and return the result. - */ +export interface BrowserFillCommand extends BrowserTargetInput { + type: 'browser:fill'; + value: string; +} + +export interface BrowserSelectCommand extends BrowserTargetInput { + type: 'browser:select'; + values: string[]; +} + +export interface BrowserCheckCommand extends BrowserTargetInput { + type: 'browser:check'; +} + +export interface BrowserUncheckCommand extends BrowserTargetInput { + type: 'browser:uncheck'; +} + +export interface BrowserHoverCommand extends BrowserTargetInput { + type: 'browser:hover'; +} + +export interface BrowserFocusCommand extends BrowserTargetInput { + type: 'browser:focus'; +} + +export interface BrowserScrollCommand extends BrowserOptionalTargetInput { + type: 'browser:scroll'; + direction: BrowserScrollDirection; + amount?: number; +} + +export interface BrowserScrollIntoViewCommand extends BrowserTargetInput { + type: 'browser:scroll_into_view'; +} + +export interface BrowserWaitForSelectorCommand { + type: 'browser:wait_for_selector'; + selector: string; + state?: BrowserWaitForSelectorState; + timeout?: number; +} + +export interface BrowserWaitForUrlCommand { + type: 'browser:wait_for_url'; + url: string; + timeout?: number; +} + export interface BrowserEvalCommand { type: 'browser:eval'; - /** JavaScript code to execute */ script: string; } -/** - * Capture a screenshot or page info. - */ export interface BrowserScreenshotCommand { type: 'browser:screenshot'; } -/** - * Get HTML content of an element or the entire page. - */ -export interface BrowserGetHtmlCommand { +export interface BrowserGetHtmlCommand extends BrowserOptionalTargetInput { type: 'browser:get_html'; - /** CSS selector (optional, returns document HTML if not provided) */ - selector?: string; + outer?: boolean; } -/** - * Get text content of an element or the entire page. - */ -export interface BrowserGetTextCommand { +export interface BrowserGetTextCommand extends BrowserOptionalTargetInput { type: 'browser:get_text'; - /** CSS selector (optional, returns document text if not provided) */ - selector?: string; } -/** - * Navigate back in browser history. - */ +export interface BrowserIsVisibleCommand extends BrowserTargetInput { + type: 'browser:is_visible'; +} + +export interface BrowserIsEnabledCommand extends BrowserTargetInput { + type: 'browser:is_enabled'; +} + +export interface BrowserGetAttributeCommand extends BrowserTargetInput { + type: 'browser:get_attribute'; + name: string; +} + +export interface BrowserBoundingBoxCommand extends BrowserTargetInput { + type: 'browser:bounding_box'; +} + +export interface BrowserCountCommand { + type: 'browser:count'; + selector: string; +} + +export interface BrowserCookiesGetCommand { + type: 'browser:cookies_get'; + name?: string; + domain?: string; +} + +export interface BrowserCookiesClearCommand { + type: 'browser:cookies_clear'; + name?: string; + domain?: string; +} + +export interface BrowserStorageGetCommand { + type: 'browser:storage_get'; + key: string; + store?: BrowserStorageStore; +} + +export interface BrowserStorageSetCommand { + type: 'browser:storage_set'; + key: string; + value: string; + store?: BrowserStorageStore; +} + +export interface BrowserStorageClearCommand { + type: 'browser:storage_clear'; + store?: BrowserStorageStore; +} + +export interface BrowserNetworkRequestsCommand { + type: 'browser:network_requests'; + filter?: { + url?: string; + method?: string; + status?: number; + }; +} + +export interface BrowserConsoleLogsCommand { + type: 'browser:console_logs'; + level?: BrowserConsoleLogLevel; +} + export interface BrowserBackCommand { type: 'browser:back'; } -/** - * Navigate forward in browser history. - */ export interface BrowserForwardCommand { type: 'browser:forward'; } -/** - * Reload the current page. - */ export interface BrowserReloadCommand { type: 'browser:reload'; } -/** - * Close the browser. - */ export interface BrowserCloseCommand { type: 'browser:close'; } -/** - * Get console logs from the browser. - */ -export interface BrowserConsoleLogsCommand { - type: 'browser:console_logs'; +export interface BrowserRuntimeInfoCommand { + type: 'browser:runtime_info'; } -/** - * Union type of all browser commands. - */ export type BrowserCommand = | BrowserOpenCommand | BrowserNavigateCommand + | BrowserSnapshotCommand + | BrowserGetUrlCommand + | BrowserGetTitleCommand | BrowserClickCommand | BrowserTypeCommand + | BrowserFillCommand + | BrowserSelectCommand + | BrowserCheckCommand + | BrowserUncheckCommand + | BrowserHoverCommand + | BrowserFocusCommand + | BrowserScrollCommand + | BrowserScrollIntoViewCommand + | BrowserWaitForSelectorCommand + | BrowserWaitForUrlCommand | BrowserEvalCommand | BrowserScreenshotCommand | BrowserGetHtmlCommand | BrowserGetTextCommand + | BrowserIsVisibleCommand + | BrowserIsEnabledCommand + | BrowserGetAttributeCommand + | BrowserBoundingBoxCommand + | BrowserCountCommand + | BrowserCookiesGetCommand + | BrowserCookiesClearCommand + | BrowserStorageGetCommand + | BrowserStorageSetCommand + | BrowserStorageClearCommand + | BrowserNetworkRequestsCommand + | BrowserConsoleLogsCommand | BrowserBackCommand | BrowserForwardCommand | BrowserReloadCommand | BrowserCloseCommand - | BrowserConsoleLogsCommand; + | BrowserRuntimeInfoCommand; -/** - * Extract the command type string from a BrowserCommand. - */ export type BrowserCommandType = BrowserCommand['type']; -// ============================================ -// Data Types for Specific Commands -// ============================================ - -/** - * Screenshot/page info data. - */ -export interface BrowserScreenshotData { - url: string; - title: string; - width: number; - height: number; - scrollX: number; - scrollY: number; - devicePixelRatio: number; -} - -/** - * Console log entry. - */ -export interface BrowserConsoleLogEntry { - level: 'log' | 'info' | 'warn' | 'error'; - message: string; - timestamp: number; -} - // ============================================ // Helper Type Guards // ============================================ -/** - * Type guard to check if a result is successful. - */ export function isSuccess(result: BrowserToolResult): result is BrowserToolResultTyped { return result.success; } -/** - * Type guard to check if a result is an error. - */ export function isError(result: BrowserToolResult): result is BrowserToolError { return !result.success; } diff --git a/agent-bridge/src/common/batching/text-event-batcher.ts b/agent-bridge/src/common/batching/text-event-batcher.ts index edb20d4c3..3f7e4dc17 100644 --- a/agent-bridge/src/common/batching/text-event-batcher.ts +++ b/agent-bridge/src/common/batching/text-event-batcher.ts @@ -2,7 +2,8 @@ * Text Event Batcher - Reduces event flooding from Claude Agent SDK * * The SDK emits ~200 text events per response (one per token/word). - * This batcher accumulates them at 50ms intervals before forwarding to frontend. + * This batcher can accumulate them over a short interval before forwarding to + * frontend, or emit immediately when configured with a non-positive interval. * * Result: ~200 events → ~4-8 batched events per response * @@ -68,6 +69,20 @@ export class TextEventBatcher { if (content === '') return; const key = `${sessionId}${KEY_DELIMITER}${messageId}`; + + // Track total accumulated length (for contentOffset calculation) + const currentLength = this.accumulatedLengths.get(key) ?? 0; + this.accumulatedLengths.set(key, currentLength + content.length); + + if (this.BATCH_INTERVAL <= 0) { + try { + this.emitFn({ sessionId, messageId, content }); + } catch (error) { + logger.error({ sessionId, error }, 'Failed to emit immediate text event'); + } + return; + } + const existing = this.buffer.get(key); if (existing) { @@ -81,10 +96,6 @@ export class TextEventBatcher { this.buffer.set(key, { sessionId, messageId, content }); } - // Track total accumulated length (for contentOffset calculation) - const currentLength = this.accumulatedLengths.get(key) ?? 0; - this.accumulatedLengths.set(key, currentLength + content.length); - // Schedule flush if not already scheduled (nullish coalescing assignment) this.timer ??= setTimeout(() => { this.flush(); diff --git a/agent-bridge/src/index.ts b/agent-bridge/src/index.ts index 93ae53ff6..dcf5e471a 100644 --- a/agent-bridge/src/index.ts +++ b/agent-bridge/src/index.ts @@ -60,9 +60,11 @@ import { } from './agent/session/session-storage.js'; import { CanvasSessionManager } from './canvas/index.js'; import { createLogger } from './common/logging/logger.js'; +import { canEnableIOS } from './ios/ios-service.js'; import { BridgeRequestSchema } from './protocol/schemas.js'; import { captureAgentError, captureFatalError, initSentry } from './sentry/index.js'; +import type { IOSService } from './ios/ios-service.js'; import type { BridgeCommandResponse, BridgeEvent, @@ -118,7 +120,7 @@ function setupGlobalErrorHandlers(): void { /** * Main entry point */ -function main(): void { +async function main(): Promise { // Initialize Sentry FIRST for early error capture initSentry(); @@ -131,8 +133,14 @@ function main(): void { // This handles cases where the bridge process was killed and restarted invalidateSessionCache(); + let iosService: IOSService | undefined; + if (await canEnableIOS()) { + const { IOSService } = await import('./ios/ios-service.js'); + iosService = new IOSService(); + } + // Create session managers - const sessionManager = new SessionManager(); + const sessionManager = new SessionManager({ iosService }); const canvasSessionManager = new CanvasSessionManager(); // Wire up agent session event handlers @@ -329,21 +337,23 @@ function main(): void { logger.info({ requestType: request.type }, 'Received request'); - handleRequest(request, sessionManager, canvasSessionManager).catch((error: unknown) => { - const errorMessage = error instanceof Error ? error.message : String(error); - logger.error({ requestType: request.type, error: errorMessage }, 'Error handling request'); - // Capture request handler errors to Sentry - const handlerError = error instanceof Error ? error : new Error(errorMessage); - captureAgentError(handlerError, { - source: 'request_handler', - extra: { requestType: request.type }, - }); - sendResponse({ - type: 'error', - requestType: request.type, - error: errorMessage, - }); - }); + handleRequest(request, sessionManager, canvasSessionManager, iosService).catch( + (error: unknown) => { + const errorMessage = error instanceof Error ? error.message : String(error); + logger.error({ requestType: request.type, error: errorMessage }, 'Error handling request'); + // Capture request handler errors to Sentry + const handlerError = error instanceof Error ? error : new Error(errorMessage); + captureAgentError(handlerError, { + source: 'request_handler', + extra: { requestType: request.type }, + }); + sendResponse({ + type: 'error', + requestType: request.type, + error: errorMessage, + }); + } + ); }); /** @@ -354,39 +364,42 @@ function main(): void { const SHUTDOWN_GRACE_MS = 500; let shuttingDown = false; - function gracefulShutdown(reason: string): void { + async function gracefulShutdown(reason: string): Promise { if (shuttingDown) return; shuttingDown = true; logger.info(`${reason}, shutting down...`); - // Wrap dispose calls in try/catch — if dispose throws synchronously, - // the process could exit with code 1 before the grace timeout fires, - // potentially corrupting session state files. - // (Code review: Opus cycle 1, issue #11) + try { - sessionManager.dispose(); + await sessionManager.shutdownAsync(); } catch (e) { - logger.error({ error: e }, 'sessionManager.dispose() failed'); + logger.error({ error: e }, 'sessionManager.shutdownAsync() failed'); + sessionManager.dispose(); } try { canvasSessionManager.dispose(); } catch (e) { logger.error({ error: e }, 'canvasSessionManager.dispose() failed'); } - // Allow async disposal (stopSession/interrupt) to complete before hard exit + try { + await iosService?.dispose(); + } catch (e) { + logger.error({ error: e }, 'iosService.dispose() failed'); + } + setTimeout(() => process.exit(0), SHUTDOWN_GRACE_MS); } rl.on('close', () => { - gracefulShutdown('stdin closed'); + void gracefulShutdown('stdin closed'); }); // Handle process signals process.on('SIGTERM', () => { - gracefulShutdown('SIGTERM received'); + void gracefulShutdown('SIGTERM received'); }); process.on('SIGINT', () => { - gracefulShutdown('SIGINT received'); + void gracefulShutdown('SIGINT received'); }); // Clean up old sessions on startup (30 days default) @@ -415,7 +428,8 @@ function main(): void { async function handleRequest( request: BridgeRequest, sessionManager: SessionManager, - canvasSessionManager: CanvasSessionManager + canvasSessionManager: CanvasSessionManager, + iosService?: IOSService ): Promise { switch (request.type) { case 'create_session': { @@ -680,10 +694,7 @@ async function handleRequest( } case 'generate_title': { - const title = await sessionManager.generateTitle( - request.userMessage, - request.assistantResponse - ); + const title = await sessionManager.generateTitle(request.userMessage); sendResponse({ type: 'string', requestType: request.type, value: title }); break; } @@ -728,8 +739,16 @@ async function handleRequest( case 'shutdown': { logger.info('Shutdown requested'); sendResponse({ type: 'success', requestType: request.type }); - sessionManager.dispose(); + try { + await sessionManager.shutdownAsync(); + } catch (error) { + logger.error({ error }, 'sessionManager.shutdownAsync() failed during shutdown request'); + sessionManager.dispose(); + } canvasSessionManager.dispose(); + await iosService?.dispose().catch((error: unknown) => { + logger.error({ error }, 'iosService.dispose() failed during shutdown request'); + }); process.exit(0); // Note: process.exit() never returns, but break needed to satisfy linter } @@ -747,12 +766,10 @@ async function handleRequest( } // Run main with Sentry error capture -try { - main(); -} catch (error: unknown) { +void main().catch((error: unknown) => { // Capture fatal error and flush Sentry before exit void captureFatalError(error, 'startup').finally(() => { logger.error({ error }, 'Fatal error'); process.exit(1); }); -} +}); diff --git a/agent-bridge/src/ios/index.ts b/agent-bridge/src/ios/index.ts new file mode 100644 index 000000000..78b1e133c --- /dev/null +++ b/agent-bridge/src/ios/index.ts @@ -0,0 +1,59 @@ +/** + * iOS module exports. + */ + +export { IOSManager } from './ios-manager.js'; +export { IOSService, canEnableIOS } from './ios-service.js'; +export { executeIOSAction } from './ios-actions.js'; +export { createIOSMcpServer, getIOSToolNames } from './ios-mcp-server.js'; +export type { + IOSAppiumProcess, + IOSAutomationSession, + IOSCheckResult, + IOSConsoleLogLevel, + IOSConsoleLogsResult, + IOSDevice, + IOSDeviceListResult, + IOSDeviceState, + IOSEvalResult, + IOSFillResult, + IOSHtmlResult, + IOSLaunchResult, + IOSLifecycleState, + IOSManagerDependencies, + IOSManagerFactory, + IOSManagerLike, + IOSManagerOptions, + IOSOptionalTargetInput, + IOSScrollDirection, + IOSScrollResult, + IOSSelectResult, + IOSSnapshotOptions, + IOSScreenshotResult, + IOSServiceOptions, + IOSSuccessResult, + IOSSwipeDirection, + IOSSwipeResult, + IOSTapResult, + IOSTargetInput, + IOSTextResult, + IOSToolName, + IOSTypeResult, + IOSUncheckResult, + IOSWaitForSelectorResult, + IOSWaitForSelectorState, +} from './types.js'; +export { + IOSConsoleLogLevelSchema, + IOSConsoleLogLevels, + IOSOptionalTargetSchema, + IOSScrollDirectionSchema, + IOSScrollDirections, + IOSSwipeDirectionSchema, + IOSSwipeDirections, + IOSTargetSchema, + IOSTargetShape, + IOSToolNames, + IOSWaitForSelectorStateSchema, + IOSWaitForSelectorStates, +} from './types.js'; diff --git a/agent-bridge/src/ios/ios-actions.ts b/agent-bridge/src/ios/ios-actions.ts new file mode 100644 index 000000000..2b41ad589 --- /dev/null +++ b/agent-bridge/src/ios/ios-actions.ts @@ -0,0 +1,197 @@ +/** + * iOS action dispatcher. + * + * [warning] TESTED: This dispatcher is covered by integration tests. + * If you modify this, run: cd agent-bridge && bun test + * Test file: src/__tests__/ios-mcp-server.test.ts + */ + +import { validateTargetInput } from '../browser/types.js'; + +import type { IOSService } from './ios-service.js'; +import type { + IOSCheckResult, + IOSConsoleLogLevel, + IOSConsoleLogsResult, + IOSDeviceListResult, + IOSEvalResult, + IOSFillResult, + IOSHtmlResult, + IOSLaunchResult, + IOSScrollResult, + IOSSelectResult, + IOSScreenshotResult, + IOSScrollDirection, + IOSSuccessResult, + IOSSwipeDirection, + IOSSwipeResult, + IOSTapResult, + IOSTargetInput, + IOSTextResult, + IOSToolName, + IOSTypeResult, + IOSUncheckResult, + IOSWaitForSelectorResult, + IOSWaitForSelectorState, +} from './types.js'; +import type { SnapshotResponse } from '../browser/types.js'; + +const DEFAULT_SWIPE_DURATION_MS = 300; +const DEFAULT_SCROLL_AMOUNT = 300; +const DEFAULT_WAIT_TIMEOUT_MS = 30000; +const MAX_WAIT_TIMEOUT_MS = 120000; + +function getRequiredTarget(args: Record): IOSTargetInput { + return validateTargetInput(args); +} + +function getOptionalNestedTarget( + args: Record, + key: string +): IOSTargetInput | undefined { + const rawTarget = args[key]; + if (rawTarget === undefined) { + return undefined; + } + + return validateTargetInput(rawTarget); +} + +function normalizeDuration(value: unknown): number { + if (typeof value !== 'number' || !Number.isFinite(value) || value <= 0) { + return DEFAULT_SWIPE_DURATION_MS; + } + return Math.trunc(value); +} + +function normalizeScrollAmount(value: unknown): number { + if (typeof value !== 'number' || !Number.isFinite(value) || value <= 0) { + return DEFAULT_SCROLL_AMOUNT; + } + return Math.trunc(value); +} + +function normalizeWaitTimeout(value: unknown): number { + if (typeof value !== 'number' || !Number.isFinite(value) || value <= 0) { + return DEFAULT_WAIT_TIMEOUT_MS; + } + return Math.min(Math.trunc(value), MAX_WAIT_TIMEOUT_MS); +} + +function requireLease(service: IOSService, sessionId: string): void { + if (!service.hasLease(sessionId)) { + throw new Error('No iOS lease for session'); + } +} + +export type IOSActionResult = + | IOSDeviceListResult + | IOSLaunchResult + | IOSSuccessResult + | SnapshotResponse + | IOSTextResult + | IOSHtmlResult + | IOSScreenshotResult + | IOSTapResult + | IOSFillResult + | IOSTypeResult + | IOSSelectResult + | IOSCheckResult + | IOSUncheckResult + | IOSSwipeResult + | IOSScrollResult + | IOSEvalResult + | IOSWaitForSelectorResult + | IOSConsoleLogsResult; + +export async function executeIOSAction( + service: IOSService, + sessionId: string, + toolName: IOSToolName, + args: Record +): Promise { + if (toolName === 'ios_device_list') { + const devices = await service.listDevices(); + return { devices }; + } + + if (toolName === 'ios_launch') { + const device = + typeof args.device === 'string' && args.device.length > 0 ? args.device : undefined; + const manager = await service.acquire(sessionId, device); + const result = manager.getLaunchResult(); + if (result === null) { + throw new Error('iOS not ready'); + } + return result; + } + + if (toolName === 'ios_close') { + requireLease(service, sessionId); + await service.release(sessionId); + return { success: true }; + } + + const manager = await service.getManager(sessionId); + + switch (toolName) { + case 'ios_navigate': + return manager.navigate(args.url as string); + case 'ios_back': + return manager.back(); + case 'ios_forward': + return manager.forward(); + case 'ios_reload': + return manager.reload(); + case 'ios_snapshot': + return manager.snapshot({ + interactive: args.interactive !== false, + cursor: args.cursor === true, + compact: args.compact === true, + }); + case 'ios_get_text': + return manager.getText(getRequiredTarget(args)); + case 'ios_get_html': + return manager.getHtml(getRequiredTarget(args), args.outer === true); + case 'ios_screenshot': + return manager.screenshot(); + case 'ios_tap': + return manager.tap(getRequiredTarget(args)); + case 'ios_fill': + return manager.fill(getRequiredTarget(args), args.value as string); + case 'ios_type': + return manager.type(getRequiredTarget(args), args.text as string); + case 'ios_select': + return manager.select(getRequiredTarget(args), args.values as string[]); + case 'ios_check': + return manager.check(getRequiredTarget(args)); + case 'ios_uncheck': + return manager.uncheck(getRequiredTarget(args)); + case 'ios_swipe': + return manager.swipe( + args.direction as IOSSwipeDirection, + getOptionalNestedTarget(args, 'target'), + normalizeDuration(args.duration) + ); + case 'ios_scroll': + return manager.scroll( + args.direction as IOSScrollDirection, + normalizeScrollAmount(args.amount) + ); + case 'ios_eval': + return manager.evaluate(args.script as string); + case 'ios_wait_for_selector': + return manager.waitForSelector( + args.selector as string, + args.state as IOSWaitForSelectorState | undefined, + normalizeWaitTimeout(args.timeout) + ); + case 'ios_console_logs': + return manager.consoleLogs(args.level as IOSConsoleLogLevel | undefined); + default: { + const exhaustiveCheck: never = toolName; + void exhaustiveCheck; + throw new Error('Unsupported iOS tool'); + } + } +} diff --git a/agent-bridge/src/ios/ios-manager.ts b/agent-bridge/src/ios/ios-manager.ts new file mode 100644 index 000000000..062df6338 --- /dev/null +++ b/agent-bridge/src/ios/ios-manager.ts @@ -0,0 +1,446 @@ +/** + * iOS simulator/Appium lifecycle manager. + * + * [warning] TESTED: This class is covered by integration-style lifecycle tests. + * If you modify this, run: cd agent-bridge && bun test + * Test file: src/__tests__/ios-manager-lifecycle.test.ts + */ + +import { createLogger } from '../common/logging/logger.js'; + +import type { + IOSAppiumProcess, + IOSAutomationSession, + IOSCheckResult, + IOSConsoleLogLevel, + IOSConsoleLogsResult, + IOSDevice, + IOSFillResult, + IOSHtmlResult, + IOSEvalResult, + IOSLaunchResult, + IOSLifecycleState, + IOSManagerDependencies, + IOSManagerLike, + IOSManagerOptions, + IOSScrollDirection, + IOSScrollResult, + IOSSelectResult, + IOSSnapshotOptions, + IOSScreenshotResult, + IOSSuccessResult, + IOSSwipeDirection, + IOSSwipeResult, + IOSTapResult, + IOSTargetInput, + IOSTextResult, + IOSTypeResult, + IOSUncheckResult, + IOSWaitForSelectorState, + IOSWaitForSelectorResult, +} from './types.js'; +import type { SnapshotResponse } from '../browser/types.js'; + +const logger = createLogger('IOSManager'); +const APPIUM_EXIT_TIMEOUT_MS = 5000; +const APPIUM_RESTART_MESSAGE_PREFIX = 'Appium process exited unexpectedly'; + +function createUnconfiguredDependencies(): IOSManagerDependencies { + const error = (): Error => new Error('IOSManager dependencies are not configured'); + + return { + listDevices(): Promise { + return Promise.reject(error()); + }, + bootDevice(): Promise { + return Promise.reject(error()); + }, + shutdownDevice(): Promise { + return Promise.reject(error()); + }, + startAppium(): Promise { + return Promise.reject(error()); + }, + createAutomationSession(): Promise { + return Promise.reject(error()); + }, + }; +} + +function normalizeExitCode(code: number | null): number { + return code ?? -1; +} + +function createLaunchAbortError(): Error { + return new Error('IOSManager launch aborted'); +} + +function throwIfAborted(signal: AbortSignal): void { + if (signal.aborted) { + throw createLaunchAbortError(); + } +} + +function isMatchingDevice(device: IOSDevice, query: string): boolean { + const normalizedQuery = query.trim().toLowerCase(); + const normalizedName = device.name.trim().toLowerCase(); + const normalizedUdid = device.udid.trim().toLowerCase(); + return ( + normalizedName === normalizedQuery || + normalizedName.includes(normalizedQuery) || + normalizedUdid === normalizedQuery + ); +} + +export class IOSManager implements IOSManagerLike { + private readonly dependencies: IOSManagerDependencies; + + private appiumProcess: IOSAppiumProcess | null = null; + private automationSession: IOSAutomationSession | null = null; + private activeDevice: IOSDevice | null = null; + private launchResult: IOSLaunchResult | null = null; + private appiumExitUnsubscribe: (() => void) | null = null; + private appiumExitCode: number | null = null; + private closePromise: Promise | null = null; + private launchAbortController: AbortController | null = null; + private _state: IOSLifecycleState = 'idle'; + + constructor(options: IOSManagerOptions = {}) { + this.dependencies = options.dependencies ?? createUnconfiguredDependencies(); + } + + get state(): IOSLifecycleState { + return this._state; + } + + getLaunchResult(): IOSLaunchResult | null { + return this.launchResult; + } + + async listDevices(): Promise { + this.assertNotDisposed(); + const controller = new AbortController(); + return this.dependencies.listDevices(controller.signal); + } + + assertUsable(): void { + if (this._state === 'disposed') { + throw new Error('IOSManager disposed'); + } + + if (this._state !== 'ready') { + throw new Error('iOS not ready'); + } + + if (this.appiumExitCode !== null) { + throw new Error( + `${APPIUM_RESTART_MESSAGE_PREFIX} (code ${String(this.appiumExitCode)}). Call ios_launch to restart.` + ); + } + } + + async launch(deviceQuery?: string): Promise { + this.assertNotDisposed(); + + if (this._state === 'ready' && this.launchResult !== null) { + return this.launchResult; + } + + if (this._state === 'launching') { + throw new Error('IOSManager launch already in progress'); + } + + if (this._state === 'closing') { + throw new Error('IOSManager is closing'); + } + + this._state = 'launching'; + this.appiumExitCode = null; + + const abortController = new AbortController(); + this.launchAbortController = abortController; + + try { + const device = await this.findDevice(deviceQuery, abortController.signal); + throwIfAborted(abortController.signal); + + await this.dependencies.bootDevice(device, abortController.signal); + throwIfAborted(abortController.signal); + + const appiumProcess = await this.dependencies.startAppium(abortController.signal); + throwIfAborted(abortController.signal); + + this.appiumProcess = appiumProcess; + this.appiumExitUnsubscribe = appiumProcess.onExit((code) => { + this.appiumExitCode = normalizeExitCode(code); + }); + + const automationSession = await this.dependencies.createAutomationSession( + device, + appiumProcess.port, + abortController.signal + ); + throwIfAborted(abortController.signal); + + this.automationSession = automationSession; + this.activeDevice = device; + this.launchResult = { + udid: device.udid, + name: device.name, + runtime: device.runtime, + appiumPort: appiumProcess.port, + }; + this._state = 'ready'; + + return this.launchResult; + } catch (error) { + await this.close().catch((closeError: unknown) => { + logger.warn({ error: closeError }, 'Failed to clean up after iOS launch error'); + }); + throw error; + } finally { + if (this.launchAbortController === abortController) { + this.launchAbortController = null; + } + } + } + + async close(): Promise { + if (this._state === 'disposed' || this._state === 'idle') { + return; + } + + if (this.closePromise !== null) { + return this.closePromise; + } + + this._state = 'closing'; + + const launchAbortController = this.launchAbortController; + this.launchAbortController = null; + + const automationSession = this.automationSession; + this.automationSession = null; + + const appiumProcess = this.appiumProcess; + this.appiumProcess = null; + + const appiumExitUnsubscribe = this.appiumExitUnsubscribe; + this.appiumExitUnsubscribe = null; + + const activeDevice = this.activeDevice; + this.activeDevice = null; + this.launchResult = null; + + this.closePromise = (async () => { + launchAbortController?.abort(); + + if (automationSession !== null) { + try { + await automationSession.deleteSession(); + } catch (error) { + logger.warn({ error }, 'Failed to delete iOS automation session'); + } + } + + appiumExitUnsubscribe?.(); + + if (appiumProcess !== null) { + await this.stopAppiumProcess(appiumProcess); + } + + if (activeDevice !== null) { + try { + await this.dependencies.shutdownDevice(activeDevice); + } catch (error) { + logger.warn({ error, udid: activeDevice.udid }, 'Failed to shut down iOS simulator'); + } + } + + this.appiumExitCode = null; + + if (this._state !== 'disposed') { + this._state = 'idle'; + } + })().finally(() => { + this.closePromise = null; + }); + + return this.closePromise; + } + + async dispose(): Promise { + if (this._state === 'disposed') { + return; + } + + await this.close(); + this._state = 'disposed'; + } + + async navigate(url: string): Promise { + await this.getAutomationSession().navigate(url); + return { success: true }; + } + + async back(): Promise { + await this.getAutomationSession().back(); + return { success: true }; + } + + async forward(): Promise { + await this.getAutomationSession().forward(); + return { success: true }; + } + + async reload(): Promise { + await this.getAutomationSession().reload(); + return { success: true }; + } + + async snapshot(options: IOSSnapshotOptions): Promise { + return this.getAutomationSession().snapshot(options); + } + + async getText(target: IOSTargetInput): Promise { + const text = await this.getAutomationSession().getText(target); + return { text }; + } + + async getHtml(target: IOSTargetInput, outer?: boolean): Promise { + const html = await this.getAutomationSession().getHtml(target, outer); + return { html }; + } + + async screenshot(): Promise { + return this.getAutomationSession().screenshot(); + } + + async tap(target: IOSTargetInput): Promise { + await this.getAutomationSession().tap(target); + return { tapped: true }; + } + + async fill(target: IOSTargetInput, value: string): Promise { + await this.getAutomationSession().fill(target, value); + return { filled: true }; + } + + async type(target: IOSTargetInput, text: string): Promise { + await this.getAutomationSession().type(target, text); + return { typed: true }; + } + + async select(target: IOSTargetInput, values: string[]): Promise { + await this.getAutomationSession().select(target, values); + return { selected: true }; + } + + async check(target: IOSTargetInput): Promise { + await this.getAutomationSession().check(target); + return { checked: true }; + } + + async uncheck(target: IOSTargetInput): Promise { + await this.getAutomationSession().uncheck(target); + return { unchecked: true }; + } + + async swipe( + direction: IOSSwipeDirection, + target?: IOSTargetInput, + duration?: number + ): Promise { + await this.getAutomationSession().swipe(direction, target, duration); + return { swiped: true }; + } + + async scroll(direction: IOSScrollDirection, amount?: number): Promise { + return this.getAutomationSession().scroll(direction, amount); + } + + async evaluate(script: string): Promise { + const result = await this.getAutomationSession().evaluate(script); + return { result }; + } + + async waitForSelector( + selector: string, + state?: IOSWaitForSelectorState, + timeout?: number + ): Promise { + await this.getAutomationSession().waitForSelector(selector, state, timeout); + return { matched: true }; + } + + async consoleLogs(level?: IOSConsoleLogLevel): Promise { + const logs = await this.getAutomationSession().consoleLogs(level); + return { logs }; + } + + private assertNotDisposed(): void { + if (this._state === 'disposed') { + throw new Error('IOSManager disposed'); + } + } + + private getAutomationSession(): IOSAutomationSession { + this.assertUsable(); + const session = this.automationSession; + if (session === null) { + throw new Error('iOS not ready'); + } + return session; + } + + private async findDevice(query: string | undefined, signal: AbortSignal): Promise { + const devices = await this.dependencies.listDevices(signal); + const availableDevices = devices.filter((device) => device.isAvailable); + + if (query === undefined || query.trim() === '') { + const bootedDevice = availableDevices.find((device) => device.state === 'Booted'); + if (bootedDevice !== undefined) { + return bootedDevice; + } + + const firstDevice = availableDevices[0]; + if (firstDevice !== undefined) { + return firstDevice; + } + } else { + const matchedDevice = availableDevices.find((device) => isMatchingDevice(device, query)); + if (matchedDevice !== undefined) { + return matchedDevice; + } + } + + throwIfAborted(signal); + + const availableNames = availableDevices.map((device) => device.name).join(', ') || 'none'; + const requestedDevice = query ?? 'default'; + throw new Error( + `No simulator found matching '${requestedDevice}'. Available: ${availableNames}` + ); + } + + private async stopAppiumProcess(appiumProcess: IOSAppiumProcess): Promise { + try { + await appiumProcess.terminate('SIGTERM'); + } catch (error) { + logger.warn({ error, pid: appiumProcess.pid }, 'Failed to send SIGTERM to Appium'); + } + + const exitCode = await appiumProcess.waitForExit(APPIUM_EXIT_TIMEOUT_MS); + if (exitCode !== null) { + return; + } + + try { + await appiumProcess.terminate('SIGKILL'); + } catch (error) { + logger.warn({ error, pid: appiumProcess.pid }, 'Failed to send SIGKILL to Appium'); + } + + await appiumProcess.waitForExit(APPIUM_EXIT_TIMEOUT_MS); + } +} diff --git a/agent-bridge/src/ios/ios-mcp-server.ts b/agent-bridge/src/ios/ios-mcp-server.ts new file mode 100644 index 000000000..f5068b659 --- /dev/null +++ b/agent-bridge/src/ios/ios-mcp-server.ts @@ -0,0 +1,336 @@ +/** + * iOS MCP server. + * + * [warning] TESTED: This server is covered by integration tests. + * If you modify this, run: cd agent-bridge && bun test + * Test file: src/__tests__/ios-mcp-server.test.ts + */ + +import { createSdkMcpServer, tool } from '@anthropic-ai/claude-agent-sdk'; +import { z } from 'zod'; + +import { createLogger } from '../common/logging/logger.js'; + +import { executeIOSAction } from './ios-actions.js'; +import { + IOSConsoleLogLevelSchema, + IOSScrollDirectionSchema, + IOSSwipeDirectionSchema, + IOSTargetShape, + IOSToolNames, + IOSWaitForSelectorStateSchema, +} from './types.js'; + +import type { IOSService } from './ios-service.js'; +import type { IOSToolName } from './types.js'; + +const logger = createLogger('IOSMcpServer'); +const MCP_SERVER_NAME = 'orbit-ios'; +const MCP_SERVER_VERSION = '1.0.0'; + +type ToolSchemaShape = Record; + +function addExecutionTarget(text: string, target: 'desktop' | 'ios'): string { + try { + const parsed: unknown = JSON.parse(text); + if (typeof parsed === 'object' && parsed !== null && !Array.isArray(parsed)) { + return JSON.stringify({ ...(parsed as Record), executionTarget: target }); + } + } catch { + // Not JSON — append as text line + } + return `${text}\nexecutionTarget: ${target}`; +} + +function safeStringify(value: unknown): string { + if (value === undefined) { + return 'null'; + } + + try { + return JSON.stringify(value); + } catch { + if (typeof value === 'bigint') { + return value.toString(); + } + if (typeof value === 'object' && value !== null) { + return '[Unserializable Object]'; + } + if (typeof value === 'string') { + return value; + } + if (typeof value === 'number' || typeof value === 'boolean') { + return JSON.stringify(value); + } + return 'null'; + } +} + +function createSuccessResponse(text: string): { content: [{ type: 'text'; text: string }] } { + return { + content: [{ type: 'text' as const, text }], + }; +} + +function createExactErrorResponse(error: unknown): { + content: [{ type: 'text'; text: string }]; + isError: true; +} { + return { + content: [ + { + type: 'text' as const, + text: error instanceof Error ? error.message : String(error), + }, + ], + isError: true, + }; +} + +function createIOSTool( + service: IOSService, + sessionId: string, + toolName: IOSToolName, + description: string, + shape: ToolSchemaShape +): ReturnType { + return tool(toolName, description, shape, async (args) => { + logger.debug({ toolName, sessionId }, 'Running iOS MCP tool'); + try { + const result = await executeIOSAction(service, sessionId, toolName, args); + return createSuccessResponse(addExecutionTarget(safeStringify(result), 'ios')); + } catch (error: unknown) { + return createExactErrorResponse(error); + } + }); +} + +/** + * Create the iOS MCP server for a specific session. + */ +export function createIOSMcpServer( + service: IOSService, + sessionId: string +): ReturnType { + const tools = [ + createIOSTool( + service, + sessionId, + 'ios_device_list', + 'List available iOS simulators via xcrun simctl.', + {} + ), + createIOSTool( + service, + sessionId, + 'ios_launch', + 'Boot the iOS simulator, start Appium, and acquire an iOS lease for this session.', + { + device: z.string().describe('Simulator device name or UDID, for example "iPhone 16 Pro"'), + } + ), + createIOSTool( + service, + sessionId, + 'ios_close', + "Release this session's iOS lease. If it is the last lease, the simulator and Appium shut down.", + {} + ), + createIOSTool(service, sessionId, 'ios_navigate', 'Navigate mobile Safari to a URL.', { + url: z.string().describe('URL to load in mobile Safari'), + }), + createIOSTool(service, sessionId, 'ios_back', 'Navigate back in mobile Safari history.', {}), + createIOSTool( + service, + sessionId, + 'ios_forward', + 'Navigate forward in mobile Safari history.', + {} + ), + createIOSTool(service, sessionId, 'ios_reload', 'Reload the current mobile Safari page.', {}), + createIOSTool( + service, + sessionId, + 'ios_snapshot', + 'Capture an accessibility snapshot from iOS Safari using the same ref system as browser_snapshot.', + { + interactive: z + .boolean() + .optional() + .describe('When false, include refs on all elements, not only interactive ones'), + cursor: z + .boolean() + .optional() + .describe('Include cursor:pointer or tabindex elements as ref targets'), + compact: z + .boolean() + .optional() + .describe('Omit non-ref-bearing nodes from the snapshot output'), + } + ), + createIOSTool( + service, + sessionId, + 'ios_get_text', + 'Get trimmed text content from an element targeted by snapshot ref or CSS selector.', + { + ...IOSTargetShape, + } + ), + createIOSTool( + service, + sessionId, + 'ios_get_html', + 'Get HTML content from an element targeted by snapshot ref or CSS selector.', + { + ...IOSTargetShape, + outer: z.boolean().optional().describe('Return outerHTML instead of innerHTML'), + } + ), + createIOSTool( + service, + sessionId, + 'ios_screenshot', + 'Capture a base64-encoded PNG screenshot of the iOS Safari viewport.', + {} + ), + createIOSTool( + service, + sessionId, + 'ios_tap', + 'Tap an element using a snapshot ref (preferred) or CSS selector.', + { + ...IOSTargetShape, + } + ), + createIOSTool( + service, + sessionId, + 'ios_fill', + 'Clear an input field and replace its value using a snapshot ref or CSS selector.', + { + ...IOSTargetShape, + value: z.string().describe('Replacement value'), + } + ), + createIOSTool( + service, + sessionId, + 'ios_type', + 'Append text to an input field using a snapshot ref or CSS selector.', + { + ...IOSTargetShape, + text: z.string().describe('Text to append'), + } + ), + createIOSTool( + service, + sessionId, + 'ios_select', + 'Select one or more option values on a dropdown using a snapshot ref or CSS selector.', + { + ...IOSTargetShape, + values: z.array(z.string()).min(1).describe('Option values to select'), + } + ), + createIOSTool( + service, + sessionId, + 'ios_check', + 'Check a checkbox or switch using a snapshot ref or CSS selector.', + { + ...IOSTargetShape, + } + ), + createIOSTool( + service, + sessionId, + 'ios_uncheck', + 'Uncheck a checkbox or switch using a snapshot ref or CSS selector.', + { + ...IOSTargetShape, + } + ), + createIOSTool( + service, + sessionId, + 'ios_swipe', + 'Perform a native iOS swipe gesture in the given direction. Optional target narrows the gesture origin.', + { + direction: IOSSwipeDirectionSchema.describe('Swipe direction: up, down, left, or right'), + target: z + .object(IOSTargetShape) + .optional() + .describe('Optional target element to anchor the swipe'), + duration: z + .number() + .int() + .positive() + .optional() + .describe('Gesture duration in milliseconds (default: 300)'), + } + ), + createIOSTool( + service, + sessionId, + 'ios_scroll', + 'Scroll the page in iOS Safari via JavaScript window.scrollBy.', + { + direction: IOSScrollDirectionSchema.describe('Scroll direction: up, down, left, or right'), + amount: z + .number() + .int() + .positive() + .optional() + .describe('Scroll amount in pixels (default: 300)'), + } + ), + createIOSTool( + service, + sessionId, + 'ios_eval', + 'Execute JavaScript in the iOS Safari page context and return the result.', + { + script: z.string().describe('JavaScript code to execute'), + } + ), + createIOSTool( + service, + sessionId, + 'ios_wait_for_selector', + 'Wait for a CSS selector to reach the requested state in iOS Safari.', + { + selector: z.string().describe('CSS selector to wait for'), + state: IOSWaitForSelectorStateSchema.optional().describe( + 'Desired selector state (default: visible)' + ), + timeout: z + .number() + .int() + .positive() + .max(120000) + .optional() + .describe('Timeout in milliseconds (default: 30000, max: 120000)'), + } + ), + createIOSTool( + service, + sessionId, + 'ios_console_logs', + 'Read captured console logs from the current iOS Safari session.', + { + level: IOSConsoleLogLevelSchema.optional().describe('Optional log level filter'), + } + ), + ]; + + return createSdkMcpServer({ + name: MCP_SERVER_NAME, + version: MCP_SERVER_VERSION, + tools, + }); +} + +export function getIOSToolNames(): string[] { + return IOSToolNames.map((toolName) => `mcp__${MCP_SERVER_NAME}__${toolName}`); +} diff --git a/agent-bridge/src/ios/ios-service.ts b/agent-bridge/src/ios/ios-service.ts new file mode 100644 index 000000000..b7935c522 --- /dev/null +++ b/agent-bridge/src/ios/ios-service.ts @@ -0,0 +1,292 @@ +/** + * Process-level iOS simulator lease service. + * + * [warning] TESTED: This service is covered by integration-style lease tests. + * If you modify this, run: cd agent-bridge && bun test + * Test files: + * - src/__tests__/ios-service-lease.test.ts + * - src/__tests__/ios-platform-gate.test.ts + */ + +import { execFile } from 'node:child_process'; +import { promisify } from 'node:util'; + +import { createLogger } from '../common/logging/logger.js'; + +import { IOSManager } from './ios-manager.js'; + +import type { IOSDevice, IOSManagerLike, IOSLifecycleState, IOSServiceOptions } from './types.js'; + +const logger = createLogger('IOSService'); +const execFileAsync = promisify(execFile); +const IOS_CHECK_TIMEOUT_MS = 5000; +const APPIUM_EXIT_MESSAGE_PREFIX = 'Appium process exited unexpectedly'; + +interface ExecFileResult { + stdout: string; + stderr: string; +} + +export interface CanEnableIOSOptions { + platform?: NodeJS.Platform; + env?: NodeJS.ProcessEnv; + execFile?: ( + file: string, + args: readonly string[], + options?: { timeout?: number } + ) => Promise; +} + +async function defaultExecFile( + file: string, + args: readonly string[], + options?: { timeout?: number } +): Promise { + const result = await execFileAsync(file, [...args], { + timeout: options?.timeout, + encoding: 'utf8', + }); + + return { + stdout: result.stdout, + stderr: result.stderr, + }; +} + +export async function canEnableIOS(options: CanEnableIOSOptions = {}): Promise { + const platform = options.platform ?? process.platform; + const env = options.env ?? process.env; + + if (platform !== 'darwin') { + return false; + } + + if (env.ORBIT_ENABLE_IOS_TOOLS !== '1') { + return false; + } + + const runExecFile = options.execFile ?? defaultExecFile; + + try { + await runExecFile('xcrun', ['simctl', 'help'], { timeout: IOS_CHECK_TIMEOUT_MS }); + await runExecFile('which', ['appium'], { timeout: IOS_CHECK_TIMEOUT_MS }); + return true; + } catch { + return false; + } +} + +export class IOSService { + private readonly managerFactory: () => IOSManagerLike; + private readonly leaseholders = new Set(); + + private manager: IOSManagerLike | null = null; + private launchPromise: Promise | null = null; + private shutdownPromise: Promise | null = null; + private _state: IOSLifecycleState = 'idle'; + private launchVersion = 0; + + constructor(options: IOSServiceOptions = {}) { + this.managerFactory = options.managerFactory ?? (() => new IOSManager()); + } + + get state(): IOSLifecycleState { + return this._state; + } + + get leaseCount(): number { + return this.leaseholders.size; + } + + hasLease(sessionId: string): boolean { + return this.leaseholders.has(sessionId); + } + + async listDevices(): Promise { + this.assertNotDisposed(); + + const activeManager = this.manager; + if (activeManager !== null) { + return activeManager.listDevices(); + } + + const manager = this.managerFactory(); + try { + return await manager.listDevices(); + } finally { + await manager.dispose().catch((error: unknown) => { + logger.warn({ error }, 'Failed to dispose temporary iOS manager after device list'); + }); + } + } + + async acquire(sessionId: string, deviceQuery?: string): Promise { + this.assertNotDisposed(); + + if (this._state === 'closing' && this.shutdownPromise !== null) { + await this.shutdownPromise; + } + + this.assertNotDisposed(); + this.leaseholders.add(sessionId); + + if (this._state === 'ready') { + return this.getManager(sessionId); + } + + if (this._state === 'launching' && this.launchPromise !== null) { + return this.launchPromise; + } + + const manager = this.managerFactory(); + this.manager = manager; + this._state = 'launching'; + + const launchVersion = ++this.launchVersion; + const currentLaunch = (async (): Promise => { + try { + await manager.launch(deviceQuery); + if (this._state === 'disposed') { + throw new Error('IOSService disposed'); + } + this._state = 'ready'; + return await this.getManager(sessionId); + } catch (error) { + await this.handleLaunchFailure(manager, error); + throw error; + } finally { + if (this.launchVersion === launchVersion) { + this.launchPromise = null; + } + } + })(); + + this.launchPromise = currentLaunch; + return currentLaunch; + } + + async getManager(sessionId: string): Promise { + if (!this.leaseholders.has(sessionId)) { + throw new Error('No iOS lease for session'); + } + + if (this._state === 'launching' && this.launchPromise !== null) { + return this.launchPromise; + } + + const manager = this.manager; + if (manager === null) { + throw new Error('iOS not ready'); + } + + try { + manager.assertUsable(); + return manager; + } catch (error) { + if (error instanceof Error && error.message.startsWith(APPIUM_EXIT_MESSAGE_PREFIX)) { + await this.handleAppiumCrash(error); + } + throw error; + } + } + + async release(sessionId: string): Promise { + if (!this.leaseholders.delete(sessionId)) { + return; + } + + if (this.leaseholders.size === 0 && (this._state === 'ready' || this._state === 'launching')) { + await this.shutdown(); + } + } + + async forceRelease(sessionId: string): Promise { + if (!this.leaseholders.has(sessionId)) { + return; + } + + await this.release(sessionId); + } + + async shutdown(): Promise { + if (this._state === 'disposed') { + return; + } + + if (this._state === 'idle') { + this.leaseholders.clear(); + this.manager = null; + this.launchPromise = null; + return; + } + + if (this.shutdownPromise !== null) { + return this.shutdownPromise; + } + + this._state = 'closing'; + const manager = this.manager; + + const currentShutdown = (async (): Promise => { + try { + await manager?.close(); + } catch (error) { + logger.warn({ error }, 'Failed to close iOS manager during shutdown'); + } finally { + if (this.manager === manager) { + this.manager = null; + } + this.launchPromise = null; + this.leaseholders.clear(); + if (this._state !== 'disposed') { + this._state = 'idle'; + } + } + })().finally(() => { + if (this.shutdownPromise === currentShutdown) { + this.shutdownPromise = null; + } + }); + + this.shutdownPromise = currentShutdown; + return currentShutdown; + } + + async dispose(): Promise { + if (this._state === 'disposed') { + return; + } + + await this.shutdown(); + this._state = 'disposed'; + } + + private assertNotDisposed(): void { + if (this._state === 'disposed') { + throw new Error('IOSService disposed'); + } + } + + private async handleLaunchFailure(manager: IOSManagerLike, error: unknown): Promise { + logger.warn({ error }, 'iOS launch failed'); + + try { + await manager.close(); + } catch (closeError) { + logger.warn({ error: closeError }, 'Failed to clean up failed iOS launch'); + } + + if (this.manager === manager) { + this.manager = null; + } + this.leaseholders.clear(); + if (this._state !== 'disposed') { + this._state = 'idle'; + } + } + + private async handleAppiumCrash(error: Error): Promise { + logger.warn({ error: error.message }, 'Appium exited unexpectedly, resetting iOS service'); + await this.shutdown(); + } +} diff --git a/agent-bridge/src/ios/types.ts b/agent-bridge/src/ios/types.ts new file mode 100644 index 000000000..9df3e4ffa --- /dev/null +++ b/agent-bridge/src/ios/types.ts @@ -0,0 +1,255 @@ +/** + * iOS integration types shared by the service, manager, and MCP layer. + */ + +import { z } from 'zod'; + +import { + BrowserConsoleLogLevels, + BrowserScrollDirections, + BrowserTargetShape, + BrowserWaitForSelectorStates, + OptionalTargetSchema, + TargetSchema, +} from '../browser/types.js'; + +import type { + BrowserConsoleLogEntry, + BrowserConsoleLogLevel, + BrowserOptionalTargetInput, + BrowserScrollDirection, + BrowserTargetInput, + BrowserWaitForSelectorState, + SnapshotResponse, +} from '../browser/types.js'; + +export type IOSLifecycleState = 'idle' | 'launching' | 'ready' | 'closing' | 'disposed'; + +export type IOSDeviceState = 'Shutdown' | 'Booted' | 'Creating'; + +export const IOSSwipeDirections = BrowserScrollDirections; +export const IOSScrollDirections = BrowserScrollDirections; +export const IOSConsoleLogLevels = BrowserConsoleLogLevels; +export const IOSWaitForSelectorStates = BrowserWaitForSelectorStates; +export const IOSTargetShape = BrowserTargetShape; +export const IOSTargetSchema = TargetSchema; +export const IOSOptionalTargetSchema = OptionalTargetSchema; + +export const IOSToolNames = [ + 'ios_device_list', + 'ios_launch', + 'ios_close', + 'ios_navigate', + 'ios_back', + 'ios_forward', + 'ios_reload', + 'ios_snapshot', + 'ios_get_text', + 'ios_get_html', + 'ios_screenshot', + 'ios_tap', + 'ios_fill', + 'ios_type', + 'ios_select', + 'ios_check', + 'ios_uncheck', + 'ios_swipe', + 'ios_scroll', + 'ios_eval', + 'ios_wait_for_selector', + 'ios_console_logs', +] as const; + +export type IOSToolName = (typeof IOSToolNames)[number]; +export type IOSTargetInput = BrowserTargetInput; +export type IOSOptionalTargetInput = BrowserOptionalTargetInput; +export type IOSScrollDirection = BrowserScrollDirection; +export type IOSSwipeDirection = (typeof IOSSwipeDirections)[number]; +export type IOSConsoleLogLevel = BrowserConsoleLogLevel; +export type IOSWaitForSelectorState = BrowserWaitForSelectorState; + +export interface IOSDevice { + name: string; + udid: string; + state: IOSDeviceState; + runtime: string; + isAvailable: boolean; +} + +export interface IOSLaunchResult { + udid: string; + name: string; + runtime: string; + appiumPort: number; +} + +export interface IOSSnapshotOptions { + interactive?: boolean; + cursor?: boolean; + compact?: boolean; +} + +export interface IOSSuccessResult { + success: true; +} + +export interface IOSTextResult { + text: string; +} + +export interface IOSHtmlResult { + html: string; +} + +export interface IOSScreenshotResult { + image: string; + format: 'png'; +} + +export interface IOSTapResult { + tapped: true; +} + +export interface IOSFillResult { + filled: true; +} + +export interface IOSTypeResult { + typed: true; +} + +export interface IOSSelectResult { + selected: true; +} + +export interface IOSCheckResult { + checked: true; +} + +export interface IOSUncheckResult { + unchecked: true; +} + +export interface IOSSwipeResult { + swiped: true; +} + +export interface IOSScrollResult { + scrollX: number; + scrollY: number; +} + +export interface IOSWaitForSelectorResult { + matched: true; +} + +export interface IOSConsoleLogsResult { + logs: BrowserConsoleLogEntry[]; +} + +export interface IOSEvalResult { + result: unknown; +} + +export interface IOSDeviceListResult { + devices: IOSDevice[]; +} + +export interface IOSAppiumProcess { + readonly port: number; + readonly pid?: number; + onExit(callback: (code: number | null) => void): () => void; + terminate(signal: 'SIGTERM' | 'SIGKILL'): Promise; + waitForExit(timeoutMs: number): Promise; +} + +export interface IOSAutomationSession { + deleteSession(): Promise; + navigate(url: string): Promise; + back(): Promise; + forward(): Promise; + reload(): Promise; + snapshot(options: IOSSnapshotOptions): Promise; + getText(target: IOSTargetInput): Promise; + getHtml(target: IOSTargetInput, outer?: boolean): Promise; + screenshot(): Promise; + tap(target: IOSTargetInput): Promise; + fill(target: IOSTargetInput, value: string): Promise; + type(target: IOSTargetInput, text: string): Promise; + select(target: IOSTargetInput, values: string[]): Promise; + check(target: IOSTargetInput): Promise; + uncheck(target: IOSTargetInput): Promise; + swipe(direction: IOSSwipeDirection, target?: IOSTargetInput, duration?: number): Promise; + scroll(direction: IOSScrollDirection, amount?: number): Promise; + evaluate(script: string): Promise; + waitForSelector( + selector: string, + state?: IOSWaitForSelectorState, + timeout?: number + ): Promise; + consoleLogs(level?: IOSConsoleLogLevel): Promise; +} + +export interface IOSManagerDependencies { + listDevices(signal: AbortSignal): Promise; + bootDevice(device: IOSDevice, signal: AbortSignal): Promise; + shutdownDevice(device: IOSDevice): Promise; + startAppium(signal: AbortSignal): Promise; + createAutomationSession( + device: IOSDevice, + appiumPort: number, + signal: AbortSignal + ): Promise; +} + +export interface IOSManagerOptions { + dependencies?: IOSManagerDependencies; +} + +export interface IOSManagerLike { + readonly state: IOSLifecycleState; + listDevices(): Promise; + launch(deviceQuery?: string): Promise; + navigate(url: string): Promise; + back(): Promise; + forward(): Promise; + reload(): Promise; + snapshot(options: IOSSnapshotOptions): Promise; + getText(target: IOSTargetInput): Promise; + getHtml(target: IOSTargetInput, outer?: boolean): Promise; + screenshot(): Promise; + tap(target: IOSTargetInput): Promise; + fill(target: IOSTargetInput, value: string): Promise; + type(target: IOSTargetInput, text: string): Promise; + select(target: IOSTargetInput, values: string[]): Promise; + check(target: IOSTargetInput): Promise; + uncheck(target: IOSTargetInput): Promise; + swipe( + direction: IOSSwipeDirection, + target?: IOSTargetInput, + duration?: number + ): Promise; + scroll(direction: IOSScrollDirection, amount?: number): Promise; + evaluate(script: string): Promise; + waitForSelector( + selector: string, + state?: IOSWaitForSelectorState, + timeout?: number + ): Promise; + consoleLogs(level?: IOSConsoleLogLevel): Promise; + close(): Promise; + dispose(): Promise; + assertUsable(): void; + getLaunchResult(): IOSLaunchResult | null; +} + +export type IOSManagerFactory = () => IOSManagerLike; + +export interface IOSServiceOptions { + managerFactory?: IOSManagerFactory; +} + +export const IOSSwipeDirectionSchema = z.enum(IOSSwipeDirections); +export const IOSScrollDirectionSchema = z.enum(IOSScrollDirections); +export const IOSConsoleLogLevelSchema = z.enum(IOSConsoleLogLevels); +export const IOSWaitForSelectorStateSchema = z.enum(IOSWaitForSelectorStates); diff --git a/agent-bridge/src/protocol/protocol.ts b/agent-bridge/src/protocol/protocol.ts index 0b9fefaac..c3f5c98b3 100644 --- a/agent-bridge/src/protocol/protocol.ts +++ b/agent-bridge/src/protocol/protocol.ts @@ -364,7 +364,6 @@ export interface EnhanceBugReportRequest { export interface GenerateTitleRequest { type: 'generate_title'; userMessage: string; - assistantResponse: string; } /** diff --git a/agent-bridge/src/protocol/schemas.ts b/agent-bridge/src/protocol/schemas.ts index f789320d7..3075c9342 100644 --- a/agent-bridge/src/protocol/schemas.ts +++ b/agent-bridge/src/protocol/schemas.ts @@ -495,7 +495,6 @@ export const GenerateTitleRequestSchema = z .object({ type: z.literal('generate_title'), userMessage: z.string(), - assistantResponse: z.string(), }) .strict(); export type GenerateTitleRequest = z.infer; diff --git a/apps/Canvas-UI-Builder/src/components/inspector/PropertiesPanel.tsx b/apps/Canvas-UI-Builder/src/components/inspector/PropertiesPanel.tsx index 784b33596..cf9ad857f 100644 --- a/apps/Canvas-UI-Builder/src/components/inspector/PropertiesPanel.tsx +++ b/apps/Canvas-UI-Builder/src/components/inspector/PropertiesPanel.tsx @@ -537,7 +537,7 @@ export const PropertiesPanel: FC = ({ {/* Feedback */} {copyFeedback ? ( -
+
{copyFeedback}
) : null} diff --git a/apps/Canvas-UI-Builder/src/components/layout/left-sidebar/CanvasLeftSidebar.tsx b/apps/Canvas-UI-Builder/src/components/layout/left-sidebar/CanvasLeftSidebar.tsx index 8969fbe5e..263d76e98 100644 --- a/apps/Canvas-UI-Builder/src/components/layout/left-sidebar/CanvasLeftSidebar.tsx +++ b/apps/Canvas-UI-Builder/src/components/layout/left-sidebar/CanvasLeftSidebar.tsx @@ -191,7 +191,7 @@ export const CanvasLeftSidebar: FC = ({ width, onCompone }} > + + +
+ + + +
+

Interactive Targets

+
+ Example link + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+
Editable content
+
+
+ +
+

Event Log

+ 

+      

+    
+  
+
diff --git a/src-tauri/src/commands/browser/test-fixtures/csp-test.js b/src-tauri/src/commands/browser/test-fixtures/csp-test.js
new file mode 100644
index 000000000..2743a7c0f
--- /dev/null
+++ b/src-tauri/src/commands/browser/test-fixtures/csp-test.js
@@ -0,0 +1,79 @@
+const currentUrl = document.getElementById('current-url');
+const currentTitle = document.getElementById('current-title');
+const navStatus = document.getElementById('nav-status');
+const eventLog = document.getElementById('event-log');
+
+function updateMeta(label) {
+  if (currentUrl instanceof HTMLElement) {
+    currentUrl.textContent = window.location.href;
+  }
+  if (currentTitle instanceof HTMLElement) {
+    currentTitle.textContent = document.title;
+  }
+  if (navStatus instanceof HTMLElement) {
+    navStatus.textContent = label;
+  }
+}
+
+function appendLog(message) {
+  if (!(eventLog instanceof HTMLElement)) {
+    return;
+  }
+  const lines = eventLog.textContent ? `${eventLog.textContent}\n${message}` : message;
+  eventLog.textContent = lines;
+}
+
+function bindClick(id, handler) {
+  const element = document.getElementById(id);
+  if (!(element instanceof HTMLButtonElement)) {
+    return;
+  }
+  element.addEventListener('click', handler);
+}
+
+document.getElementById('fixture-button')?.addEventListener('click', () => {
+  appendLog('fixture-button clicked');
+});
+
+document.getElementById('fixture-checkbox')?.addEventListener('change', (event) => {
+  const target = event.currentTarget;
+  if (target instanceof HTMLInputElement) {
+    appendLog(`fixture-checkbox changed: ${String(target.checked)}`);
+  }
+});
+
+document.getElementById('fixture-select')?.addEventListener('change', (event) => {
+  const target = event.currentTarget;
+  if (target instanceof HTMLSelectElement) {
+    appendLog(`fixture-select changed: ${target.value}`);
+  }
+});
+
+bindClick('push-state', () => {
+  history.pushState({ via: 'pushState' }, '', '/pushed?mode=push#push-state');
+  document.title = 'Orbit CSP Fixture (PushState)';
+  updateMeta('pushState applied');
+  appendLog(`pushState -> ${window.location.href}`);
+});
+
+bindClick('replace-state', () => {
+  history.replaceState({ via: 'replaceState' }, '', '/replaced?mode=replace#replace-state');
+  document.title = 'Orbit CSP Fixture (ReplaceState)';
+  updateMeta('replaceState applied');
+  appendLog(`replaceState -> ${window.location.href}`);
+});
+
+bindClick('hash-nav', () => {
+  window.location.hash = 'hash-only';
+  document.title = 'Orbit CSP Fixture (Hash)';
+  updateMeta('hash navigation applied');
+  appendLog(`hash navigation -> ${window.location.href}`);
+});
+
+window.addEventListener('popstate', () => {
+  updateMeta('popstate observed');
+  appendLog(`popstate -> ${window.location.href}`);
+});
+
+updateMeta('fixture loaded');
+appendLog(`loaded -> ${window.location.href}`);
diff --git a/src-tauri/src/commands/canvas/preview.rs b/src-tauri/src/commands/canvas/preview.rs
index 90c21e5f7..14cb36c6e 100644
--- a/src-tauri/src/commands/canvas/preview.rs
+++ b/src-tauri/src/commands/canvas/preview.rs
@@ -60,7 +60,6 @@
 use std::fmt;
 use std::io::{BufRead as _, BufReader};
 use std::net::TcpListener;
-use std::path::PathBuf;
 use std::process::{Child, Command, Stdio};
 use std::thread;
 
@@ -68,6 +67,8 @@ use parking_lot::Mutex;
 use serde::Serialize;
 use tauri::State;
 
+use crate::commands::common::path_utils::augmented_bun_path;
+
 /// Port range for the preview server (tries 5199, then 5200-5209)
 const PORT_RANGE_START: u16 = 5199;
 const PORT_RANGE_END: u16 = 5209;
@@ -157,39 +158,6 @@ fn find_available_port() -> Option {
     (PORT_RANGE_START..=PORT_RANGE_END).find(|&port| TcpListener::bind(("127.0.0.1", port)).is_ok())
 }
 
-/// Build an augmented PATH that includes common bun install locations.
-///
-/// When the app is launched from Finder (not terminal), macOS provides a minimal
-/// PATH (typically just `/usr/bin:/bin:/usr/sbin:/sbin`). This function appends
-/// common locations where `bun` is installed so that `Command::new("bun")` works
-/// in production builds.
-#[expect(
-    clippy::disallowed_methods,
-    reason = "std::env::var(PATH) is needed to read the system PATH for augmentation - not a config value"
-)]
-fn augmented_path() -> String {
-    let current_path = std::env::var("PATH").unwrap_or_default();
-
-    let home = dirs::home_dir().unwrap_or_else(|| PathBuf::from("/tmp"));
-    let extra_paths = [
-        home.join(".bun/bin"),
-        home.join(".local/bin"),
-        home.join(".nvm/current/bin"),
-        PathBuf::from("/usr/local/bin"),
-        PathBuf::from("/opt/homebrew/bin"),
-    ];
-
-    let mut parts: Vec = vec![current_path];
-    for p in &extra_paths {
-        let s = p.to_string_lossy().to_string();
-        if !parts.contains(&s) {
-            parts.push(s);
-        }
-    }
-
-    parts.join(":")
-}
-
 /// Attempt graceful shutdown with SIGTERM, fallback to SIGKILL after timeout.
 ///
 /// Uses blocking sleep because this function:
@@ -1137,7 +1105,7 @@ pub async fn canvas_install_preview_deps() -> Result<(), String> {
 
     let output = Command::new("bun")
         .arg("install")
-        .env("PATH", augmented_path())
+        .env("PATH", augmented_bun_path())
         .current_dir(&preview_path)
         .stdout(Stdio::piped())
         .stderr(Stdio::piped())
@@ -1225,7 +1193,7 @@ pub async fn canvas_start_preview_server(
     // Spawn process (outside lock scope)
     let mut child = Command::new("bun")
         .args(["run", "dev"])
-        .env("PATH", augmented_path())
+        .env("PATH", augmented_bun_path())
         .env("VITE_PORT", port.to_string())
         .current_dir(&preview_path)
         .stdout(Stdio::piped())
diff --git a/src-tauri/src/commands/common/icons.rs b/src-tauri/src/commands/common/icons.rs
new file mode 100644
index 000000000..d5c0dede0
--- /dev/null
+++ b/src-tauri/src/commands/common/icons.rs
@@ -0,0 +1,226 @@
+//! App icon commands.
+//!
+//! This is the safe Tauri-facing wrapper around `orbit-app-icons`.
+
+#![allow(
+    clippy::needless_pass_by_value,
+    reason = "Tauri commands receive owned types from JSON deserialization"
+)]
+
+use std::fs;
+use std::path::PathBuf;
+use std::sync::mpsc::sync_channel;
+
+use orbit_app_icons::{
+    icon_path_for_appearance, resolve_active_id, scan_icons_dir, AppIconInfo, AppIconMeta,
+};
+use orbit_core::{Error, Result};
+use orbit_settings::SettingsManager;
+use tauri::{AppHandle, Manager as _, State};
+
+use crate::core::sentry_utils::SentryCapture as _;
+
+/// Resolve the bundled app-icons directory.
+fn bundled_icons_dir(app: &AppHandle) -> Option {
+    let resource_dir = app.path().resource_dir().ok()?;
+    let bundled_dir = resource_dir.join("app-icons");
+    if bundled_dir.is_dir() {
+        return Some(bundled_dir);
+    }
+
+    let dev_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("icons/app-icons");
+    dev_dir.is_dir().then_some(dev_dir)
+}
+
+/// List all bundled app icons.
+#[tauri::command]
+pub fn list_app_icons(
+    app: AppHandle,
+    settings: State<'_, SettingsManager>,
+) -> Result> {
+    #[cfg(not(target_os = "macos"))]
+    {
+        let _ = (app, settings);
+        return Err(Error::Other(
+            "App icon switching is only supported on macOS".to_owned(),
+        ))
+        .capture("list_app_icons");
+    }
+
+    #[cfg(target_os = "macos")]
+    {
+        (|| {
+            let icons_dir = bundled_icons_dir(&app).ok_or_else(|| {
+                Error::Other("Bundled app-icons directory was not found".to_owned())
+            })?;
+
+            let discovered = scan_icons_dir(&icons_dir);
+            let metas: Vec = discovered.iter().map(|(_, meta)| meta.clone()).collect();
+
+            let stored_active_id = settings.get().active_icon_id;
+            let active_id = resolve_active_id(&stored_active_id, &metas);
+
+            let mut icons: Vec = discovered
+                .into_iter()
+                .filter_map(|(path, meta)| {
+                    // Use the full 1024×1024 Dock icon for the picker —
+                    // the WebView downscales natively for crisp Retina rendering.
+                    let preview = icon_path_for_appearance(&path);
+                    if !preview.is_file() {
+                        return None;
+                    }
+                    let is_active = match active_id.as_deref() {
+                        Some(id) => meta.id == id,
+                        None => meta.default,
+                    };
+
+                    Some(AppIconInfo {
+                        id: meta.id,
+                        name: meta.name,
+                        theme: meta.theme,
+                        is_default: meta.default,
+                        is_active,
+                        preview_path: preview.to_string_lossy().into_owned(),
+                    })
+                })
+                .collect();
+
+            icons.sort_by(|left, right| {
+                left.theme
+                    .cmp(&right.theme)
+                    .then(left.name.cmp(&right.name))
+            });
+
+            if stored_active_id.is_some() && active_id.is_none() {
+                let mut current = settings.get();
+                current.active_icon_id = None;
+                if let Err(error) = settings
+                    .update(¤t)
+                    .capture("list_app_icons_clear_invalid")
+                {
+                    log::warn!("Failed to clear stale active app icon id: {error}");
+                }
+            }
+
+            Ok(icons)
+        })()
+        .capture("list_app_icons")
+    }
+}
+
+/// Switch the active bundled app icon.
+#[tauri::command]
+pub fn set_app_icon(
+    id: String,
+    app: AppHandle,
+    settings: State<'_, SettingsManager>,
+) -> Result<()> {
+    #[cfg(not(target_os = "macos"))]
+    {
+        let _ = (id, app, settings);
+        return Err(Error::Other(
+            "App icon switching is only supported on macOS".to_owned(),
+        ))
+        .capture("set_app_icon");
+    }
+
+    #[cfg(target_os = "macos")]
+    {
+        (|| {
+            let icons_dir = bundled_icons_dir(&app).ok_or_else(|| {
+                Error::Other("Bundled app-icons directory was not found".to_owned())
+            })?;
+            let icon_dir = icons_dir.join(&id);
+            let icon_path = icon_path_for_appearance(&icon_dir);
+            if !icon_path.is_file() {
+                return Err(Error::FileNotFound(icon_path.display().to_string()));
+            }
+
+            // Validate that the icon directory has a parseable meta.json.
+            let meta_path = icon_dir.join("meta.json");
+            let meta_content = fs::read_to_string(&meta_path)?;
+            drop(serde_json::from_str::(&meta_content)?);
+
+            // Use set_dock_icon for ALL icons (including default) so they
+            // all go through the same rendering pipeline with consistent sizing.
+            let (tx, rx) = sync_channel(1);
+            let dispatch_result = app.run_on_main_thread(move || {
+                let result = orbit_app_icons::set_dock_icon(&icon_path);
+                if tx.send(result).is_err() {
+                    log::debug!("set_app_icon receiver dropped before icon result");
+                }
+            });
+
+            dispatch_result.map_err(|error| Error::Other(error.to_string()))?;
+
+            match rx.recv() {
+                Ok(true) => {},
+                Ok(false) => {
+                    return Err(Error::Other(
+                        "Failed to switch Dock icon via NSWorkspace".to_owned(),
+                    ));
+                },
+                Err(error) => {
+                    return Err(Error::Other(format!(
+                        "Failed to receive Dock icon switch result: {error}"
+                    )));
+                },
+            }
+
+            let mut current = settings.get();
+            current.active_icon_id = Some(id);
+            settings.update(¤t)?;
+
+            Ok(())
+        })()
+        .capture("set_app_icon")
+    }
+}
+
+/// Re-apply the persisted icon selection at startup.
+#[cfg(target_os = "macos")]
+pub fn reapply_persisted_icon(app: &AppHandle) {
+    let settings = app.state::();
+    let Some(icon_id) = settings.get().active_icon_id else {
+        return;
+    };
+
+    let Some(icons_dir) = bundled_icons_dir(app) else {
+        return;
+    };
+
+    let icon_path = icon_path_for_appearance(&icons_dir.join(&icon_id));
+    if !icon_path.is_file() {
+        log::warn!("Persisted app icon '{icon_id}' no longer exists; clearing it");
+        let mut current = settings.get();
+        current.active_icon_id = None;
+        drop(settings.update(¤t));
+        return;
+    }
+
+    let success = if orbit_app_icons::is_main_thread() {
+        orbit_app_icons::set_dock_icon(&icon_path)
+    } else {
+        let (tx, rx) = sync_channel(1);
+        let dispatch_result = app.run_on_main_thread(move || {
+            let result = orbit_app_icons::set_dock_icon(&icon_path);
+            if tx.send(result).is_err() {
+                log::debug!("reapply_persisted_icon receiver dropped before icon result");
+            }
+        });
+
+        if let Err(error) = dispatch_result {
+            log::warn!("Failed to dispatch persisted icon reapply to main thread: {error}");
+            false
+        } else {
+            rx.recv().is_ok_and(|result| result)
+        }
+    };
+
+    if !success {
+        log::warn!("Failed to re-apply persisted app icon '{icon_id}'; clearing it");
+        let mut current = settings.get();
+        current.active_icon_id = None;
+        drop(settings.update(¤t));
+    }
+}
diff --git a/src-tauri/src/commands/common/mod.rs b/src-tauri/src/commands/common/mod.rs
index f2c5bc5d9..76c5145b8 100644
--- a/src-tauri/src/commands/common/mod.rs
+++ b/src-tauri/src/commands/common/mod.rs
@@ -5,7 +5,9 @@ pub mod dev_monitor;
 pub mod diagnostics;
 pub mod files;
 pub mod git;
+pub mod icons;
 pub mod lsp;
+pub mod path_utils;
 pub mod providers;
 pub mod search;
 pub mod settings;
diff --git a/src-tauri/src/commands/common/path_utils.rs b/src-tauri/src/commands/common/path_utils.rs
new file mode 100644
index 000000000..7f0f6c4fa
--- /dev/null
+++ b/src-tauri/src/commands/common/path_utils.rs
@@ -0,0 +1,92 @@
+//! Shared PATH helpers for Bun-based subprocess execution.
+
+use std::env;
+use std::ffi::OsString;
+use std::path::PathBuf;
+
+/// Common Bun installation locations appended to PATH.
+fn bun_candidate_paths() -> Vec {
+    let home = dirs::home_dir().unwrap_or_else(|| PathBuf::from("/tmp"));
+    vec![
+        home.join(".bun/bin"),
+        home.join(".local/bin"),
+        home.join(".nvm/current/bin"),
+        PathBuf::from("/usr/local/bin"),
+        PathBuf::from("/opt/homebrew/bin"),
+    ]
+}
+
+/// Merge PATH entries with extra paths while preserving existing order.
+fn merge_paths(base: Option, extras: &[PathBuf]) -> OsString {
+    let mut entries: Vec = base
+        .as_ref()
+        .map(|value| env::split_paths(value).collect())
+        .unwrap_or_default();
+
+    for extra in extras {
+        if !entries.iter().any(|existing| existing == extra) {
+            entries.push(extra.clone());
+        }
+    }
+
+    env::join_paths(&entries).unwrap_or_else(|_| base.unwrap_or_default())
+}
+
+/// Build an augmented PATH that includes common Bun install locations.
+///
+/// Desktop apps launched outside a terminal can inherit a minimal PATH.
+/// This helper appends common Bun install directories so `bun`/`bunx`
+/// subprocesses resolve reliably in production builds.
+#[must_use]
+pub(crate) fn augmented_bun_path() -> OsString {
+    let base = env::var_os("PATH");
+    merge_paths(base, &bun_candidate_paths())
+}
+
+#[cfg(test)]
+#[expect(
+    clippy::expect_used,
+    reason = "expect is appropriate in tests to assert expected parsing behavior"
+)]
+mod tests {
+    use std::path::Path;
+
+    use super::*;
+
+    #[test]
+    fn merge_paths_appends_missing_entries() {
+        let base =
+            env::join_paths([PathBuf::from("/usr/bin")]).expect("base path should serialize");
+        let base = Some(base);
+        let merged = merge_paths(base, &[PathBuf::from("/opt/homebrew/bin")]);
+        let parts: Vec = env::split_paths(&merged).collect();
+
+        assert!(parts.contains(&PathBuf::from("/usr/bin")));
+        assert!(parts.contains(&PathBuf::from("/opt/homebrew/bin")));
+    }
+
+    #[test]
+    fn merge_paths_deduplicates_existing_entries() {
+        let base = env::join_paths([
+            PathBuf::from("/usr/bin"),
+            PathBuf::from("/opt/homebrew/bin"),
+        ])
+        .expect("base path should serialize");
+        let base = Some(base);
+        let merged = merge_paths(base, &[PathBuf::from("/opt/homebrew/bin")]);
+        let parts: Vec = env::split_paths(&merged).collect();
+        let homebrew = Path::new("/opt/homebrew/bin");
+        let count = parts.iter().filter(|path| **path == homebrew).count();
+
+        assert_eq!(count, 1);
+    }
+
+    #[test]
+    fn augmented_bun_path_is_non_empty_when_path_exists() {
+        let augmented = augmented_bun_path();
+        assert!(
+            env::split_paths(&augmented).next().is_some(),
+            "Augmented path should contain at least one entry"
+        );
+    }
+}
diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rs
index 025a40e66..2c9067439 100644
--- a/src-tauri/src/lib.rs
+++ b/src-tauri/src/lib.rs
@@ -17,7 +17,7 @@ use parking_lot::RwLock;
 
 use agent::SessionManager;
 use commands::agent::lifecycle as agent_cmd;
-use commands::agent::{ai, conversations};
+use commands::agent::{ai, conversations, marketplace};
 use commands::browser::{self, BrowserResultState, BrowserWindowState};
 use commands::canvas::download as canvas_download;
 use commands::canvas::lifecycle as canvas_cmd;
@@ -28,7 +28,7 @@ use commands::canvas::setup as canvas_setup;
 use commands::canvas::transform as canvas_transform;
 use commands::canvas::PreviewServerState;
 use commands::common::{
-    credentials, dev_monitor, diagnostics, files, git, lsp, providers,
+    credentials, dev_monitor, diagnostics, files, git, icons, lsp, providers,
     search::{self, FileIndexState},
     settings, sf_symbols, terminal, window, workspace,
 };
@@ -264,6 +264,7 @@ pub fn run() {
     // Initialize browser window state
     let browser_state = Arc::new(BrowserWindowState::new());
     let browser_result_state = Arc::new(BrowserResultState::new());
+    let browser_result_state_for_events = Arc::clone(&browser_result_state);
 
     // Clone browser state for the main-window focus listener (Arc is moved into .manage())
     #[cfg(target_os = "macos")]
@@ -280,6 +281,7 @@ pub fn run() {
         .manage(browser_state)
         .manage(browser_result_state)
         .manage(PreviewServerState::new())
+        .manage(marketplace::MarketplaceCache::new())
         .manage(file_index_state)
         // Plugins
         .plugin(build_log_plugin().build())
@@ -295,6 +297,12 @@ pub fn run() {
         // Setup event callbacks for agent and configure window
         .setup(move |app| {
             agent_cmd::setup_event_callbacks(app.handle(), &session_manager);
+            #[cfg(target_os = "macos")]
+            icons::reapply_persisted_icon(app.handle());
+            browser::register_browser_large_eval_result_listener(
+                app.handle(),
+                Arc::clone(&browser_result_state_for_events),
+            );
 
             #[cfg(target_os = "macos")]
             {
@@ -337,6 +345,15 @@ pub fn run() {
                         // (checks NSWindow.isVisible before calling orderFront:).
                         orbit_plugin_decorum::order_child_windows_front();
                     });
+
+                    // Re-apply the Dock icon when the system theme changes so the
+                    // correct light/dark rendition is shown automatically.
+                    let theme_app = app.handle().clone();
+                    window.on_window_event(move |event| {
+                        if matches!(event, tauri::WindowEvent::ThemeChanged(_)) {
+                            icons::reapply_persisted_icon(&theme_app);
+                        }
+                    });
                 }
             }
 
@@ -372,6 +389,10 @@ pub fn run() {
             agent_cmd::agent_delete_agent,
             // Skill definition commands
             agent_cmd::agent_list_skills,
+            marketplace::skills_marketplace_search,
+            marketplace::skills_marketplace_browse,
+            marketplace::skills_marketplace_install,
+            marketplace::skills_marketplace_installed,
             // Command definition commands
             agent_cmd::agent_list_commands,
             agent_cmd::agent_get_command,
@@ -549,6 +570,8 @@ pub fn run() {
             settings::remove_ssh_host,
             settings::clear_ssh_hosts,
             settings::pick_directory,
+            icons::list_app_icons,
+            icons::set_app_icon,
             // Diagnostics commands
             diagnostics::check_previous_crash,
             diagnostics::clear_crash_log,
@@ -584,8 +607,15 @@ pub fn run() {
             browser::browser_has,
             browser::browser_info,
             browser::browser_eval,
+            browser::browser_invoke_runtime,
+            browser::browser_runtime_version,
+            browser::browser_ensure_runtime,
+            browser::browser_get_url,
+            browser::browser_get_title,
             browser::browser_js_callback,
             browser::browser_eval_async,
+            browser::browser_wait_for_selector,
+            browser::browser_wait_for_url,
             browser::browser_screenshot,
             browser::browser_open_devtools,
             browser::app_open_devtools,
diff --git a/src-tauri/tauri.conf.json b/src-tauri/tauri.conf.json
index 10b790a24..e634f05f5 100644
--- a/src-tauri/tauri.conf.json
+++ b/src-tauri/tauri.conf.json
@@ -38,14 +38,31 @@
         "script-src": "'self' 'unsafe-eval' https://streamdown.ai http://localhost:5176 http://localhost:5199",
         "style-src": "'self' 'unsafe-inline' https://streamdown.ai http://localhost:*",
         "connect-src": "'self' https://api.anthropic.com ipc://localhost https://streamdown.ai http://localhost:* ws://localhost:* https://github.com https://*.githubusercontent.com",
-        "img-src": "'self' data: blob:",
+        "img-src": "'self' data: blob: asset: http://asset.localhost",
         "font-src": "'self' data: https://streamdown.ai",
         "worker-src": "'self' blob:",
         "frame-src": "'self' http://localhost:*",
         "object-src": "'none'"
       },
       "freezePrototype": false,
-      "dangerousDisableAssetCspModification": false
+      "dangerousDisableAssetCspModification": false,
+      "assetProtocol": {
+        "enable": true,
+        "scope": {
+          "allow": [
+            "$HOME/**",
+            "$TEMP/**",
+            "$DESKTOP/**",
+            "$DOCUMENT/**",
+            "$DOWNLOAD/**",
+            "/Volumes/**",
+            "/tmp/**",
+            "/private/tmp/**"
+          ],
+          "deny": [],
+          "requireLiteralLeadingDot": false
+        }
+      }
     }
   },
   "bundle": {
@@ -62,7 +79,7 @@
       "minimumSystemVersion": "15.0",
       "signingIdentity": "Developer ID Application: Pranit sharma (QSAMZF2NJ7)",
       "files": {
-        "Resources/Assets.car": "resources/Assets.car"
+        "Resources/app-icons": "icons/app-icons"
       }
     }
   },
diff --git a/vite.config.ts b/vite.config.ts
index fb961bcf3..87f4144ca 100644
--- a/vite.config.ts
+++ b/vite.config.ts
@@ -201,6 +201,9 @@ export default defineConfig({
       '@pierre/diffs',
       '@pierre/diffs/react',
       '@pierre/precision-diffs',
+      // Milkdown Crepe — lazy-loaded by VaultCrepeEditor, must be pre-bundled to prevent
+      // runtime discovery triggering 504 "Outdated Optimize Dep" in Tauri's WKWebView.
+      '@milkdown/crepe',
     ],
   },
   // Tauri expects a fixed port