diff --git a/CHANGELOG.md b/CHANGELOG.md index a3480a5e5..892cad6e6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,8 +2,17 @@ ## Unreleased +### Changed + +- Browser: keep flattened text as the `auto` bundle format for text-only uploads, matching the previous contract. ZIP is used when raw/native files are present or when `--browser-bundle-format zip` is set, and ChatGPT gets a short extract instruction in those cases. Multiple uploaded text/source files still collapse to one bundle; small `auto` inputs still paste inline unless `--browser-bundle-files` forces an upload. +- Dependencies: update provider SDKs, browser tooling, terminal utilities, development tools, pnpm, and transitive overrides while retaining the two-day release-age policy; refresh the Pages pnpm action pin. +- **Breaking** — Remote: accept only conversation-scoped fields from remote clients. The service overrode six known-dangerous `browserConfig` fields and passed the rest of `BrowserSessionConfig` through to `runBrowserMode` verbatim. The remainder is not inert: `chromePath` names an executable the host spawns, `remoteChrome` a debugger to attach to, `copyProfileSource` a directory to copy a signed-in profile out of, `debugPort` one to expose, and `attachRunning`/`browserTabRef` select an existing ChatGPT tab — with an empty or "current" ref resolving to the first ChatGPT tab in the browser. That makes a bridge token a permission to run code on the host rather than to ask ChatGPT a question. A client may now describe the conversation it wants — URL, model, effort, archive mode, resume target, time budgets — and nothing about the machine. A caller that was setting host-scoped fields is now ignored on them rather than obeyed. +- Remote: stop advertising addresses the service is not listening on. `oracle serve --host 127.0.0.1` printed the host's LAN and tailnet addresses in its startup banner while bound to loopback only. That banner is how an operator decides whether a port needs a tunnel or a firewall rule, and for browser automation behind a bearer token, erring toward "more exposed than it is" is the wrong direction. + ### Fixed +- Browser: honor `--browser-bundle-files` (and MCP `browserBundleFiles`) with the default `auto` attachment policy, so small files are uploaded as one bundle instead of being pasted inline. +- Browser: create fallback source bundles only when ChatGPT rejects an inline paste, including remote runs, and delete generated `oracle-browser-bundle-*` directories after browser runs and dry-runs. - Browser: retain per-file attachment evidence through local/remote upload and send checks, including filename-less images; activate and stabilize the send target without replaying a dispatched prompt. Fixes #418. Thanks @hubofvalley! - Remote: allocate unique upload basenames for primary and fallback attachments that collide after sanitization, preserving original display paths and every payload. Fixes #387. Thanks @postoso! - Browser: attach to running Chrome when `DevToolsActivePort` metadata is absent, with IPv6 support and bounded endpoint retries that include response-body reads. Fixes #414. Thanks @devYRPauli! @@ -15,13 +24,6 @@ - Browser: apply the configured input timeout to prompt preparation so stalled local file assembly fails clearly before launching Chrome. Fixes #381. - Browser: report ChatGPT's rate limit as a rate limit. When ChatGPT covers the page with its "Too many requests — we've temporarily limited access to your conversations" modal, the model-switcher scrape walked it like any other menu and reported its "Got it" button as an available model, so a throttled run failed with `Unable to find model option matching "…". Available: Got it` — a message that sends the reader after a model-naming bug when the correct response is to wait a few minutes. Model selection now probes for the notice first and raises a `chatgpt-throttled` error carrying `retryable: true` and the notice text; without a notice the original diagnosis is unchanged. -### Changed - -- Dependencies: update provider SDKs, browser tooling, terminal utilities, development tools, pnpm, and transitive overrides while retaining the two-day release-age policy; refresh the Pages pnpm action pin. - -- **Breaking** — Remote: accept only conversation-scoped fields from remote clients. The service overrode six known-dangerous `browserConfig` fields and passed the rest of `BrowserSessionConfig` through to `runBrowserMode` verbatim. The remainder is not inert: `chromePath` names an executable the host spawns, `remoteChrome` a debugger to attach to, `copyProfileSource` a directory to copy a signed-in profile out of, `debugPort` one to expose, and `attachRunning`/`browserTabRef` select an existing ChatGPT tab — with an empty or "current" ref resolving to the first ChatGPT tab in the browser. That makes a bridge token a permission to run code on the host rather than to ask ChatGPT a question. A client may now describe the conversation it wants — URL, model, effort, archive mode, resume target, time budgets — and nothing about the machine. A caller that was setting host-scoped fields is now ignored on them rather than obeyed. -- Remote: stop advertising addresses the service is not listening on. `oracle serve --host 127.0.0.1` printed the host's LAN and tailnet addresses in its startup banner while bound to loopback only. That banner is how an operator decides whether a port needs a tunnel or a firewall rule, and for browser automation behind a bearer token, erring toward "more exposed than it is" is the wrong direction. - ## 0.18.0 — 2026-08-14 ### Changed diff --git a/README.md b/README.md index 768874586..ddb8369df 100644 --- a/README.md +++ b/README.md @@ -82,7 +82,7 @@ oracle --dry-run summary --files-report \ --file "!**/*.test.ts" ``` -Generated text bundles include stable line numbers so answers can cite `path:line`. Binary and large browser inputs can be uploaded or bundled without converting their contents. The [CLI reference](docs/cli-reference.md) lists the file, size, output, and browser controls. +Generated text bundles include stable line numbers so answers can cite `path:line`. In browser mode, one uploaded text/source file stays native and multiple text/source files are packed into one bundle: flattened text for text-only `auto` uploads, or a ZIP when raw files are present or `--browser-bundle-format zip` is set. Native images and documents remain direct attachments when possible. The [CLI reference](docs/cli-reference.md) lists the file, size, output, and browser controls. ## Sessions and follow-ups diff --git a/bin/oracle-cli.ts b/bin/oracle-cli.ts index ab53c6c51..f0d380856 100755 --- a/bin/oracle-cli.ts +++ b/bin/oracle-cli.ts @@ -875,13 +875,13 @@ program .addOption( new Option( "--browser-bundle-files", - "Bundle all attachments into a single archive before uploading.", + "Force one browser upload bundle; auto/zip includes all resolved files. Multi-file text/source uploads already bundle by default (flattened text unless ZIP is selected).", ).default(false), ) .addOption( new Option( "--browser-bundle-format ", - "Bundle format for browser uploads when files are bundled: auto (default), text, or zip.", + "Bundle format for browser uploads: auto (flattened text for text-only, ZIP when raw files are present), text, or zip.", ) .choices(["auto", "text", "zip"]) .default("auto"), diff --git a/docs/browser-mode.md b/docs/browser-mode.md index 89a645f39..571b1541f 100644 --- a/docs/browser-mode.md +++ b/docs/browser-mode.md @@ -82,7 +82,7 @@ Notes: - Launcher mode can optionally copy cookies from the requested browser profile via Oracle’s built-in cookie reader (Keychain/DPAPI aware), but this requires `--browser-cookie-sync` or `browser.cookieSync=true`. - Navigates to `chatgpt.com`, switches the model to the requested GPT-5.5 / GPT-5.4 / GPT-5.2 variant (including `Advanced` → `Model` in the unified picker), optionally activates Deep Research, pastes the prompt, waits for completion, and copies the markdown via the built-in “copy turn” button. - Immediately probes the cookie-authenticated `/api/auth/session` endpoint in the ChatGPT tab and checks only whether it contains a user; returned tokens are never logged. If that endpoint is unavailable, Oracle falls back to the legacy `/backend-api/me` probe and a visible composer plus profile or chat-history authentication signals. Auth pages, visible login controls, resolved sessions without a user, composer-only shells, and pages without profile/history signals still fail with login guidance. - - When `--file` inputs would push the pasted composer content over ~60k characters, we switch to uploading attachments (optionally bundled) and wait for ChatGPT to re-enable the send button before submitting the combined system+user prompt. + - When `--file` inputs would push the pasted composer content over ~60k characters, we switch to uploads and wait for ChatGPT to re-enable the send button before submitting the combined system+user prompt. A single text/source file is uploaded directly; multiple text/source files are packed into one bundle. Text-only `auto` bundles stay flattened text; ZIP is used when raw files are present or `--browser-bundle-format zip` is set. - Launcher mode cleans up the temporary profile unless `--browser-keep-browser` is passed. 3. **Session integration** – browser sessions use the normal log writer, add `mode: "browser"` plus `browser.config/runtime` metadata, and persist Chrome pid/port or websocket attach metadata plus the Oracle-owned target/tab URL for reattach. @@ -123,8 +123,8 @@ Notes: - Filename-less attachment previews are accepted only when that file's assignment creates a distinct removal control in the active composer. Oracle retains that per-file evidence through local/remote upload, completion, and send readiness; removing or replacing the control invalidates it. A generic file count or an old attachment is not proof of a new upload. - Before sending, Oracle activates the target and checks stable, unobstructed button coordinates. It scrolls only if the button is offscreen, then measures again. Once a click is dispatched, Oracle waits for the original turn to commit and never retries with Enter or an upload fallback merely because the prompt remains staged. An ambiguous commit timeout preserves diagnostics for inspection; do not blindly rerun it. Truncation detected before dispatch can still use the normal upload fallback. - `--browser-inline-files`: alias for `--browser-attachments never` (forces inline paste; never uploads attachments). -- `--browser-bundle-files`: bundle all resolved attachments into a single temp file before uploading (only used when uploads are enabled/selected). -- `--browser-bundle-format `: choose the bundle format. `auto` uses a text bundle for text-only inputs and a byte-preserving ZIP when bundled inputs include raw files; `text` keeps the single Markdown-style text bundle; `zip` archives the original file bytes. ZIP bundle inputs are capped at 128 MiB because bundle creation is in-memory. +- `--browser-bundle-files`: force one browser upload bundle, including when `auto` would otherwise paste small files inline. With `auto` or `zip`, it contains all resolved attachments. Explicit `text` can flatten only text/source files, leaving native attachments separate. Without this flag, Oracle already bundles multiple text/source uploads while leaving images, PDFs, archives, and other native attachments separate when the 10-attachment limit permits. Generated `oracle-browser-bundle-*` directories are deleted after the run or dry-run. +- `--browser-bundle-format `: choose the bundle format. `auto` keeps the established flattened-text bundle for text-only uploads and uses a byte-preserving ZIP when raw/native files are present; `text` always flattens; `zip` always archives. ZIP inputs are capped at 128 MiB because bundle creation is in-memory. Oracle adds a short composer instruction telling ChatGPT to extract ZIP bundles into its sandbox before inspection. - sqlite bindings: automatic rebuilds now require `ORACLE_ALLOW_SQLITE_REBUILD=1`. Without it, the CLI logs instructions instead of running `pnpm rebuild` on your behalf. - `--model`: the same GPT-5.6 aliases work in API and browser mode. Use `gpt-5.6` for the current GPT-5.6 default or `gpt-5.6-sol` to pin Sol; browser mode maps either alias to the `GPT-5.6 Sol` picker entry, while API mode sends the corresponding first-party OpenAI model ID. GPT-5.2 base, Instant, and Thinking aliases remain available through the API but browser mode rejects them because ChatGPT retired those picker entries. Legacy Pro aliases still resolve to the latest Pro picker target. - Live Chrome cookie copying is disabled by default. The recommended migration is `--browser-manual-login`, which keeps token rotation inside a dedicated persistent automation profile. To retain the old launcher behavior, pass `--browser-cookie-sync` or set `browser.cookieSync=true` in the user config; Oracle warns about the live-session invalidation risk. When enabled, cookie copy is mandatory—if Oracle cannot copy cookies, the run exits early. Oracle copies a small ChatGPT auth/Cloudflare allowlist to avoid oversized request headers; use `--browser-cookie-names` only when you need to override that set. @@ -386,7 +386,7 @@ This mode is ideal when you have a macOS VM (or spare Mac mini) logged into Chat ## Limitations / Follow-Up Plan -- **Attachment lifecycle** – in `auto` mode we prefer inlining files into the composer (fewer moving parts). When we do upload, each `--file` path is uploaded separately (or bundled) so ChatGPT can ingest filenames/content. The automation waits for uploads to finish (send button enabled, upload chips visible) before submitting. When inline paste is rejected by ChatGPT (too large), Oracle retries automatically with uploads. +- **Attachment lifecycle** – in `auto` mode we prefer inlining small text inputs into the composer. When uploads are selected, one text/source file stays native and multiple text/source files become one bundle. Text-only `auto` bundles stay flattened text so existing workflows keep direct text ingestion; `--browser-bundle-format zip` (or mixed raw inputs) creates a ZIP plus an extract instruction. Images, PDFs, archives, and other native attachments stay separate unless `--browser-bundle-files` is set or the upload cap requires a single archive. `--browser-bundle-files` selects the upload plan even for small auto inputs. Fallback bundles are created only if ChatGPT rejects the inline paste, including on remote hosts. Generated bundle directories are removed after the run. The automation waits for uploads to finish (send button enabled, upload chips visible) before submitting. - **Model picker drift** – we rely on heuristics to pick GPT-5.6 / GPT-5.5 / GPT-5.4 / GPT-5.2 variants. If OpenAI changes the DOM we need to refresh the selectors quickly. Consider snapshot tests or a small “self check” command. - **Non-mac platforms** – window hiding uses AppleScript today; Linux/Windows just ignore the flag. We should detect platforms explicitly and document the behavior. - **Streaming UX** – browser runs cannot stream tokens, so we emit heartbeat/status logs while waiting. Investigate whether we can stream clipboard deltas via mutation observers for a closer UX. diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 1732986c5..443b113c9 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -100,7 +100,7 @@ See [OpenAI / Azure / OpenRouter](openai-endpoints.md) and [OpenRouter](openrout | `--browser-keep-browser` | Keep the browser open after the run. | | `--browser-headless`, `--browser-hide-window` | Visibility controls. | | `--browser-attachments ` | Attach files inline vs upload. | -| `--browser-bundle-files`, `--browser-bundle-format ` | Bundle browser uploads as text or byte-preserving ZIP. | +| `--browser-bundle-files`, `--browser-bundle-format ` | Multi-file source uploads bundle by default; `auto` keeps flattened text unless ZIP is selected or raw files are present. | | `--browser-chrome-path`, `--browser-cookie-path` | Override Chrome / cookie store discovery (Linux / Windows). | See [Browser Mode](browser-mode.md) for usage. diff --git a/docs/manual-tests.md b/docs/manual-tests.md index f6d24e831..03d4dad13 100644 --- a/docs/manual-tests.md +++ b/docs/manual-tests.md @@ -182,6 +182,11 @@ Confirm the logs report a verified GPT-5.5 model followed by `Thinking time: Pro `pnpm run oracle -- --engine browser --browser-manual-login --model gpt-5.5 --prompt "Summarize the key idea from the attached note" --file /tmp/browser-md.txt` Ensure upload logs show “Attachment queued” and the answer references the file contents explicitly. +3b. **GPT-5.5 + multi-file ZIP** +Create `/tmp/oracle-zip-smoke/src/one.txt` and `/tmp/oracle-zip-smoke/src/two.txt` with distinct sentinel text, then run +`pnpm run oracle -- --engine browser --browser-manual-login --model gpt-5.5 --browser-attachments always --browser-bundle-format zip --prompt "Extract the attached bundle, report both relative paths, and quote each sentinel." --file /tmp/oracle-zip-smoke/src` +Confirm Oracle uploads one `attachments-bundle.zip`, the submitted composer text includes the extraction instruction, and the answer reports both paths and sentinels from the extracted tree. + 4. **GPT-5.5 + attachment (verbose)** Prepare `/tmp/browser-report.txt` with faux metrics, then run `pnpm run oracle -- --engine browser --browser-manual-login --model gpt-5.5 --prompt "Use the attachment to report current CPU and memory figures" --file /tmp/browser-report.txt --verbose` diff --git a/docs/mcp.md b/docs/mcp.md index 4fd63bb28..7964bce17 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -29,6 +29,7 @@ Claude Code can call `oracle-mcp` and ask a subscription-backed ChatGPT browser - Inputs: `prompt` (required), `files?: string[]` (globs), `model?: string` (defaults to CLI), `engine?: "api" | "browser"` (optional; Oracle follows CLI defaults: `ORACLE_ENGINE` and the effective config first, then API when `OPENAI_API_KEY` is set, otherwise browser), `slug?: string`. - Presets: `preset?: "chatgpt-pro-heavy"` applies browser mode + current Pro model alias + extended thinking, unless the request overrides those fields. - Browser-only extras: `browserAttachments?: "auto"|"never"|"always"`, `browserBundleFiles?: boolean`, `browserBundleFormat?: "auto"|"text"|"zip"`, `browserThinkingTime?: "light"|"standard"|"extended"|"extra-high"|"pro"|"heavy"`, `browserResearchMode?: "deep"`, `browserFollowUps?: string[]`, `browserArchive?: "auto"|"always"|"never"`, `browserKeepBrowser?: boolean`, `browserModelLabel?: string`, `browserModelStrategy?: "select"|"current"|"ignore"`, `generateImage?: string`, `outputPath?: string`. +- Browser file uploads: one text/source file stays native, while multiple text/source files default to one bundle. `browserBundleFormat:"auto"` keeps flattened text for text-only uploads and uses ZIP when raw files are present. Set `browserBundleFormat:"zip"` for a filesystem tree, or `browserBundleFiles:true` to force one all-file bundle. - Dry runs: set `dryRun: true` to preview the resolved request without creating a session or touching the browser. - Behavior: starts a session, runs it with the chosen engine, returns final output + metadata. Background/foreground follows the CLI (e.g., GPT‑5 Pro detaches by default). If API mode fails because `OPENAI_API_KEY` is missing and you have ChatGPT Pro, retry with `engine: "browser"` or `preset: "chatgpt-pro-heavy"` to use your signed-in ChatGPT session instead of an API key. - Logging: emits MCP logs (`info` per line, `debug` for streamed chunks with byte sizes). If browser prerequisites are missing, returns an error payload instead of running. diff --git a/skills/oracle/SKILL.md b/skills/oracle/SKILL.md index 05c152c86..e5e2b7ae9 100644 --- a/skills/oracle/SKILL.md +++ b/skills/oracle/SKILL.md @@ -147,7 +147,10 @@ are essential to the question. 4.x; availability depends on engine and provider. - API runs require explicit user consent because they may incur usage costs. - Browser attachments use `--browser-attachments auto|never|always`. -- For many files, add `--browser-bundle-files --browser-bundle-format auto|zip`. +- Browser uploads keep one text/source file native and bundle multiple + text/source files. `auto` keeps flattened text for text-only uploads; use + `--browser-bundle-format zip` for a filesystem tree, or `--browser-bundle-files` + to force every resolved attachment into one bundle. - Reuse an existing Chrome session with `--browser-tab `, `--browser-attach-running`, or `--remote-chrome `. - Use `--browser-model-strategy select|current|ignore` to control picker diff --git a/src/browser/index.ts b/src/browser/index.ts index 86bf4a035..824f2cb37 100644 --- a/src/browser/index.ts +++ b/src/browser/index.ts @@ -795,6 +795,7 @@ async function captureDeepResearchTargetBaseline( type BrowserSubmissionFallback = { prompt: string; attachments: BrowserAttachment[]; + prepare?: () => Promise; }; async function runSubmissionWithRecovery({ @@ -832,12 +833,15 @@ async function runSubmissionWithRecovery({ const isPromptTooLarge = hasBrowserErrorCode(error, "prompt-too-large"); if (fallbackSubmission && isPromptTooLarge && !usedFallbackSubmission) { + usedFallbackSubmission = true; + logger("[browser] Inline prompt too large; retrying with file uploads."); + if (fallbackSubmission.prepare) { + await fallbackSubmission.prepare(); + } assertUniqueAttachmentBasenames(fallbackSubmission.attachments, { stage: "upload-fallback", subject: "The inline prompt was too large, but its upload fallback", }); - usedFallbackSubmission = true; - logger("[browser] Inline prompt too large; retrying with file uploads."); await prepareFallbackSubmission(); currentPrompt = fallbackSubmission.prompt; currentAttachments = fallbackSubmission.attachments; diff --git a/src/browser/prompt.ts b/src/browser/prompt.ts index d2529571b..0a03dc9e1 100644 --- a/src/browser/prompt.ts +++ b/src/browser/prompt.ts @@ -102,6 +102,11 @@ export function isRawUploadFile(filePath: string): boolean { return MEDIA_EXTENSIONS.has(ext) || ARCHIVE_EXTENSIONS.has(ext); } +export interface BrowserPendingFallbackBundle { + format: Exclude; + scope: "text-only" | "all"; +} + export interface BrowserPromptArtifacts { markdown: string; composerText: string; @@ -115,6 +120,7 @@ export interface BrowserPromptArtifacts { composerText: string; attachments: BrowserAttachment[]; bundled?: BrowserBundleMetadata | null; + pendingBundle?: BrowserPendingFallbackBundle | null; } | null; bundled?: BrowserBundleMetadata | null; } @@ -144,6 +150,19 @@ interface BrowserBundleSource { } type ResolvedBrowserBundleFormat = Exclude; +type BrowserBundleScope = "none" | "text-only" | "all"; + +interface PendingFallbackBundle { + format: ResolvedBrowserBundleFormat; + scope: Exclude; + sections: FileSection[]; + textSources: BrowserBundleSource[]; + allSources: BrowserBundleSource[]; + rawUploadAttachments: BrowserAttachment[]; +} + +const GENERATED_BUNDLE_DIR_PREFIX = "oracle-browser-bundle-"; +const pendingFallbackBundles = new WeakMap(); function formatSectionsForBundle( sections: Array<{ displayPath: string; content: string }>, @@ -157,36 +176,61 @@ function formatSectionsForBundle( function resolveBrowserBundleFormat( format: BrowserBundleFormat, - sources: { hasRawUploadFiles: boolean }, + { hasRawUploadFiles }: { hasRawUploadFiles: boolean }, ): ResolvedBrowserBundleFormat { if (format !== "auto") { return format; } - return sources.hasRawUploadFiles ? "zip" : "text"; + return hasRawUploadFiles ? "zip" : "text"; } -function shouldWriteBrowserBundle( +function resolveBrowserBundleScope( format: ResolvedBrowserBundleFormat, { - attachmentCount, bundleRequested, - textSourceCount, - textPlanShouldBundle, + rawAttachmentCount, + textAttachmentCount, }: { - attachmentCount: number; bundleRequested: boolean; - textSourceCount: number; - textPlanShouldBundle: boolean; + rawAttachmentCount: number; + textAttachmentCount: number; }, -): boolean { - if (format === "zip") { - return ( - textPlanShouldBundle || - (bundleRequested && attachmentCount > 0) || - attachmentCount > MAX_BROWSER_ATTACHMENTS - ); +): BrowserBundleScope { + const attachmentCount = textAttachmentCount + rawAttachmentCount; + if (attachmentCount === 0) { + return "none"; + } + + if (format === "text") { + if (textAttachmentCount === 0) { + return "none"; + } + const shouldBundleText = + bundleRequested || textAttachmentCount > 1 || attachmentCount > MAX_BROWSER_ATTACHMENTS; + return shouldBundleText ? "text-only" : "none"; } - return textSourceCount > 0 && (textPlanShouldBundle || attachmentCount > MAX_BROWSER_ATTACHMENTS); + + if (bundleRequested) { + return "all"; + } + + // Preserve native uploads for images, PDFs, archives, and other raw inputs. + // Multiple text/source files benefit from a real filesystem tree, so bundle + // those into one ZIP while leaving any native attachments alongside it. + if (textAttachmentCount > 1) { + return rawAttachmentCount + 1 <= MAX_BROWSER_ATTACHMENTS ? "text-only" : "all"; + } + + return attachmentCount > MAX_BROWSER_ATTACHMENTS ? "all" : "none"; +} + +function appendZipBundleInstruction(composerText: string, originalCount: number): string { + const fileLabel = originalCount === 1 ? "file" : "files"; + const instruction = [ + `The attached \`attachments-bundle.zip\` contains ${originalCount} selected ${fileLabel} with relative paths preserved.`, + "Extract it into a temporary directory, then inspect the resulting file tree with filesystem and search tools before answering.", + ].join(" "); + return [composerText, instruction].filter(Boolean).join("\n\n").trim(); } function assertAttachmentCount( @@ -212,6 +256,143 @@ function assertUniqueAttachmentBasenames(attachments: BrowserAttachment[], cwd: ); } +async function applyWrittenBundle({ + sections, + sources, + format, + scope, + rawUploadAttachments, + composerText, +}: { + sections: FileSection[]; + sources: BrowserBundleSource[]; + format: ResolvedBrowserBundleFormat; + scope: Exclude; + rawUploadAttachments: BrowserAttachment[]; + composerText: string; +}): Promise<{ + attachments: BrowserAttachment[]; + bundled: BrowserBundleMetadata; + composerText: string; + tokenEstimateText: string; +}> { + const writtenBundle = await writeBrowserBundle(sections, sources, format); + const attachments = [writtenBundle.attachment]; + if (scope === "text-only") { + attachments.push(...rawUploadAttachments); + } + assertAttachmentCount(attachments, format); + return { + attachments, + bundled: writtenBundle.metadata, + composerText: + format === "zip" + ? appendZipBundleInstruction(composerText, writtenBundle.metadata.originalCount) + : composerText, + tokenEstimateText: writtenBundle.tokenEstimateText, + }; +} + +function generatedBundleDirectory(attachment: BrowserAttachment): string | null { + if (!attachment.generatedBundle) return null; + const dir = path.dirname(attachment.path); + return path.basename(dir).startsWith(GENERATED_BUNDLE_DIR_PREFIX) ? dir : null; +} + +export function listGeneratedBrowserBundleDirs(artifacts: BrowserPromptArtifacts): string[] { + const dirs = new Set(); + for (const attachment of [...artifacts.attachments, ...(artifacts.fallback?.attachments ?? [])]) { + const dir = generatedBundleDirectory(attachment); + if (dir) dirs.add(dir); + } + return [...dirs]; +} + +export async function cleanupGeneratedBrowserBundles( + artifacts: BrowserPromptArtifacts, +): Promise { + await Promise.all( + listGeneratedBrowserBundleDirs(artifacts).map((dir) => + fs.rm(dir, { recursive: true, force: true }), + ), + ); +} + +export async function materializeBrowserFallback( + artifacts: BrowserPromptArtifacts, +): Promise { + const pending = pendingFallbackBundles.get(artifacts); + if (!pending || !artifacts.fallback) { + return artifacts.fallback ?? null; + } + pendingFallbackBundles.delete(artifacts); + const applied = await applyWrittenBundle({ + sections: pending.sections, + sources: pending.scope === "all" ? pending.allSources : pending.textSources, + format: pending.format, + scope: pending.scope, + rawUploadAttachments: pending.rawUploadAttachments, + composerText: artifacts.fallback.composerText, + }); + artifacts.fallback.composerText = applied.composerText; + artifacts.fallback.attachments = applied.attachments; + artifacts.fallback.bundled = applied.bundled; + artifacts.fallback.pendingBundle = null; + return artifacts.fallback; +} + +export async function materializeStagedFallbackBundle({ + composerText, + attachments, + format, + scope, +}: { + composerText: string; + attachments: BrowserAttachment[]; + format: BrowserPendingFallbackBundle["format"]; + scope: BrowserPendingFallbackBundle["scope"]; +}): Promise<{ + composerText: string; + attachments: BrowserAttachment[]; + bundled: BrowserBundleMetadata; +}> { + const textAttachments = attachments.filter((attachment) => !isRawUploadFile(attachment.path)); + const rawAttachments = attachments.filter((attachment) => isRawUploadFile(attachment.path)); + const textSources: BrowserBundleSource[] = textAttachments.map((attachment) => ({ + absolutePath: attachment.path, + displayPath: attachment.displayPath, + sizeBytes: attachment.sizeBytes ?? 0, + })); + const allSources: BrowserBundleSource[] = [ + ...textSources, + ...rawAttachments.map((attachment) => ({ + absolutePath: attachment.path, + displayPath: attachment.displayPath, + sizeBytes: attachment.sizeBytes ?? 0, + })), + ]; + const sections: FileSection[] = await Promise.all( + textAttachments.map(async (attachment, index) => { + const content = await fs.readFile(attachment.path, "utf8"); + return { + index: index + 1, + absolutePath: attachment.path, + displayPath: attachment.displayPath, + content, + sectionText: "", + }; + }), + ); + return applyWrittenBundle({ + sections, + sources: scope === "all" ? allSources : textSources, + format, + scope, + rawUploadAttachments: rawAttachments, + composerText, + }); +} + async function writeBrowserBundle( sections: FileSection[], sources: BrowserBundleSource[], @@ -342,10 +523,10 @@ export async function assembleBrowserPrompt( .join("\n\n") .trim(); const selectedPlan = - attachmentsPolicy === "always" - ? uploadPlan - : attachmentsPolicy === "never" - ? inlinePlan + attachmentsPolicy === "never" + ? inlinePlan + : attachmentsPolicy === "always" || bundleRequested + ? uploadPlan : inlineComposerText.length <= DEFAULT_BROWSER_INLINE_CHAR_BUDGET || sections.length === 0 ? inlinePlan : uploadPlan; @@ -366,13 +547,13 @@ export async function assembleBrowserPrompt( const resolvedBundleFormat = resolveBrowserBundleFormat(bundleFormat, { hasRawUploadFiles: rawUploadAttachments.length > 0, }); - const shouldBundle = shouldWriteBrowserBundle(resolvedBundleFormat, { - attachmentCount: attachments.length, + const bundleScope = resolveBrowserBundleScope(resolvedBundleFormat, { bundleRequested, - textSourceCount: textBundleSources.length, - textPlanShouldBundle: selectedPlan.shouldBundle, + rawAttachmentCount: rawUploadAttachments.length, + textAttachmentCount: selectedPlan.attachments.length, }); - const composerText = ( + const shouldBundle = bundleScope !== "none"; + let composerText = ( !shouldBundle && selectedPlan.inlineBlock ? [...baseComposerSections, selectedPlan.inlineBlock] : baseComposerSections @@ -383,21 +564,23 @@ export async function assembleBrowserPrompt( let bundleText: string | null = null; let bundled: BrowserBundleMetadata | null = null; - if (shouldBundle) { - const writtenBundle = await writeBrowserBundle( + if (bundleScope !== "none") { + const writtenBundle = await applyWrittenBundle({ sections, - resolvedBundleFormat === "zip" ? allBundleSources : textBundleSources, - resolvedBundleFormat, - ); + sources: bundleScope === "all" ? allBundleSources : textBundleSources, + format: resolvedBundleFormat, + scope: bundleScope, + rawUploadAttachments, + composerText, + }); bundleText = writtenBundle.tokenEstimateText; attachments.length = 0; - attachments.push(writtenBundle.attachment); - if (resolvedBundleFormat === "text") { - attachments.push(...rawUploadAttachments); - } - bundled = writtenBundle.metadata; + attachments.push(...writtenBundle.attachments); + bundled = writtenBundle.bundled; + composerText = writtenBundle.composerText; + } else { + assertAttachmentCount(attachments, resolvedBundleFormat); } - assertAttachmentCount(attachments, resolvedBundleFormat); assertUniqueAttachmentBasenames(attachments, cwd); const inlineFileCount = shouldBundle ? 0 : selectedPlan.inlineFileCount; @@ -405,13 +588,17 @@ export async function assembleBrowserPrompt( ? MODEL_CONFIGS[runOptions.model] : MODEL_CONFIGS["gpt-5.1"]; const tokenizer = deps.tokenizeImpl ?? modelConfig.tokenizer; - const tokenizerUserContent = - inlineFileCount > 0 && selectedPlan.inlineBlock - ? [userPrompt, selectedPlan.inlineBlock] - .filter((value) => Boolean(value?.trim())) - .join("\n\n") - .trim() - : userPrompt; + const tokenizerUserSections = [userPrompt]; + if (inlineFileCount > 0 && selectedPlan.inlineBlock) { + tokenizerUserSections.push(selectedPlan.inlineBlock); + } + if (shouldBundle && resolvedBundleFormat === "zip" && bundled) { + tokenizerUserSections.push(appendZipBundleInstruction("", bundled.originalCount)); + } + const tokenizerUserContent = tokenizerUserSections + .filter((value) => Boolean(value?.trim())) + .join("\n\n") + .trim(); const tokenizerMessages = [ systemPrompt ? { role: "system", content: systemPrompt } : null, tokenizerUserContent ? { role: "user", content: tokenizerUserContent } : null, @@ -431,41 +618,42 @@ export async function assembleBrowserPrompt( } let fallback: BrowserPromptArtifacts["fallback"] = null; + let pendingFallback: PendingFallbackBundle | undefined; if (attachmentsPolicy === "auto" && selectedPlan.mode === "inline" && sections.length > 0) { const fallbackComposerText = baseComposerSections.join("\n\n").trim(); const fallbackAttachments = [...uploadPlan.attachments, ...rawUploadAttachments]; - let fallbackBundled: BrowserBundleMetadata | null = null; const fallbackBundleFormat = resolveBrowserBundleFormat(bundleFormat, { hasRawUploadFiles: rawUploadAttachments.length > 0, }); - const fallbackShouldBundle = shouldWriteBrowserBundle(fallbackBundleFormat, { - attachmentCount: fallbackAttachments.length, + const fallbackBundleScope = resolveBrowserBundleScope(fallbackBundleFormat, { bundleRequested, - textSourceCount: textBundleSources.length, - textPlanShouldBundle: uploadPlan.shouldBundle, + rawAttachmentCount: rawUploadAttachments.length, + textAttachmentCount: uploadPlan.attachments.length, }); - if (fallbackShouldBundle) { - const writtenBundle = await writeBrowserBundle( - sections, - fallbackBundleFormat === "zip" ? allBundleSources : textBundleSources, - fallbackBundleFormat, - ); - fallbackAttachments.length = 0; - fallbackAttachments.push(writtenBundle.attachment); - if (fallbackBundleFormat === "text") { - fallbackAttachments.push(...rawUploadAttachments); - } - fallbackBundled = writtenBundle.metadata; - } - assertAttachmentCount(fallbackAttachments, fallbackBundleFormat); fallback = { composerText: fallbackComposerText, attachments: fallbackAttachments, - bundled: fallbackBundled, + bundled: null, + pendingBundle: + fallbackBundleScope === "none" + ? null + : { format: fallbackBundleFormat, scope: fallbackBundleScope }, }; + if (fallbackBundleScope !== "none") { + pendingFallback = { + format: fallbackBundleFormat, + scope: fallbackBundleScope, + sections, + textSources: textBundleSources, + allSources: allBundleSources, + rawUploadAttachments, + }; + } else { + assertAttachmentCount(fallbackAttachments, fallbackBundleFormat); + } } - return { + const artifacts: BrowserPromptArtifacts = { markdown, composerText, estimatedInputTokens, @@ -483,4 +671,8 @@ export async function assembleBrowserPrompt( fallback, bundled, }; + if (pendingFallback) { + pendingFallbackBundles.set(artifacts, pendingFallback); + } + return artifacts; } diff --git a/src/browser/sessionRunner.ts b/src/browser/sessionRunner.ts index 26e317c3a..2f245052b 100644 --- a/src/browser/sessionRunner.ts +++ b/src/browser/sessionRunner.ts @@ -10,9 +10,13 @@ import type { SessionArtifact, } from "../sessionStore.js"; import { runBrowserMode } from "../browserMode.js"; -import type { BrowserRunResult } from "../browserMode.js"; +import type { BrowserRunOptions, BrowserRunResult } from "../browserMode.js"; import { DEFAULT_BROWSER_CONFIG } from "./config.js"; -import { assembleBrowserPrompt } from "./prompt.js"; +import { + assembleBrowserPrompt, + cleanupGeneratedBrowserBundles, + materializeBrowserFallback, +} from "./prompt.js"; import { BrowserAutomationError } from "../oracle/errors.js"; import type { BrowserArchiveResult, BrowserLogger } from "./types.js"; import { @@ -137,6 +141,7 @@ export async function runBrowserSessionExecution( ): Promise { const assemblePrompt = deps.assemblePrompt ?? assembleBrowserPrompt; const executeBrowser = deps.executeBrowser ?? runBrowserMode; + const persistRuntimeHint = deps.persistRuntimeHint ?? (() => {}); const inputTimeoutMs = browserConfig.inputTimeoutMs ?? DEFAULT_BROWSER_CONFIG.inputTimeoutMs; let preparationTimeout: ReturnType | undefined; let promptArtifacts: Awaited>; @@ -163,6 +168,35 @@ export async function runBrowserSessionExecution( clearTimeout(preparationTimeout); } } + try { + return await executeAssembledBrowserSession({ + runOptions, + browserConfig, + log, + promptArtifacts, + executeBrowser, + persistRuntimeHint, + }); + } finally { + await cleanupGeneratedBrowserBundles(promptArtifacts); + } +} + +async function executeAssembledBrowserSession({ + runOptions, + browserConfig, + log, + promptArtifacts, + executeBrowser, + persistRuntimeHint, +}: { + runOptions: RunOracleOptions; + browserConfig: BrowserSessionConfig; + log: (message?: string) => void; + promptArtifacts: Awaited>; + executeBrowser: NonNullable; + persistRuntimeHint: NonNullable; +}): Promise { if (runOptions.verbose) { log( chalk.dim( @@ -223,7 +257,21 @@ export async function runBrowserSessionExecution( if (runOptions.verbose) { log(chalk.dim("Chrome automation does not stream output; this may take a minute...")); } - const persistRuntimeHint = deps.persistRuntimeHint ?? (() => {}); + let fallbackSubmission: BrowserRunOptions["fallbackSubmission"] = undefined; + if (promptArtifacts.fallback) { + fallbackSubmission = { + prompt: promptArtifacts.fallback.composerText, + attachments: promptArtifacts.fallback.attachments, + pendingBundle: promptArtifacts.fallback.pendingBundle ?? undefined, + prepare: async () => { + const prepared = await materializeBrowserFallback(promptArtifacts); + if (!prepared || !fallbackSubmission) return; + fallbackSubmission.prompt = prepared.composerText; + fallbackSubmission.attachments = prepared.attachments; + fallbackSubmission.pendingBundle = undefined; + }, + }; + } const executionBrowserConfig = runOptions.browserResumeConversationUrl ? { ...browserConfig, resumeConversationUrl: runOptions.browserResumeConversationUrl } : browserConfig; @@ -232,12 +280,7 @@ export async function runBrowserSessionExecution( browserResult = await executeBrowser({ prompt: promptArtifacts.composerText, attachments: promptArtifacts.attachments, - fallbackSubmission: promptArtifacts.fallback - ? { - prompt: promptArtifacts.fallback.composerText, - attachments: promptArtifacts.fallback.attachments, - } - : undefined, + fallbackSubmission, config: executionBrowserConfig, log: automationLogger, heartbeatIntervalMs: runOptions.heartbeatIntervalMs, diff --git a/src/browser/types.ts b/src/browser/types.ts index 58c701a62..b0744c766 100644 --- a/src/browser/types.ts +++ b/src/browser/types.ts @@ -127,7 +127,15 @@ export interface BrowserRunOptions { * Optional secondary submission to try if the initial prompt is rejected by ChatGPT * (e.g. inline file paste exceeds composer limits). Intended for auto inline->upload fallback. */ - fallbackSubmission?: { prompt: string; attachments: BrowserAttachment[] }; + fallbackSubmission?: { + prompt: string; + attachments: BrowserAttachment[]; + prepare?: () => Promise; + pendingBundle?: { + format: "text" | "zip"; + scope: "text-only" | "all"; + }; + }; config?: BrowserAutomationConfig; log?: BrowserLogger; heartbeatIntervalMs?: number; diff --git a/src/cli/dryRun.ts b/src/cli/dryRun.ts index dec86b1da..442f17946 100644 --- a/src/cli/dryRun.ts +++ b/src/cli/dryRun.ts @@ -11,7 +11,11 @@ import { type PreviewMode, } from "../oracle.js"; import { isKnownModel } from "../oracle/modelResolver.js"; -import { assembleBrowserPrompt, type BrowserPromptArtifacts } from "../browser/prompt.js"; +import { + assembleBrowserPrompt, + cleanupGeneratedBrowserBundles, + type BrowserPromptArtifacts, +} from "../browser/prompt.js"; import type { BrowserAttachment } from "../browser/types.js"; import type { BrowserSessionConfig } from "../sessionStore.js"; import { buildTokenEstimateSuffix, formatAttachmentLabel } from "../browser/promptSummary.js"; @@ -113,19 +117,23 @@ async function runBrowserDryRun( validateBrowserFollowUps(runOptions, browserConfig); const assemblePromptImpl = deps.assembleBrowserPromptImpl ?? assembleBrowserPrompt; const artifacts = await assemblePromptImpl(runOptions, { cwd }); - const suffix = buildTokenEstimateSuffix(artifacts); - const displayModel = formatBrowserModelTarget({ - model: runOptions.model, - desiredModel: browserConfig?.desiredModel, - modelStrategy: browserConfig?.modelStrategy, - }); - const headerLine = `[dry-run] Oracle (${version}) would launch browser mode (${displayModel}) with ~${artifacts.estimatedInputTokens.toLocaleString()} tokens${suffix}.`; - log(chalk.cyan(headerLine)); - logBrowserControlPlan(browserConfig, log, "dry-run"); - logBrowserFollowUpSummary(runOptions.browserFollowUps, log, "dry-run"); - logBrowserCookieStrategy(browserConfig, log, "dry-run"); - logBrowserArchivePolicy(browserConfig, log, "dry-run"); - logBrowserFileSummary(artifacts, log, "dry-run"); + try { + const suffix = buildTokenEstimateSuffix(artifacts); + const displayModel = formatBrowserModelTarget({ + model: runOptions.model, + desiredModel: browserConfig?.desiredModel, + modelStrategy: browserConfig?.modelStrategy, + }); + const headerLine = `[dry-run] Oracle (${version}) would launch browser mode (${displayModel}) with ~${artifacts.estimatedInputTokens.toLocaleString()} tokens${suffix}.`; + log(chalk.cyan(headerLine)); + logBrowserControlPlan(browserConfig, log, "dry-run"); + logBrowserFollowUpSummary(runOptions.browserFollowUps, log, "dry-run"); + logBrowserCookieStrategy(browserConfig, log, "dry-run"); + logBrowserArchivePolicy(browserConfig, log, "dry-run"); + logBrowserFileSummary(artifacts, log, "dry-run"); + } finally { + await cleanupGeneratedBrowserBundles(artifacts); + } } function logBrowserControlPlan( @@ -211,42 +219,46 @@ export async function runBrowserPreview( validateBrowserFollowUps(runOptions, browserConfig); const assemblePromptImpl = deps.assembleBrowserPromptImpl ?? assembleBrowserPrompt; const artifacts = await assemblePromptImpl(runOptions, { cwd }); - const suffix = buildTokenEstimateSuffix(artifacts); - const displayModel = formatBrowserModelTarget({ - model: runOptions.model, - desiredModel: browserConfig?.desiredModel, - modelStrategy: browserConfig?.modelStrategy, - }); - const headerLine = `[preview] Oracle (${version}) browser mode (${displayModel}) with ~${artifacts.estimatedInputTokens.toLocaleString()} tokens${suffix}.`; - log(chalk.cyan(headerLine)); - logBrowserControlPlan(browserConfig, log, "preview"); - logBrowserFollowUpSummary(runOptions.browserFollowUps, log, "preview"); - logBrowserCookieStrategy(browserConfig, log, "preview"); - logBrowserFileSummary(artifacts, log, "preview"); - if (previewMode === "json" || previewMode === "full") { - const attachmentSummary = artifacts.attachments.map((attachment) => ({ - path: attachment.path, - displayPath: attachment.displayPath, - sizeBytes: attachment.sizeBytes, - })); - const previewPayload = { + try { + const suffix = buildTokenEstimateSuffix(artifacts); + const displayModel = formatBrowserModelTarget({ model: runOptions.model, - engine: "browser" as const, - composerText: artifacts.composerText, - attachments: attachmentSummary, - inlineFileCount: artifacts.inlineFileCount, - bundled: artifacts.bundled, - tokenEstimate: artifacts.estimatedInputTokens, - browserFollowUps: runOptions.browserFollowUps ?? [], - }; - log(""); - log(chalk.bold("Preview JSON")); - log(JSON.stringify(previewPayload, null, 2)); - } - if (previewMode === "full") { - log(""); - log(chalk.bold("Composer Text")); - log(artifacts.composerText || chalk.dim("(empty prompt)")); + desiredModel: browserConfig?.desiredModel, + modelStrategy: browserConfig?.modelStrategy, + }); + const headerLine = `[preview] Oracle (${version}) browser mode (${displayModel}) with ~${artifacts.estimatedInputTokens.toLocaleString()} tokens${suffix}.`; + log(chalk.cyan(headerLine)); + logBrowserControlPlan(browserConfig, log, "preview"); + logBrowserFollowUpSummary(runOptions.browserFollowUps, log, "preview"); + logBrowserCookieStrategy(browserConfig, log, "preview"); + logBrowserFileSummary(artifacts, log, "preview"); + if (previewMode === "json" || previewMode === "full") { + const attachmentSummary = artifacts.attachments.map((attachment) => ({ + path: attachment.path, + displayPath: attachment.displayPath, + sizeBytes: attachment.sizeBytes, + })); + const previewPayload = { + model: runOptions.model, + engine: "browser" as const, + composerText: artifacts.composerText, + attachments: attachmentSummary, + inlineFileCount: artifacts.inlineFileCount, + bundled: artifacts.bundled, + tokenEstimate: artifacts.estimatedInputTokens, + browserFollowUps: runOptions.browserFollowUps ?? [], + }; + log(""); + log(chalk.bold("Preview JSON")); + log(JSON.stringify(previewPayload, null, 2)); + } + if (previewMode === "full") { + log(""); + log(chalk.bold("Composer Text")); + log(artifacts.composerText || chalk.dim("(empty prompt)")); + } + } finally { + await cleanupGeneratedBrowserBundles(artifacts); } } diff --git a/src/mcp/tools/consult.ts b/src/mcp/tools/consult.ts index 96f43374b..7341ae4ed 100644 --- a/src/mcp/tools/consult.ts +++ b/src/mcp/tools/consult.ts @@ -87,12 +87,14 @@ const consultInputShape = { browserBundleFiles: z .boolean() .optional() - .describe("Browser-only: bundle many files into a single upload (helps with upload limits)."), + .describe( + "Browser-only: force one upload bundle; auto/zip includes all resolved files. Multiple text/source uploads already bundle by default (flattened text unless ZIP is selected).", + ), browserBundleFormat: z .enum(["auto", "text", "zip"]) .optional() .describe( - 'Browser-only: bundle upload format when browserBundleFiles is true or auto-bundling is needed. Defaults to "auto"; "auto" uses ZIP when bundled inputs include raw/binary files.', + 'Browser-only: bundle upload format. Defaults to "auto", which keeps flattened text for text-only uploads and uses ZIP when raw files are present; "zip" forces a byte-preserving archive.', ), browserThinkingTime: browserThinkingTimeRawSchema .optional() diff --git a/src/remote/client.ts b/src/remote/client.ts index 2e690047c..2c175d04d 100644 --- a/src/remote/client.ts +++ b/src/remote/client.ts @@ -42,6 +42,7 @@ export function createRemoteBrowserExecutor({ host, token }: RemoteExecutorOptio ? { prompt: options.fallbackSubmission.prompt, attachments: await serializeAttachments(options.fallbackSubmission.attachments ?? []), + bundle: options.fallbackSubmission.pendingBundle, } : undefined, browserConfig: options.config ?? {}, diff --git a/src/remote/server.ts b/src/remote/server.ts index 5fa4e709e..6c54c87df 100644 --- a/src/remote/server.ts +++ b/src/remote/server.ts @@ -9,6 +9,7 @@ import { spawn, spawnSync } from "node:child_process"; import { mkdtemp, rm, mkdir, writeFile, stat, realpath } from "node:fs/promises"; import chalk from "chalk"; import type { BrowserAttachment, BrowserLogger, CookieParam } from "../browser/types.js"; +import { materializeStagedFallbackBundle } from "../browser/prompt.js"; import type { BrowserSessionConfig } from "../sessionManager.js"; import { runBrowserMode } from "../browserMode.js"; import type { BrowserRunResult } from "../browserMode.js"; @@ -223,6 +224,7 @@ export async function createRemoteServer( | { prompt: string; attachments: BrowserAttachment[]; + prepare?: () => Promise; } | undefined; try { @@ -247,6 +249,20 @@ export async function createRemoteServer( prompt: payload.fallbackSubmission.prompt, attachments: fallbackAttachments, }; + const pendingBundle = payload.fallbackSubmission.bundle; + if (pendingBundle) { + fallbackSubmission.prepare = async () => { + if (!fallbackSubmission) return; + const prepared = await materializeStagedFallbackBundle({ + composerText: fallbackSubmission.prompt, + attachments: fallbackSubmission.attachments, + format: pendingBundle.format, + scope: pendingBundle.scope, + }); + fallbackSubmission.prompt = prepared.composerText; + fallbackSubmission.attachments = prepared.attachments; + }; + } } // Reuse the existing browser logger surface so clients see the same log stream. diff --git a/src/remote/types.ts b/src/remote/types.ts index 2946acfa4..ba0e69fff 100644 --- a/src/remote/types.ts +++ b/src/remote/types.ts @@ -18,6 +18,10 @@ export interface RemoteRunPayload { fallbackSubmission?: { prompt: string; attachments: RemoteAttachmentPayload[]; + bundle?: { + format: "text" | "zip"; + scope: "text-only" | "all"; + }; }; browserConfig: BrowserSessionConfig; options: { diff --git a/tests/browser/index.test.ts b/tests/browser/index.test.ts index 80768eca3..0c1dfc081 100644 --- a/tests/browser/index.test.ts +++ b/tests/browser/index.test.ts @@ -900,6 +900,47 @@ describe("runSubmissionWithRecoveryForTest", () => { ]); }); + test("materializes fallback attachments before retrying a prompt-too-large submit", async () => { + const fallbackSubmission = { + prompt: "unbundled fallback", + attachments: [{ path: "/tmp/one.txt", displayPath: "one.txt", sizeBytes: 3 }], + prepare: vi.fn(async () => { + fallbackSubmission.prompt = "bundled fallback"; + fallbackSubmission.attachments = [ + { + path: "/tmp/attachments-bundle.zip", + displayPath: "attachments-bundle.zip", + sizeBytes: 12, + }, + ]; + }), + }; + const submit = vi + .fn() + .mockRejectedValueOnce( + new BrowserAutomationError("prompt too large", { code: "prompt-too-large" }), + ) + .mockResolvedValueOnce({ + baselineTurns: 1, + baselineAssistantText: "ok", + }); + + await runSubmissionWithRecoveryForTest({ + prompt: "inline prompt", + attachments: [], + fallbackSubmission, + submit, + reloadPromptComposer: vi.fn().mockResolvedValue(undefined), + prepareFallbackSubmission: vi.fn().mockResolvedValue(undefined), + logger: vi.fn<(message: string) => void>(), + }); + + expect(fallbackSubmission.prepare).toHaveBeenCalledTimes(1); + expect(submit).toHaveBeenNthCalledWith(2, "bundled fallback", [ + expect.objectContaining({ displayPath: "attachments-bundle.zip" }), + ]); + }); + test("throws when prompt-too-large happens again after fallback", async () => { const submit = vi .fn() diff --git a/tests/browser/prompt.test.ts b/tests/browser/prompt.test.ts index 478e95b9f..074c36be2 100644 --- a/tests/browser/prompt.test.ts +++ b/tests/browser/prompt.test.ts @@ -2,7 +2,13 @@ import fs from "node:fs/promises"; import os from "node:os"; import path from "node:path"; import { describe, expect, test } from "vitest"; -import { assembleBrowserPrompt, isRawUploadFile } from "../../src/browser/prompt.js"; +import { + assembleBrowserPrompt, + cleanupGeneratedBrowserBundles, + isRawUploadFile, + listGeneratedBrowserBundleDirs, + materializeBrowserFallback, +} from "../../src/browser/prompt.js"; import { findAttachmentBasenameCollisions } from "../../src/browser/attachmentValidation.js"; import { createStoredZip } from "../../src/browser/zipBundle.js"; import { DEFAULT_SYSTEM_PROMPT, type MODEL_CONFIGS } from "../../src/oracle.js"; @@ -172,6 +178,50 @@ describe("assembleBrowserPrompt", () => { ); }); + test("auto inline fallback defers text-bundle creation until materialize", async () => { + const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-inline-fallback-text-")); + let result: Awaited> | undefined; + try { + await fs.writeFile(path.join(tempDir, "one.txt"), "one", "utf8"); + await fs.writeFile(path.join(tempDir, "two.txt"), "two", "utf8"); + + result = await assembleBrowserPrompt( + buildOptions({ + file: ["one.txt", "two.txt"], + browserAttachments: "auto", + }), + { cwd: tempDir, tokenizeImpl: fastTokenizer }, + ); + + expect(result.attachmentMode).toBe("inline"); + expect(result.attachments).toEqual([]); + expect(listGeneratedBrowserBundleDirs(result)).toEqual([]); + expect(result.fallback?.bundled).toBeNull(); + expect(result.fallback?.pendingBundle).toEqual({ format: "text", scope: "text-only" }); + expect(result.fallback?.composerText).toBe("Explain the bug"); + expect(result.fallback?.attachments).toEqual([ + expect.objectContaining({ displayPath: "one.txt" }), + expect.objectContaining({ displayPath: "two.txt" }), + ]); + + const fallback = await materializeBrowserFallback(result); + expect(fallback?.attachments).toHaveLength(1); + expect(fallback?.attachments[0]?.displayPath).toMatch(/attachments-bundle\.txt$/); + expect(fallback?.bundled?.format).toBe("text"); + expect(fallback?.pendingBundle).toBeNull(); + const bundleText = await fs.readFile(fallback!.attachments[0]!.path, "utf8"); + expect(bundleText).toContain("### File: one.txt"); + expect(bundleText).toContain("1 | one"); + expect(bundleText).toContain("### File: two.txt"); + expect(bundleText).toContain("1 | two"); + } finally { + if (result) { + await cleanupGeneratedBrowserBundles(result); + } + await fs.rm(tempDir, { recursive: true, force: true }); + } + }); + test("always mode forces uploads even when small", async () => { const options = buildOptions({ prompt: "Explain the bug", @@ -199,15 +249,15 @@ describe("assembleBrowserPrompt", () => { expect(result.fallback).toBeNull(); }); - test("always mode rejects upload attachments with the same basename", async () => { + test("always mode rejects native uploads with the same basename", async () => { const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-duplicate-basename-")); - const firstPath = path.join(tempDir, "first", "SKILL.md"); - const secondPath = path.join(tempDir, "second", "SKILL.md"); + const firstPath = path.join(tempDir, "first", "photo.png"); + const secondPath = path.join(tempDir, "second", "photo.png"); try { await fs.mkdir(path.dirname(firstPath), { recursive: true }); await fs.mkdir(path.dirname(secondPath), { recursive: true }); - await fs.writeFile(firstPath, "first"); - await fs.writeFile(secondPath, "second"); + await fs.writeFile(firstPath, Buffer.from([0x89, 0x50, 0x4e, 0x47])); + await fs.writeFile(secondPath, Buffer.from([0x89, 0x50, 0x4e, 0x47])); await expect( assembleBrowserPrompt( @@ -218,9 +268,9 @@ describe("assembleBrowserPrompt", () => { { cwd: tempDir, tokenizeImpl: fastTokenizer }, ), ).rejects.toMatchObject({ - message: expect.stringContaining('multiple files named "SKILL.md"'), + message: expect.stringContaining('multiple files named "photo.png"'), details: { - collisions: [{ basename: "SKILL.md", files: [firstPath, secondPath] }], + collisions: [{ basename: "photo.png", files: [firstPath, secondPath] }], files: [firstPath, secondPath], }, }); @@ -229,48 +279,40 @@ describe("assembleBrowserPrompt", () => { } }); - test("auto mode rejects basename collisions when it selects upload", async () => { + test("auto mode bundles large duplicate-basename text files as flattened text", async () => { const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-auto-upload-collision-")); const firstPath = path.join(tempDir, "first", "SKILL.md"); const secondPath = path.join(tempDir, "second", "SKILL.md"); - const firstUniquePath = path.join(tempDir, "first", "FIRST.md"); - const secondUniquePath = path.join(tempDir, "second", "SECOND.md"); const largeContent = "x".repeat(31_000); + let result: Awaited> | undefined; try { await fs.mkdir(path.dirname(firstPath), { recursive: true }); await fs.mkdir(path.dirname(secondPath), { recursive: true }); await Promise.all([ fs.writeFile(firstPath, largeContent), fs.writeFile(secondPath, largeContent), - fs.writeFile(firstUniquePath, largeContent), - fs.writeFile(secondUniquePath, largeContent), ]); - const uploadControl = await assembleBrowserPrompt( + result = await assembleBrowserPrompt( buildOptions({ - file: [firstUniquePath, secondUniquePath], + file: [firstPath, secondPath], browserAttachments: "auto", }), { cwd: tempDir, tokenizeImpl: fastTokenizer }, ); - expect(uploadControl.attachmentMode).toBe("upload"); - expect(uploadControl.fallback).toBeNull(); - await expect( - assembleBrowserPrompt( - buildOptions({ - file: [firstPath, secondPath], - browserAttachments: "auto", - }), - { cwd: tempDir, tokenizeImpl: fastTokenizer }, - ), - ).rejects.toMatchObject({ - details: { - collisions: [{ basename: "SKILL.md", files: [firstPath, secondPath] }], - files: [firstPath, secondPath], - }, - }); + expect(result.attachmentMode).toBe("bundle"); + expect(result.fallback).toBeNull(); + expect(result.attachments).toHaveLength(1); + expect(result.attachments[0]?.displayPath).toMatch(/attachments-bundle\.txt$/); + expect(result.bundled?.format).toBe("text"); + const bundleText = await fs.readFile(result.attachments[0]!.path, "utf8"); + expect(bundleText).toContain("### File: first/SKILL.md"); + expect(bundleText).toContain("### File: second/SKILL.md"); } finally { + if (result) { + await cleanupGeneratedBrowserBundles(result); + } await fs.rm(tempDir, { recursive: true, force: true }); } }); @@ -338,13 +380,14 @@ describe("assembleBrowserPrompt", () => { const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-bundled-collision-")); const firstPath = path.join(tempDir, "first", "SKILL.md"); const secondPath = path.join(tempDir, "second", "SKILL.md"); + let result: Awaited> | undefined; try { await fs.mkdir(path.dirname(firstPath), { recursive: true }); await fs.mkdir(path.dirname(secondPath), { recursive: true }); await fs.writeFile(firstPath, "first"); await fs.writeFile(secondPath, "second"); - const result = await assembleBrowserPrompt( + result = await assembleBrowserPrompt( buildOptions({ file: [firstPath, secondPath], browserAttachments: "always", @@ -361,6 +404,9 @@ describe("assembleBrowserPrompt", () => { expect(bundleText).toContain("### File: second/SKILL.md"); expect(bundleText).toContain("1 | second"); } finally { + if (result) { + await cleanupGeneratedBrowserBundles(result); + } await fs.rm(tempDir, { recursive: true, force: true }); } }); @@ -385,8 +431,11 @@ describe("assembleBrowserPrompt", () => { expect(result.fallback).toBeNull(); }); - test("respects custom cwd and multiple files", async () => { - const options = buildOptions({ file: ["docs/one.md", "docs/two.md"] }); + test("respects custom cwd and multiple inline files", async () => { + const options = buildOptions({ + file: ["docs/one.md", "docs/two.md"], + browserAttachments: "never", + }); const result = await assembleBrowserPrompt(options, { cwd: "/root/project", readFilesImpl: async (paths) => @@ -479,6 +528,34 @@ describe("assembleBrowserPrompt", () => { } }); + test("explicit text format keeps the legacy multi-file text bundle", async () => { + const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-explicit-text-bundle-")); + try { + await fs.writeFile(path.join(tempDir, "one.txt"), "one", "utf8"); + await fs.writeFile(path.join(tempDir, "two.txt"), "two", "utf8"); + const result = await assembleBrowserPrompt( + buildOptions({ + file: ["one.txt", "two.txt"], + browserAttachments: "always", + browserBundleFormat: "text", + }), + { cwd: tempDir, tokenizeImpl: fastTokenizer }, + ); + + expect(result.attachments).toHaveLength(1); + expect(result.attachments[0]?.displayPath).toMatch(/attachments-bundle\.txt$/); + expect(result.bundled?.format).toBe("text"); + expect(result.composerText).toBe("Explain the bug"); + const bundleText = await fs.readFile(result.attachments[0]!.path, "utf8"); + expect(bundleText).toContain("### File: one.txt"); + expect(bundleText).toContain("1 | one"); + expect(bundleText).toContain("### File: two.txt"); + expect(bundleText).toContain("1 | two"); + } finally { + await fs.rm(tempDir, { recursive: true, force: true }); + } + }); + test("explicit text format bundles text files to keep mixed uploads within the limit", async () => { const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-mixed-text-bundle-")); try { @@ -623,47 +700,51 @@ describe("assembleBrowserPrompt", () => { expect(inline.tokenEstimateIncludesInlineFiles).toBe(true); }); - test("bundles attachments when more than 10 files", async () => { - const fileNames = Array.from({ length: 11 }, (_, i) => `file${i + 1}.txt`); - const options = buildOptions({ file: fileNames, browserAttachments: "always" }); - const tokenizedContents: string[] = []; - const result = await assembleBrowserPrompt(options, { - cwd: "/repo", - readFilesImpl: async (paths) => - paths.map((entry) => { - const displayPath = path.isAbsolute(entry) ? path.relative("/repo", entry) : entry; - return { - path: path.resolve("/repo", entry), - content: `content for ${displayPath}`, - }; - }), - tokenizeImpl: (messages) => { - const typed = messages as Array<{ content: string }>; - tokenizedContents.push(...typed.map((message) => message.content)); - return fastTokenizer(messages); - }, - }); + test("auto bundles more than 10 text uploads as flattened text", async () => { + const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-many-text-zip-bundle-")); + try { + const fileNames = Array.from({ length: 11 }, (_, i) => `file${i + 1}.txt`); + await Promise.all( + fileNames.map((file) => + fs.writeFile(path.join(tempDir, file), `content for ${file}`, "utf8"), + ), + ); + const tokenizedContents: string[] = []; + const result = await assembleBrowserPrompt( + buildOptions({ file: fileNames, browserAttachments: "always" }), + { + cwd: tempDir, + tokenizeImpl: (messages) => { + const typed = messages as Array<{ content: string }>; + tokenizedContents.push(...typed.map((message) => message.content)); + return fastTokenizer(messages); + }, + }, + ); - expect(result.attachments).toHaveLength(1); - expect(result.attachments[0]?.displayPath).toMatch(/attachments-bundle\.txt$/); - expect(result.attachments[0]?.generatedBundle).toBe(true); - const bundleText = await fs.readFile(result.attachments[0]!.path, "utf8"); - expect(bundleText).toContain("### File: file1.txt"); - expect(bundleText).toContain("Lines: 1-1"); - expect(bundleText).toContain("1 | content for file1.txt"); - expect(tokenizedContents.some((content) => content.includes("1 | content for file1.txt"))).toBe( - true, - ); - expect(result.inlineFileCount).toBe(0); - expect(result.bundled).toEqual({ - originalCount: 11, - bundlePath: result.attachments[0]?.displayPath, - format: "text", - }); + expect(result.attachments).toHaveLength(1); + expect(result.attachments[0]?.displayPath).toMatch(/attachments-bundle\.txt$/); + expect(result.attachments[0]?.generatedBundle).toBe(true); + const bundleText = await fs.readFile(result.attachments[0]!.path, "utf8"); + expect(bundleText).toContain("### File: file1.txt"); + expect(bundleText).toContain("1 | content for file1.txt"); + expect( + tokenizedContents.some((content) => content.includes("1 | content for file1.txt")), + ).toBe(true); + expect(result.composerText).not.toContain("Extract it into a temporary directory"); + expect(result.inlineFileCount).toBe(0); + expect(result.bundled).toEqual({ + originalCount: 11, + bundlePath: result.attachments[0]?.displayPath, + format: "text", + }); + } finally { + await fs.rm(tempDir, { recursive: true, force: true }); + } }); - test("supports opt-in ZIP bundles for browser uploads", async () => { - const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-text-zip-bundle-")); + test("auto bundles two text uploads as flattened text", async () => { + const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-text-auto-bundle-")); try { await fs.mkdir(path.join(tempDir, "src"), { recursive: true }); await fs.writeFile(path.join(tempDir, "src", "a.ts"), "content for src/a.ts", "utf8"); @@ -671,8 +752,6 @@ describe("assembleBrowserPrompt", () => { const options = buildOptions({ file: ["src/a.ts", "src/b.ts"], browserAttachments: "always", - browserBundleFiles: true, - browserBundleFormat: "zip", }); const tokenizedContents: string[] = []; const result = await assembleBrowserPrompt(options, { @@ -685,31 +764,154 @@ describe("assembleBrowserPrompt", () => { }); expect(result.attachments).toHaveLength(1); - expect(result.attachments[0]?.displayPath).toMatch(/attachments-bundle\.zip$/); + expect(result.attachments[0]?.displayPath).toMatch(/attachments-bundle\.txt$/); expect(result.attachments[0]?.generatedBundle).toBe(true); expect(result.bundled).toEqual({ originalCount: 2, bundlePath: result.attachments[0]?.displayPath, - format: "zip", + format: "text", }); - const zipBytes = await fs.readFile(result.attachments[0]!.path); - expect(zipBytes.subarray(0, 4).toString("hex")).toBe("504b0304"); - const entries = readStoredZipEntries(zipBytes); - expect(entries.get("src/a.ts")?.toString("utf8")).toBe("content for src/a.ts"); - expect(entries.get("src/b.ts")?.toString("utf8")).toBe("content for src/b.ts"); - expect(zipBytes.toString("utf8")).not.toContain("1 | content for src/a.ts"); - expect(tokenizedContents.some((content) => content.includes("content for src/a.ts"))).toBe( - true, - ); + const bundleText = await fs.readFile(result.attachments[0]!.path, "utf8"); + expect(bundleText).toContain("### File: src/a.ts"); + expect(bundleText).toContain("1 | content for src/a.ts"); + expect(bundleText).toContain("### File: src/b.ts"); + expect(bundleText).toContain("1 | content for src/b.ts"); expect( tokenizedContents.some((content) => content.includes("1 | content for src/a.ts")), - ).toBe(false); + ).toBe(true); + expect(result.composerText).not.toContain("Extract it into a temporary directory"); + } finally { + await fs.rm(tempDir, { recursive: true, force: true }); + } + }); + + test("keeps native attachments alongside an automatic source ZIP", async () => { + const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-mixed-native-source-zip-")); + try { + await fs.writeFile(path.join(tempDir, "one.ts"), "export const one = 1;\n", "utf8"); + await fs.writeFile(path.join(tempDir, "two.ts"), "export const two = 2;\n", "utf8"); + const imageBytes = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a]); + const imagePath = path.join(tempDir, "reference.png"); + await fs.writeFile(imagePath, imageBytes); + + const result = await assembleBrowserPrompt( + buildOptions({ + file: ["one.ts", "two.ts", "reference.png"], + browserAttachments: "always", + }), + { cwd: tempDir, tokenizeImpl: fastTokenizer }, + ); + + expect(result.attachmentMode).toBe("bundle"); + expect(result.attachments).toHaveLength(2); + const zipAttachment = result.attachments.find((attachment) => attachment.generatedBundle); + expect(zipAttachment?.displayPath).toMatch(/attachments-bundle\.zip$/); + expect(result.attachments).toContainEqual( + expect.objectContaining({ path: imagePath, displayPath: "reference.png" }), + ); + expect(result.bundled?.originalCount).toBe(2); + const entries = readStoredZipEntries(await fs.readFile(zipAttachment!.path)); + expect(entries.get("one.ts")?.toString("utf8")).toBe("export const one = 1;\n"); + expect(entries.get("two.ts")?.toString("utf8")).toBe("export const two = 2;\n"); + expect(entries.has("reference.png")).toBe(false); } finally { await fs.rm(tempDir, { recursive: true, force: true }); } }); - test("auto bundle format chooses ZIP and preserves bytes when bundled inputs include an archive", async () => { + test("keeps native uploads direct when a source ZIP reduces the upload count to 10", async () => { + const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-source-zip-upload-cap-")); + try { + const textFiles = ["one.ts", "two.ts"]; + const rawFiles = Array.from({ length: 9 }, (_, index) => `archive-${index}.zip`); + await Promise.all([ + ...textFiles.map((file) => fs.writeFile(path.join(tempDir, file), file, "utf8")), + ...rawFiles.map((file) => fs.writeFile(path.join(tempDir, file), Buffer.from("PK"))), + ]); + + const result = await assembleBrowserPrompt( + buildOptions({ + file: [...textFiles, ...rawFiles], + browserAttachments: "always", + }), + { cwd: tempDir, tokenizeImpl: fastTokenizer }, + ); + + expect(result.attachments).toHaveLength(10); + const zipAttachment = result.attachments.find((attachment) => attachment.generatedBundle); + expect(zipAttachment?.displayPath).toMatch(/attachments-bundle\.zip$/); + expect(result.bundled?.originalCount).toBe(2); + expect(result.attachments.filter((attachment) => !attachment.generatedBundle)).toHaveLength( + 9, + ); + const entries = readStoredZipEntries(await fs.readFile(zipAttachment!.path)); + expect([...entries.keys()].sort()).toEqual(textFiles); + } finally { + await fs.rm(tempDir, { recursive: true, force: true }); + } + }); + + test("automatically folds native uploads into the ZIP when the upload cap still overflows", async () => { + const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-all-files-upload-cap-")); + try { + const textFiles = ["one.ts", "two.ts"]; + const rawFiles = Array.from({ length: 10 }, (_, index) => `archive-${index}.zip`); + await Promise.all([ + ...textFiles.map((file) => fs.writeFile(path.join(tempDir, file), file, "utf8")), + ...rawFiles.map((file, index) => + fs.writeFile(path.join(tempDir, file), Buffer.from(`PK-${index}`)), + ), + ]); + + const result = await assembleBrowserPrompt( + buildOptions({ + file: [...textFiles, ...rawFiles], + browserAttachments: "always", + }), + { cwd: tempDir, tokenizeImpl: fastTokenizer }, + ); + + expect(result.attachments).toHaveLength(1); + expect(result.attachments[0]?.displayPath).toMatch(/attachments-bundle\.zip$/); + expect(result.bundled?.originalCount).toBe(12); + const entries = readStoredZipEntries(await fs.readFile(result.attachments[0]!.path)); + expect([...entries.keys()].sort()).toEqual([...textFiles, ...rawFiles].sort()); + expect(entries.get("one.ts")?.toString("utf8")).toBe("one.ts"); + expect(entries.get("archive-9.zip")?.toString("utf8")).toBe("PK-9"); + } finally { + await fs.rm(tempDir, { recursive: true, force: true }); + } + }); + + test("keeps multiple native attachments separate by default", async () => { + const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-native-uploads-")); + try { + const firstPath = path.join(tempDir, "first.png"); + const secondPath = path.join(tempDir, "second.pdf"); + await fs.writeFile(firstPath, Buffer.from([0x89, 0x50, 0x4e, 0x47])); + await fs.writeFile(secondPath, Buffer.from("%PDF-1.7\n", "utf8")); + + const result = await assembleBrowserPrompt( + buildOptions({ + file: ["first.png", "second.pdf"], + browserAttachments: "always", + }), + { cwd: tempDir, tokenizeImpl: fastTokenizer }, + ); + + expect(result.attachmentMode).toBe("upload"); + expect(result.bundled).toBeNull(); + expect(result.attachments).toEqual([ + expect.objectContaining({ path: firstPath, displayPath: "first.png" }), + expect.objectContaining({ path: secondPath, displayPath: "second.pdf" }), + ]); + expect(result.composerText).toBe("Explain the bug"); + } finally { + await fs.rm(tempDir, { recursive: true, force: true }); + } + }); + + test("forced auto bundle uses ZIP and preserves archive bytes", async () => { const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-auto-zip-bundle-")); try { const notePath = path.join(tempDir, "note.txt"); @@ -751,7 +953,7 @@ describe("assembleBrowserPrompt", () => { } }); - test("auto bundle format detects archives after directory expansion", async () => { + test("forced auto bundle includes archives found by directory expansion", async () => { const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-dir-zip-bundle-")); try { const sourceDir = path.join(tempDir, "source"); @@ -828,4 +1030,125 @@ describe("assembleBrowserPrompt", () => { await fs.rm(tempDir, { recursive: true, force: true }); } }); + + test("force-bundle with auto uploads a text bundle instead of inlining small files", async () => { + const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-force-bundle-auto-")); + let result: Awaited> | undefined; + try { + await fs.writeFile(path.join(tempDir, "one.txt"), "one", "utf8"); + await fs.writeFile(path.join(tempDir, "two.txt"), "two", "utf8"); + + result = await assembleBrowserPrompt( + buildOptions({ + file: ["one.txt", "two.txt"], + browserAttachments: "auto", + browserBundleFiles: true, + }), + { cwd: tempDir, tokenizeImpl: fastTokenizer }, + ); + + expect(result.attachmentMode).toBe("bundle"); + expect(result.inlineFileCount).toBe(0); + expect(result.fallback).toBeNull(); + expect(result.attachments).toHaveLength(1); + expect(result.attachments[0]?.displayPath).toMatch(/attachments-bundle\.txt$/); + expect(result.bundled).toEqual({ + originalCount: 2, + bundlePath: result.attachments[0]?.displayPath, + format: "text", + }); + expect(result.composerText).not.toContain("Extract it into a temporary directory"); + const bundleText = await fs.readFile(result.attachments[0]!.path, "utf8"); + expect(bundleText).toContain("### File: one.txt"); + expect(bundleText).toContain("1 | one"); + expect(bundleText).toContain("### File: two.txt"); + expect(bundleText).toContain("1 | two"); + } finally { + if (result) { + await cleanupGeneratedBrowserBundles(result); + } + await fs.rm(tempDir, { recursive: true, force: true }); + } + }); + + test("explicit zip format packs text-only uploads into a ZIP", async () => { + const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-explicit-zip-text-")); + let result: Awaited> | undefined; + try { + await fs.writeFile(path.join(tempDir, "one.txt"), "one", "utf8"); + await fs.writeFile(path.join(tempDir, "two.txt"), "two", "utf8"); + + result = await assembleBrowserPrompt( + buildOptions({ + file: ["one.txt", "two.txt"], + browserAttachments: "always", + browserBundleFormat: "zip", + }), + { cwd: tempDir, tokenizeImpl: fastTokenizer }, + ); + + expect(result.attachments).toHaveLength(1); + expect(result.attachments[0]?.displayPath).toMatch(/attachments-bundle\.zip$/); + expect(result.bundled?.format).toBe("zip"); + expect(result.composerText).toContain("Extract it into a temporary directory"); + const entries = readStoredZipEntries(await fs.readFile(result.attachments[0]!.path)); + expect(entries.get("one.txt")?.toString("utf8")).toBe("one"); + expect(entries.get("two.txt")?.toString("utf8")).toBe("two"); + } finally { + if (result) { + await cleanupGeneratedBrowserBundles(result); + } + await fs.rm(tempDir, { recursive: true, force: true }); + } + }); + + test("force-bundle with auto still inlines when attachments are disabled", async () => { + const result = await assembleBrowserPrompt( + buildOptions({ + file: ["a.txt", "b.txt"], + browserAttachments: "never", + browserBundleFiles: true, + }), + { + cwd: "/repo", + readFilesImpl: async (paths) => + paths.map((entry, index) => ({ + path: path.resolve("/repo", entry), + content: `file-${index}`, + })), + tokenizeImpl: fastTokenizer, + }, + ); + + expect(result.attachmentsPolicy).toBe("never"); + expect(result.attachmentMode).toBe("inline"); + expect(result.attachments).toEqual([]); + expect(result.fallback).toBeNull(); + expect(result.composerText).toContain("### File: a.txt"); + expect(result.composerText).toContain("### File: b.txt"); + }); + + test("cleanup removes generated browser bundle directories", async () => { + const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-cleanup-bundle-")); + try { + await fs.writeFile(path.join(tempDir, "one.ts"), "one", "utf8"); + await fs.writeFile(path.join(tempDir, "two.ts"), "two", "utf8"); + const result = await assembleBrowserPrompt( + buildOptions({ + file: ["one.ts", "two.ts"], + browserAttachments: "always", + }), + { cwd: tempDir, tokenizeImpl: fastTokenizer }, + ); + + const bundlePath = result.attachments[0]?.path; + expect(bundlePath).toBeTruthy(); + expect(listGeneratedBrowserBundleDirs(result)).toEqual([path.dirname(bundlePath!)]); + await expect(fs.access(bundlePath!)).resolves.toBeUndefined(); + await cleanupGeneratedBrowserBundles(result); + await expect(fs.access(path.dirname(bundlePath!))).rejects.toMatchObject({ code: "ENOENT" }); + } finally { + await fs.rm(tempDir, { recursive: true, force: true }); + } + }); }); diff --git a/tests/browser/sessionRunner.test.ts b/tests/browser/sessionRunner.test.ts index cd0864d39..1ea9ebf36 100644 --- a/tests/browser/sessionRunner.test.ts +++ b/tests/browser/sessionRunner.test.ts @@ -1,3 +1,6 @@ +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; import { describe, expect, test, vi } from "vitest"; import type { RunOracleOptions } from "../../src/oracle.js"; import type { BrowserSessionConfig } from "../../src/sessionStore.js"; @@ -750,14 +753,58 @@ describe("runBrowserSessionExecution", () => { ); expect(executeBrowser).toHaveBeenCalledWith( expect.objectContaining({ - fallbackSubmission: { + fallbackSubmission: expect.objectContaining({ prompt: "fallback prompt", attachments: [expect.objectContaining({ path: "/repo/a.txt", displayPath: "a.txt" })], - }, + prepare: expect.any(Function), + }), }), ); }); + test("removes generated browser bundles after execution even when the run fails", async () => { + const bundleDir = await fs.mkdtemp(path.join(os.tmpdir(), "oracle-browser-bundle-")); + const bundlePath = path.join(bundleDir, "attachments-bundle.zip"); + await fs.writeFile(bundlePath, "zip"); + const executeBrowser = vi.fn(async () => { + throw new Error("browser exploded"); + }); + + await expect( + runBrowserSessionExecution( + { + runOptions: baseRunOptions, + browserConfig: baseConfig, + cwd: "/repo", + log: vi.fn(), + }, + { + assemblePrompt: async () => ({ + markdown: "prompt", + composerText: "prompt", + estimatedInputTokens: 5, + attachments: [ + { + path: bundlePath, + displayPath: bundlePath, + sizeBytes: 3, + generatedBundle: true, + }, + ], + inlineFileCount: 0, + tokenEstimateIncludesInlineFiles: false, + attachmentsPolicy: "always", + attachmentMode: "bundle", + fallback: null, + }), + executeBrowser, + }, + ), + ).rejects.toThrow(/browser exploded/i); + + await expect(fs.access(bundleDir)).rejects.toMatchObject({ code: "ENOENT" }); + }); + test("respects verbose logging", async () => { const log = vi.fn(); await runBrowserSessionExecution( diff --git a/tests/remote/server.test.ts b/tests/remote/server.test.ts index 04b72df91..6708f6863 100644 --- a/tests/remote/server.test.ts +++ b/tests/remote/server.test.ts @@ -150,6 +150,59 @@ describe("remote browser service", () => { }, ); + test.skipIf(!CAN_LISTEN_LOCALHOST)( + "does not materialize a pending fallback bundle before the primary remote submit", + async () => { + const tmpDir = await mkdtemp(path.join(os.tmpdir(), "oracle-remote-lazy-fallback-")); + const fallbackPath = path.join(tmpDir, "fallback.txt"); + await writeFile(fallbackPath, "lazy fallback", "utf8"); + const prepare = async () => { + throw new Error("client prepare must not run for remote fallback"); + }; + + const server = await createRemoteServer( + { host: "127.0.0.1", port: 0, token: "secret", logger: () => {} }, + { + runBrowser: async (options) => { + expect(options.fallbackSubmission?.prompt).toBe("fallback prompt"); + expect(options.fallbackSubmission?.attachments).toHaveLength(1); + const stored = await readFile(options.fallbackSubmission!.attachments[0]!.path, "utf8"); + expect(stored).toBe("lazy fallback"); + expect(options.fallbackSubmission?.prepare).toEqual(expect.any(Function)); + return { + answerText: "ok", + answerMarkdown: "ok", + tookMs: 1, + answerTokens: 1, + answerChars: 2, + }; + }, + }, + ); + + try { + const executor = createRemoteBrowserExecutor({ + host: `127.0.0.1:${server.port}`, + token: "secret", + }); + const result = await executor({ + prompt: "remote", + fallbackSubmission: { + prompt: "fallback prompt", + attachments: [{ path: fallbackPath, displayPath: "fallback.txt", sizeBytes: 13 }], + prepare, + pendingBundle: { format: "text", scope: "text-only" }, + }, + config: {}, + }); + expect(result.answerText).toBe("ok"); + } finally { + await server.close(); + await rm(tmpDir, { recursive: true, force: true }); + } + }, + ); + test.skipIf(!CAN_LISTEN_LOCALHOST)( "stages colliding primary attachment names without losing payloads", async () => {