test(core): Add editor-only Docker build and local proxy for faster testing (no-changelog)#28569
Closed
test(core): Add editor-only Docker build and local proxy for faster testing (no-changelog)#28569
Conversation
Add canExecute prop to WorkflowPreview, include as query param on iframe URL. DemoLayout conditionally establishes a WebSocket push connection when canExecute=true. Refactor importWorkflowExact to no longer override workflow ID to 'demo' — the ID override is moved to callers in usePostMessageHandler so the real workflow ID flows through when execution is enabled. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update canExecuteOnCanvas to return true on demo route when canExecute=true query param is present. Add canExecute prop to Canvas and WorkflowCanvas, thread from NodeView. Add ctrl+enter shortcut to read-only+executable keymap. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Thread canExecute prop through Canvas → CanvasNode → CanvasNodeToolbar. Execute button visible when readOnly && canExecute. Delete, disable, and other toolbar buttons remain hidden in read-only mode. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When NDV opens in read-only mode and the node has execution data, default to the Output tab instead of Parameters. Avoids a flash of disabled parameters before switching. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…utton Show run button as disabled with tooltip when workflow has only a Chat Trigger in the preview context. Error display and concurrent execution handling work automatically via existing canvas node badges and iframe executionStarted handler. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove type icons from tab labels, make tabs fill full header height, and replace loading spinners with larger loader-circle icon (80px). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…g (no-changelog) Adds a Playwright e2e test that captures the bug where the last node in the instance AI workflow preview stays in "running" state (spinning border) after execution completes. The test sends a specific prompt to build and execute a 3-node workflow, then asserts that no canvas nodes remain with the .running CSS class. Includes InstanceAiPage page object, navigation helper, and test fixtures with N8N_ENABLED_MODULES=instance-ai. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ionFinished (no-changelog) The event relay watcher only forwarded the last event in the log, so when Vue coalesced multiple ref updates into one callback, intermediate events (e.g. nodeExecuteAfter for the last node) were silently dropped. This left the iframe's executing-node queue with a stale entry, keeping the last node in spinning/running state after the workflow finished. - Track relayed event count so every new event is forwarded, even when the watcher fires once for multiple log additions. - Keep the eventLog intact when executionFinished arrives (instead of clearing it immediately) so the relay can forward pending events before sending the synthetic executionFinished. - Add clearEventLog() to useExecutionPushEvents, called by the relay after all pending events have been forwarded. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add 16 Playwright e2e tests across 6 spec files covering instance AI workflow preview, artifacts, timeline, sidebar, confirmations, and chat basics. Wire up proxy-aware fetch in the AI SDK model creation so MockServer can intercept Anthropic API calls for recording/replay. - Expand InstanceAiPage page object with 30+ locators - Add InstanceAiSidebar component page object - Add data-test-id to preview close button - Add getProxyFetch() to model-factory.ts and instance-ai.service.ts so @ai-sdk/anthropic respects HTTP_PROXY in e2e containers - Rewrite fixtures with proxy service recording support - Replace single execution-state test with comprehensive suite Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…hangelog) Add two-tier trace replay system that records tool I/O during e2e test recording and replays with bidirectional ID remapping in CI. This enables deterministic replay of complex multi-step agent tests where tool execution produces dynamic IDs. - New trace-replay.ts: IdRemapper (ID-field-aware), TraceIndex (per-role cursors), TraceWriter, JSONL I/O helpers, PURE_REPLAY_TOOLS set - Modified langsmith-tracing.ts: replayWrapTool (Tier 1: real execution + ID remap), pureReplayWrapTool (Tier 2: pure replay for external deps), recordWrapTool, createTraceReplayOnlyContext stub for non-LangSmith envs - New test-only controller endpoints: POST/GET/DELETE /test/tool-trace with slug-scoped storage for parallel test isolation - Updated fixture: records trace.jsonl during recording, loads for replay, slug-scoped activate/retrieve/clear lifecycle - 23 unit tests for IdRemapper and TraceIndex - Recorded trace.jsonl files for all 15 instance AI test expectations Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…log) Add subString body matching on the system prompt to disambiguate LLM call types (title generation vs orchestrator vs sub-agent) during proxy replay. Without this, sequential expectations could be served to the wrong call when the call order differs between recording and replay. Re-record all expectations with the body matcher and remove debug logging from trace replay wrappers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When re-recording with a real API key, always use record mode (never load old trace events into the backend). Previously, existing trace files would cause the backend to enter replay mode during re-recording, resulting in trace.jsonl files with only a header and no tool calls. Re-record all trace.jsonl files with proper tool call events. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Re-record all proxy expectations after fixing the recording mode logic. Expectations now have subString body matchers on the system prompt and trace.jsonl files have proper tool call events. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The proxy's sequential mode sets the last expectation as unlimited (fallback for extra agent turns). Previously this applied to the last file alphabetically which could be a community_nodes GET. Now it finds the last /v1/messages POST expectation specifically. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…elog) Background task completion triggers `startInternalFollowUpRun`, which creates a new trace context. Previously each context got a fresh TraceIndex with cursor at 0, so the follow-up run's first tool call (e.g. list-workflows) would mismatch the first trace event (build-workflow-with-agent) and throw. Fix: store a shared TraceIndex/IdRemapper per test slug on the service. All runs within the same slug reuse the same instances, preserving cursor state across the initial run and any follow-up runs. This fixes the two confirmation e2e tests that rely on suspend/resume. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…(no-changelog) waitForAssistantResponse only waited for the first message element to appear (streaming start), not for the agent to finish. Sidebar operations then raced against the still-running agent. New waitForResponseComplete waits for the send button to reappear, which only renders when isStreaming becomes false. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…s between tests (no-changelog) Two preview tests failed because their recorded proxy expectations contained stale LLM responses from previous tests' background task follow-ups. The fixture now cancels leftover background tasks before each test via a new test-only endpoint, preventing future cross-test contamination. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ET (no-changelog) MockServer proxy connections intermittently reset when 4 parallel workers load expectations simultaneously. Add withRetry helper with exponential backoff (3 retries, 500ms base) and re-throw on failure instead of silently swallowing the error. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ation (no-changelog) Positional selectors (.last()) break when parallel tests create threads in shared containers. Switch to getThreadByTitle() with LLM-generated titles from recordings. Also handle missing expectations directories gracefully. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The instance AI workflow preview iframe needs its own push connection to receive execution events when the user runs a workflow from the preview. Three issues were preventing this from working: 1. The iframe shared sessionStorage with the parent page (same origin), so both got the same pushRef. When the iframe's WebSocket connected, the backend replaced the parent's connection — and vice versa — meaning execution completion events were lost. 2. The pushConnection store's URL was captured once at store creation, so any pushRef change after store creation had no effect. 3. DemoLayout eagerly set activeExecutionId to null at mount time, but isWorkflowRunning treats null as "execution starting", which left the run button disabled with a loading spinner before the user could click. Fixed by generating a unique pushRef when DemoLayout detects it's running in an iframe, making the push connection URL reactive, and only setting activeExecutionId to null when the iframe is in read-only relay mode. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…angelog) Add comprehensive Playwright tests covering manual execution from the instance AI workflow preview iframe: - Run button visibility after preview opens - Full workflow execution with success indicators - Per-node execution from the canvas node toolbar - NDV output panel display after execution - Re-running a workflow after a successful first run Adds InstanceAiPage page object methods for interacting with execution UI inside the preview iframe (run button, per-node execute, success indicators, NDV output). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Strip WebSocket diagnostic logging that was used during development to diagnose the iframe push connection bug. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nExecute When the iframe receives a workflow via postMessage, the post-import handler re-set activeExecutionId to null so the iframe stays receptive to push events relayed from the parent. But isWorkflowRunning treats null as "execution starting", which left the run button disabled with a loading state when canExecute was enabled. Apply the same canExecute check we added to DemoLayout — only re-set activeExecutionId to null in read-only relay mode, not when the user can trigger executions directly. Also fix the NDV output panel selector (output-panel, not ndv-output-panel) which is the wrapper data-test-id used by NodeDetailsViewV2. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… (no-changelog) Re-record proxy expectations for the instance AI workflow execution tests so they can replay deterministically in CI without hitting the real Anthropic API. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…review In executable preview mode, workflows with only a Chat Trigger no longer show a disabled execute button with a tooltip. Instead the button is hidden entirely and the "Open chat" button opens the logs panel so users can interact with the workflow via chat. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
# Conflicts: # packages/@n8n/agents/src/runtime/model-factory.ts # packages/@n8n/instance-ai/src/tracing/__tests__/trace-replay.test.ts # packages/@n8n/instance-ai/src/tracing/langsmith-tracing.ts # packages/@n8n/instance-ai/src/tracing/trace-replay.ts # packages/@n8n/instance-ai/src/types.ts # packages/cli/src/modules/instance-ai/instance-ai.service.ts # packages/frontend/editor-ui/src/features/ai/instanceAi/__tests__/useEventRelay.test.ts # packages/frontend/editor-ui/src/features/ai/instanceAi/useEventRelay.ts # packages/testing/containers/services/proxy.ts # packages/testing/playwright/expectations/instance-ai/should-auto-open-preview-panel-when-workflow-is-built/trace.jsonl # packages/testing/playwright/expectations/instance-ai/should-close-preview-panel-via-close-button/trace.jsonl # packages/testing/playwright/expectations/instance-ai/should-display-canvas-nodes-in-preview-iframe/trace.jsonl # packages/testing/playwright/helpers/NavigationHelper.ts # packages/testing/playwright/pages/InstanceAiPage.ts # packages/testing/playwright/tests/e2e/instance-ai/fixtures.ts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…gelog) These additions (requireInstanceAiEnabled guard, message validation) were already moved to a different location on this branch. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…on (no-changelog) The disabled-reason tooltip on the run button is no longer needed — chat trigger workflows now hide the button entirely and show Open chat instead. Also fix duplicate IconLucideLoaderCircle import in design-system icons. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- useChatState: stop watch when push connection times out to prevent dangling listeners - usePostMessageHandler.test: make window.parent mock configurable for safe restoration - NodeSettings: re-evaluate default tab when active node changes (was one-time snapshot) - instance-ai-workflow-execution test: wait for re-execution to actually run before asserting - InstanceAiPage: restore preview interaction methods lost during master merge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…esting (no-changelog) Two independent improvements to speed up instance-AI Playwright testing: 1. `pnpm build:docker:editor` — Rebuilds only editor-ui and patches the existing compiled/ directory, then re-dockerizes. Skips the full monorepo build for frontend-only iterations. 2. `pnpm test:local:instance-ai --proxy` — Starts a standalone MockServer container alongside the local n8n process, enabling record/replay of Anthropic API calls without the full container stack. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Contributor
Performance ComparisonComparing current → latest master → 14-day baseline Memory consumption baseline with starter plan resources
docker-stats
Idle baseline with Instance AI module loaded
How to read this table
|
…ort (no-changelog) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
No issues found across 7 files
Architecture diagram
sequenceDiagram
participant Dev as Developer
participant Runner as Test Runner (run-local-instance-ai)
participant Docker as MockServer Container
participant n8n as Local n8n Process
participant PW as Playwright
participant Ext as Anthropic API
Note over Dev,Ext: NEW: Local Instance-AI Test Flow with --proxy
Dev->>Runner: pnpm test:local:instance-ai --proxy
Runner->>Docker: NEW: docker run -d (MockServer)
Docker-->>Runner: Mapped Port (proxyUrl)
loop NEW: waitForProxy
Runner->>Docker: curl /mockserver/status
end
Runner->>n8n: CHANGED: Start with HTTP_PROXY=proxyUrl
Note over n8n: Sets NODE_TLS_REJECT_UNAUTHORIZED=0
Runner->>PW: Start Tests (env: PROXY_SERVER_URL)
PW->>n8n: Trigger AI Workflow
n8n->>Docker: NEW: Intercept Outbound Request
alt NEW: Recording Mode (ANTHROPIC_API_KEY set)
Docker->>Ext: Forward to real API
Ext-->>Docker: 200 OK
Docker->>Docker: Persist expectation to disk
else NEW: Replay Mode (No API Key)
Docker->>Docker: Match request against disk cache
end
Docker-->>n8n: Return proxied response
n8n-->>PW: Success
PW-->>Runner: Test Completion
Runner->>Docker: NEW: docker rm -f (Cleanup on Exit/SIGINT)
…hangelog) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…elog) registerChatWebhook now waits for push connection before proceeding. Without this mock, tests timeout waiting for isConnected to become true. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…log) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bundle ReportChanges will increase total bundle size by 2.91kB (0.01%) ⬆️. This is within the configured threshold ✅ Detailed changes
Affected Assets, Files, and Routes:view changes for bundle: editor-ui-esmAssets Changed:
Files in
Files in
Files in
Files in
Files in
Files in
Files in
Files in
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two independent improvements to speed up instance-AI Playwright testing iteration:
1.
pnpm build:docker:editor— Editor-only Docker rebuildRebuilds only
editor-ui(and its workspace deps like design-system, i18n) and patches the existingcompiled/directory, then re-dockerizes. Skips the full monorepo build andpnpm deploystep.pnpm build:dockerto establish thecompiled/directory2.
pnpm test:local:instance-ai --proxy— Record/replay against local serverStarts a standalone MockServer Docker container alongside the local n8n process, enabling record/replay of Anthropic API calls without the full container stack.
--proxyflag starts MockServer, wiresHTTP_PROXY/HTTPS_PROXYto n8n, passesPROXY_SERVER_URLto PlaywrightANTHROPIC_API_KEY=sk-ant-... pnpm test:local:instance-ai --proxypnpm test:local:instance-ai --proxy--proxyis unchangedFiles changed
scripts/build-docker-editor.mjspackage.jsonbuild:docker:editorscriptpackages/testing/playwright/scripts/run-local-instance-ai.mjs--proxyflag, MockServer lifecyclepackages/testing/playwright/scripts/run-local-isolated.mjsPROXY_SERVER_URLpackages/testing/playwright/fixtures/base.tsservicesfixturepackages/testing/playwright/tests/e2e/instance-ai/fixtures.tspackages/testing/containers/index.tsProxyServerRelated Linear tickets, Github issues, and Community forum posts
Review / Merge checklist
Backport to Beta,Backport to Stable, orBackport to v1(if the PR is an urgent fix that needs to be backported)🤖 PR Summary generated by AI