From 758528e4bb2c4e50b508776c6521ab90dfa8eb3d Mon Sep 17 00:00:00 2001 From: seungwon Lee Date: Tue, 14 Apr 2026 11:33:43 +0900 Subject: [PATCH 1/2] feat(FR-2562): expand e2e-healer to run full test suite and enable healing Changes to e2e-healer.md: - Replace single-file test (login.spec.ts) with full E2E suite (--grep-invert @visual) as originally intended - Enable the healing phase (previously commented out) that: - Creates branch e2e-healer/- from main - Uses Playwright planner to analyze and fix failures - Opens draft PR with fix if tests pass - Adds diagnostic comment if fix fails - Only attempts healing for WebUI-caused failures (not Backend/Server) - Increase timeout from 120 to 180 minutes for full suite + healing Note: .lock.yml recompilation required via `gh aw compile` after merge. Resolves FR-2562 --- .github/workflows/e2e-healer.md | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/.github/workflows/e2e-healer.md b/.github/workflows/e2e-healer.md index 0b4ec662bd..bf00f6b737 100644 --- a/.github/workflows/e2e-healer.md +++ b/.github/workflows/e2e-healer.md @@ -14,7 +14,7 @@ on: permissions: read-all -timeout-minutes: 120 +timeout-minutes: 180 engine: copilot @@ -84,9 +84,7 @@ steps: - name: Run E2E tests id: e2e-tests continue-on-error: true - # TODO: Remove single test restriction after environment validation - # Full test: pnpm playwright test e2e/ --grep-invert @visual --reporter=html,json --output=test-results - run: pnpm playwright test e2e/auth/login.spec.ts --reporter=html,json --output=test-results + run: pnpm playwright test e2e/ --grep-invert @visual --reporter=html,json --output=test-results - name: Save test results run: | mkdir -p /tmp/gh-aw/e2e-results @@ -120,6 +118,7 @@ You are an AI ops engineer for `${{ github.repository }}`. Run weekday Playwrigh - E2E tests have already been executed in the `steps` phase before the agent starts. - Test results are available at `/tmp/gh-aw/e2e-results/` directory. - Tests run against the deployed endpoint: `E2E_WEBUI_ENDPOINT` (set via repository variables). +- The full E2E test suite runs (excluding `@visual` tests), not just a single test file. ## CRITICAL: Secret Protection Rules **NEVER include any of the following in issues, PRs, comments, or logs:** @@ -150,14 +149,13 @@ You are an AI ops engineer for `${{ github.repository }}`. Run weekday Playwrigh - Related PRs if applicable (link to PR numbers) - Repro command - Attach key log snippets; avoid uploading large artifacts directly to the issue body. -4) Healing attempt (temporarily disabled; uncomment when ready): - +4) Healing attempt: + - Only attempt healing for failures categorized as **WebUI change** (not Backend or Server issues). + - Scope fixes to failing specs only. Avoid touching snapshots unless required. + - Create branch from `main` named `e2e-healer/-`. + - Use Playwright planner to reason about fixes, edit code, and rerun only the previously failing specs, then the full `pnpm playwright test e2e/ --grep-invert @visual` if fixes look stable. + - If fixes pass, use `safe-outputs.create-pull-request` to open a **draft** PR against `main` with title prefix `e2e-healer:`. Reference the issue, include failing cases, applied changes, and rerun results. Exclude generated reports/artifacts from the branch. + - If not fixed, leave a concise comment on the issue (via `safe-outputs.add-comment`) with diagnostics and next hints. 5) Housekeeping: keep diffs minimal, avoid unrelated refactors, and ensure branch/PR cleanup instructions are clear in the PR body. ## Output expectations From 0edc6babcfee75740c10e4fab21cc5e278c56085 Mon Sep 17 00:00:00 2001 From: seungwon Lee Date: Tue, 14 Apr 2026 15:25:05 +0900 Subject: [PATCH 2/2] feat(FR-2562): manually patch healer lock.yml with full suite and healing phase Manually patch e2e-healer.lock.yml (cannot recompile with gh-aw v0.68.1 due to secrets pattern incompatibility): - Run full E2E suite (e2e/ --grep-invert @visual) instead of login.spec.ts - Enable healing phase with full instructions in agent prompt - Increase agent timeout from 120 to 180 minutes Resolves FR-2562 --- .github/workflows/e2e-healer.lock.yml | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/.github/workflows/e2e-healer.lock.yml b/.github/workflows/e2e-healer.lock.yml index cb3a0acda2..ee489a3ea5 100644 --- a/.github/workflows/e2e-healer.lock.yml +++ b/.github/workflows/e2e-healer.lock.yml @@ -130,7 +130,7 @@ jobs: - continue-on-error: true id: e2e-tests name: Run E2E tests - run: pnpm playwright test e2e/auth/login.spec.ts --reporter=html,json --output=test-results + run: pnpm playwright test e2e/ --grep-invert @visual --reporter=html,json --output=test-results - name: Save test results run: | mkdir -p /tmp/gh-aw/e2e-results @@ -631,8 +631,13 @@ jobs: - Related PRs if applicable (link to PR numbers) - Repro command - Attach key log snippets; avoid uploading large artifacts directly to the issue body. - 4) Healing attempt (temporarily disabled; uncomment when ready): - + 4) Healing attempt: + - Only attempt healing for failures categorized as **WebUI change** (not Backend or Server issues). + - Scope fixes to failing specs only. Avoid touching snapshots unless required. + - Create branch from `main` named `e2e-healer/-`. + - Use Playwright planner to reason about fixes, edit code, and rerun only the previously failing specs, then the full `pnpm playwright test e2e/ --grep-invert @visual` if fixes look stable. + - If fixes pass, use `safe-outputs.create-pull-request` to open a **draft** PR against `main` with title prefix `e2e-healer:`. Reference the issue, include failing cases, applied changes, and rerun results. Exclude generated reports/artifacts from the branch. + - If not fixed, leave a concise comment on the issue (via `safe-outputs.add-comment`) with diagnostics and next hints. 5) Housekeeping: keep diffs minimal, avoid unrelated refactors, and ensure branch/PR cleanup instructions are clear in the PR body. ## Output expectations @@ -811,7 +816,7 @@ jobs: # --allow-tool shell(wc) # --allow-tool shell(yq) # --allow-tool write - timeout-minutes: 120 + timeout-minutes: 180 run: | set -o pipefail sudo -E awf --env-all --container-workdir "${GITHUB_WORKSPACE}" --mount /tmp:/tmp:rw --mount "${GITHUB_WORKSPACE}:${GITHUB_WORKSPACE}:rw" --mount /usr/bin/date:/usr/bin/date:ro --mount /usr/bin/gh:/usr/bin/gh:ro --mount /usr/bin/yq:/usr/bin/yq:ro --mount /usr/local/bin/copilot:/usr/local/bin/copilot:ro --mount /home/runner/.copilot:/home/runner/.copilot:rw --allow-domains api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.npms.io,bun.sh,cdn.playwright.dev,deb.nodesource.com,deno.land,get.pnpm.io,github.com,host.docker.internal,nodejs.org,npm,npm.pkg.github.com,npmjs.com,npmjs.org,playwright.download.prss.microsoft.com,raw.githubusercontent.com,registry.bower.io,registry.npmjs.com,registry.npmjs.org,registry.yarnpkg.com,repo.yarnpkg.com,skimdb.npmjs.com,www.npmjs.com,www.npmjs.org,yarnpkg.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --image-tag 0.7.0 \