Skip to content

feat: Guided Tours E2E Tests#2415

Draft
camielvs wants to merge 1 commit into
06-11-feat_guided_tours_completion_metricsfrom
06-12-feat_guided_tours_e2e_tests
Draft

feat: Guided Tours E2E Tests#2415
camielvs wants to merge 1 commit into
06-11-feat_guided_tours_completion_metricsfrom
06-12-feat_guided_tours_e2e_tests

Conversation

@camielvs

@camielvs camielvs commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR is 100% AI generated.

End-to-end tests that replicate and verify the exact flow of every guided tour, so a UI change that breaks a tour is caught here (and points at the tour + step to fix) rather than shipping a broken tour.

Rather than four bespoke specs, this is a single data-driven driver that reads each tour's real JSON definition and walks it the way a user would:

  • tests/e2e/tours/tourDriver.tsrunTour(page, tour) opens the tour, then for each step: asserts the step's spotlight anchor resolves in the DOM (the core regression signal), performs the gating interaction (or clicks Next for passive steps), and waits for the tour to advance (synchronised on the ?step=N URL the tour writes). It finishes on the completion step. A handler exists for every interaction type — drag-from-library, edge connection, task select, set-argument, undock/redock window, subgraph navigate/unpack, marquee multi-select, create-subgraph, and the full secrets flow.
  • Specs — one thin spec per tour: navigating-editor, first-pipeline, subgraphs, using-secrets. Each loads the shipped tour JSON via loadTour(...), so the test tracks the real tour and new steps are covered automatically (modulo a brand-new interaction type needing a handler).
  • eslint.config.js — teach playwright/expect-expect that runTour asserts internally.

Because the driver asserts each step's data-tour* anchor, removing or renaming an anchor a tour depends on fails the matching step by design.

The using-secrets spec exercises the no-backend path from #2405 / #2406: it fails the health ping and /api/secrets so the tour's in-memory mock backend serves the secret steps. The tour runs fully hands-on with normal gating, no /api/secrets calls fire, and the Submit Run confirm is asserted disabled (real run not wired in mock mode).

Related Issue and Pull requests

Closes https://github.com/Shopify/oasis-frontend/issues/644

Sits on top of the guided-tour stack (#2405 / #2406 and the framework/tour PRs below them).

Type of Change

  • New feature

Checklist

  • I have tested this does not break current pipelines / runs functionality
  • I have tested the changes on staging

Test Instructions

  • pnpm test:e2e tests/e2e/tours/ — all four tour specs pass (run in parallel against a local dev server).
  • The using-secrets spec drives the no-backend path: it routes the health ping and /api/secrets to fail, so the tour's mockBackend activates and the in-memory mock serves the secret steps. It asserts no /api/secrets calls fire and that Submit Run stays disabled in mock mode.

Additional Comments

The centered tour popover overlaps a few dialog controls (Add Secret / submit confirm / the "Use Secret" lock button). The driver dispatches the click directly to those elements (clickThrough) so React's handler still fires — the only spot where it doesn't use a real mouse click.

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown

🎩 Preview

A preview build has been created at: 06-12-feat_guided_tours_e2e_tests/fb6be72

camielvs commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator Author

@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 586e87d to 6bffb2c Compare June 12, 2026 23:36
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from f7adb0c to f9ccfa0 Compare June 12, 2026 23:36
@camielvs camielvs mentioned this pull request Jun 13, 2026
3 tasks
const input = row.getByTestId("argument-input");
await expect(input).toBeVisible();
await input.click();
await input.fill("tips");

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 This is an AI-generated code review comment.

performInteraction is the shared, tour-agnostic engine, but this set-argument branch hardcodes "tips" — a value meaningful only to the first-pipeline tour's dataset argument. A future tour that uses set-argument for a different field would silently get "tips" typed in and likely fail the step's gating check for a non-obvious reason.

Suggest carrying the value on the step (e.g. a targetArgumentValue field on TourStepDef, read here with a fallback) so the driver stays generic — or at minimum a comment noting this value is first-pipeline-specific. Low priority.

targetPortName?: string;
}

interface TourStepDef {

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 This is an AI-generated code review comment.

This file is new here, so flagging: loadTour's comment (line 8) sells the design as drift-proof by reading the real shipped JSON — but the type here (TourStepDef / TourDef, with the interaction and target* fields) is still a hand-maintained copy of registry.ts's TourStep / TourDefinition, and JSON.parse(...) as TourDef (line 15) casts past any check. If the app renames a field, the driver type won't flag it (typescript-standards — prefer real types over as).

It's only Low because drift surfaces as a runtime test failure (an undefined selector/target) rather than silently-wrong behavior — so it's self-catching, just less precise than a compile error. Consider importing TourStep / TourDefinition from @/components/Learn/tours/registry and deriving the JSON-shaped subset, so a field rename becomes a type error right here.

@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from f9ccfa0 to c1361f2 Compare June 15, 2026 17:55
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 6bffb2c to 18c2dd3 Compare June 15, 2026 17:55
@camielvs camielvs mentioned this pull request Jun 15, 2026
8 tasks
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 18c2dd3 to 0e1b241 Compare June 15, 2026 18:47
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch 2 times, most recently from 088f063 to ef6a2d0 Compare June 15, 2026 18:59
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 0e1b241 to 7f552ac Compare June 15, 2026 18:59
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from ef6a2d0 to b583f84 Compare June 15, 2026 19:21
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch 2 times, most recently from 3b6725f to cf9670d Compare June 15, 2026 19:32
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from b583f84 to cf0a129 Compare June 15, 2026 19:32
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from cf9670d to 9a1a38d Compare June 15, 2026 19:49
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch 2 times, most recently from 03beccf to d725087 Compare June 15, 2026 20:13
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 9a1a38d to ad996b1 Compare June 15, 2026 20:13
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from d725087 to 94310c5 Compare June 15, 2026 22:34
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 1e6db6b to d8b7cf4 Compare June 15, 2026 22:36
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from 94310c5 to 515a226 Compare June 15, 2026 22:36
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from d8b7cf4 to b8c7156 Compare June 15, 2026 22:55
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from 515a226 to e2d29d7 Compare June 15, 2026 22:55
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from b8c7156 to 4d74024 Compare June 16, 2026 00:38
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from e2d29d7 to 6e6b6d8 Compare June 16, 2026 00:38
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 4d74024 to 9954dd1 Compare June 16, 2026 00:58
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from 6e6b6d8 to 5662663 Compare June 16, 2026 00:58
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 9954dd1 to 68c9054 Compare June 16, 2026 01:15
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from 5662663 to 173e3d0 Compare June 16, 2026 01:15
Data-driven driver that walks each tour's JSON: asserts every step's
data-tour anchor, performs the gating interaction (or clicks Next), and
verifies advancement, finishing on the completion step. One thin spec per
tour (navigating-editor, first-pipeline, subgraphs, using-secrets).

The using-secrets spec drives the no-backend path: it fails the health ping
and /api/secrets so the tour's in-memory mock backend handles the secret
steps, runs them fully hands-on with normal gating, and asserts the Submit
Run confirm stays disabled in mock mode.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from 173e3d0 to fb6be72 Compare June 16, 2026 01:26
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 68c9054 to dc367c2 Compare June 16, 2026 01:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

#gsd:50583 Learning Hub

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant