From 33943f264247e62225a3e03a6a3fbb1a1a9c278b Mon Sep 17 00:00:00 2001 From: HipsterBrown Date: Fri, 8 May 2026 14:04:11 -0400 Subject: [PATCH 1/9] docs(spec): pipeline view design A new /pipeline route that auto-derives the perception pipeline from the machine config, renders it as a DAG, and spotlights the selected stage (or stages) in the existing 3D scene with per-stage tinting. Reuses the existing Threlte canvas, ECS world, hooks, and PCD worker; the only modification to shared code is an optional resource filter on providePointclouds / providePointcloudObjects. --- .../specs/2026-05-08-pipeline-view-design.md | 353 ++++++++++++++++++ 1 file changed, 353 insertions(+) create mode 100644 docs/superpowers/specs/2026-05-08-pipeline-view-design.md diff --git a/docs/superpowers/specs/2026-05-08-pipeline-view-design.md b/docs/superpowers/specs/2026-05-08-pipeline-view-design.md new file mode 100644 index 000000000..d6bdad99f --- /dev/null +++ b/docs/superpowers/specs/2026-05-08-pipeline-view-design.md @@ -0,0 +1,353 @@ +# Pipeline View — Design + +**Status:** Draft +**Date:** 2026-05-08 +**Author:** nick.hehr@viam.com +**Branch:** `feat/pipeline-view` + +## Purpose + +The visualization app currently renders every camera point cloud, every vision-service object cloud, and every framed component in one shared scene with no notion of how they relate. There is no way to trace data from a source camera through transforms and joins to a final segmenter output, and no way to compare the output of two stages of the same pipeline against each other. + +This feature adds a **Pipeline view** at `/pipeline` that: + +- Auto-derives the perception pipeline from the machine config (with a manual override escape hatch). +- Renders the pipeline as a DAG in a left-pane graph panel. +- "Spotlights" the selected stage (or stages) in the existing 3D scene, hiding non-active stages. +- Shows 2D outputs (images, detections, classifications) in a right-pane preview when the active stage is non-3D. +- Tints each active stage in compare mode so multi-stage overlays are visually distinct. + +The user value: identify which stage of a perception pipeline is misbehaving, with the source-to-output relationship made visible, so config tuning has a fast feedback loop against a live machine. + +## Goals + +- **Live monitor flavored.** Stages re-render against fresh data on the existing refresh cadence; a config edit on the live machine flows through. +- **No new "pipeline file" to author.** Auto-derivation from machine config is the default; users only touch override settings when the auto-derivation can't see something. +- **Reuse the existing canvas, ECS world, hooks, and PCD worker.** Pipeline view is a new route, not a fork of the app. +- **Minimal change surface in shared code.** The only modification to existing hooks is an optional resource filter — no breaking changes for the main app. +- **All four output kinds in v1:** point clouds, segmenter object clouds, 2D images (multi-named), detections / classifications. + +## Non-Goals + +- **Editing machine config from the pipeline view.** Config remains a Viam-platform concern; pipeline view is read-and-visualize. +- **Server-side pipeline graph.** Derivation is purely client-side; no Connect-RPC additions for v1. +- **`getPipelineMetadata` DoCommand convention.** Worthwhile follow-up for custom modules but not v1. +- **Multi-part pipelines spanning machines.** A pipeline lives within one part. +- **Per-stage refresh rates.** Existing `RefreshRates` settings apply to all stages of a kind. Per-stage cadence is a follow-up. +- **Snapshot-mode pipeline replay.** The existing `/snapshot` route handles frozen scenes; pipeline view is live. +- **Frame-system `transformPCD` alignment.** Existing entities are placed via parent frames already; revisit only if compare-mode alignment surfaces problems in practice. +- **Replacing or modifying the main app's UX.** Pipeline view is opt-in via a separate route. + +## Decisions + +| # | Decision | Rationale | +|---|---|---| +| P1 | New route at `/pipeline`; trimmed shell (`Settings` + `RefreshRate` overlays kept; tree, dashboard, XR, etc. dropped) | Focused UX for a focused use case; main app unchanged | +| P2 | Auto-derive pipeline graph from machine config + manual override (localStorage per `partID`) | Common case is free; escape hatch for custom modules | +| P3 | Edges discovered by attribute-**value** matching against known resource names (not by attribute-key heuristics) | Robust to varied attribute naming (`src`, `camera`, `cameras`, `service`, `detector`, `vision`, etc.); custom modules covered automatically | +| P4 | Output kinds discovered via runtime `getProperties()` (cached via existing TanStack pattern), not from model-name guessing | Authoritative; aligns with how `usePointclouds` / `usePointcloudObjects` already gate fetching | +| P5 | Spotlight scene paradigm (one shared ``) — graph picks active stages; non-active hidden via `Invisible` trait | Reuses existing entity renderer; no multi-canvas complexity | +| P6 | Per-stage color tint via new `PipelineTint` trait + header toggle to restore native RGB | Compare mode legibility; user-controllable when colors themselves matter | +| P7 | 2D preview in a right-pane `StagePreview2D`, mounted only when the active set contains a 2D-only output | Doesn't take space when not needed | +| P8 | 2D image stages use `getImages()` (multi-named-image) with stream fallback when supported | Aligns with current camera API; reuses existing `Camera.svelte` stream logic where applicable | +| P9 | Optional `filter` parameter added to `providePointclouds` / `providePointcloudObjects`; defaults to undefined | Pipeline route only fetches stages it shows; main app behavior unchanged | +| P10 | `usePipeline` Svelte context owns graph, active set, status, tint, preview-2D-stage; both graph panel and scene read from it | Single source of truth; predictable reactivity | +| P11 | Cycle detection drops the back-edge with a warning on the offending node | Pipeline view stays usable even with broken configs | +| P12 | All persistence (overrides, preview pane width, native-RGB toggle) keyed by `partID` in `localStorage` | Per-machine state without backend changes | + +## Architecture + +### Route shape + +``` +/pipeline +├─ providers shell: provideWorld, provideSettings, provideEnvironment, +│ providePartConfig, providePipeline +├─ +│ ├─ (grid, camera controls, environment lights — existing) +│ └─ (ECS-driven entity rendering — existing) +└─ overlays + ├─ (left, ~280px, resizable; new) + ├─ (right; mounts when preview2D !== undefined; new) + ├─ (existing component) + └─ (existing component, with new "Pipeline overrides" tab) +``` + +The page lives at `src/routes/pipeline/+page.svelte` and is a composition-only file. All meaningful code is library code under `src/lib/pipeline/`. + +### Module map + +``` +src/lib/pipeline/ + types.ts # Stage, Edge, StageKind, StageOutput, PipelineGraph, RGB + derive.ts # buildGraph(config: Struct, properties: PropertiesMap): PipelineGraph + override.ts # mergeOverrides(graph, overrides): PipelineGraph + tint.ts # tintForStageId(id: string): RGB + usePipeline.svelte.ts # context provider: graph, active, preview2D, tint, status, solo/toggle/clear + PipelineGraph.svelte # left-pane DAG renderer (FloatingPanel-style; resizable) + StageNode.svelte # one node: icon, label, status dot, tint chip, eye toggle + EdgeLayer.svelte # SVG edges between StageNodes (computed from layout) + StagePreview2D.svelte # right-pane image / detection / classification renderer + PipelineOverrides.svelte # modal/tab for overrides (visibility, outputs, manual edges) + +src/lib/pipeline/__tests__/ + derive.spec.ts + override.spec.ts + tint.spec.ts + +src/routes/pipeline/ + +page.svelte # composition only + +page.ts # part selection from URL params + +src/lib/hooks/ + usePointclouds.svelte.ts # MODIFIED — accept optional filter + usePointcloudObjects.svelte.ts # MODIFIED — accept optional filter +``` + +### Component responsibilities + +**`derive.ts`** — Pure function from `(config: Struct, properties: PropertiesMap) → PipelineGraph`. + +1. Collect `resourceNames` from `config.components[].name ∪ config.services[].name`. +2. For each component / service whose API is `camera` or `vision`, build a candidate `Stage`. Set `outputs` from the cached `properties` map: cameras → `pointcloud` iff `supportsPcd`, `image` iff `mimeTypes.length > 0`; vision services → union of `{detections, classifications, objects}` flagged by `getProperties`. +3. Stages with no relevant outputs are dropped. +4. For each candidate, recursively walk its `attributes` Struct. Every string-valued leaf and every string array element is checked against `resourceNames`. Each match (other than self) yields a candidate edge `match → candidate`. Dedupe. +5. Topologically sort; if cyclic, drop the latest-discovered back-edge and append a warning. +6. Return `{ stages: Stage[], edges: Edge[], warnings: Warning[] }`. + +**`override.ts`** — Merges user overrides into the auto-derived graph: + +```ts +interface Overrides { + hidden: Set // exclude from graph entirely + outputsByStage: Map // pin outputs (replace discovered) + extraEdges: Edge[] // user-added edges +} +``` + +Validates that referenced stage ids still exist in the derived graph and silently drops stale entries (the override-panel UI surfaces them as "unknown — remove"). + +**`tint.ts`** — `tintForStageId(id) → {r,g,b}`. Hash the id, map to evenly-distributed hue in HSL, fixed S/L. Deterministic so the same stage always gets the same color across reloads. + +**`usePipeline.svelte.ts`** — Context provider, called by `+page.svelte`: + +```ts +interface PipelineContext { + graph(): PipelineGraph + active(): ReadonlySet + preview2D(): StageId | undefined + tint(id: StageId): RGB + status(id: StageId): 'idle' | 'loading' | 'ok' | 'error' + solo(id: StageId): void + toggle(id: StageId): void // shift-click; flips membership in active + clear(): void +} +``` + +Internally, `graph` is `$derived` from `usePartConfig` + per-stage `getProperties` queries + overrides. `active` is `$state` (a `SvelteSet`). `preview2D` is `$derived` from `active`: the most-recently-toggled active stage whose outputs include any of `image | detections | classifications` (and whose outputs do **not** include `pointcloud | objects`, OR whose `pointcloud`/`objects` is hidden via override). `status` aggregates per-stage TanStack queries. + +**`PipelineGraph.svelte`** — A `FloatingPanel`-shaped left-pane component (matches the existing tree's docking style). Layout computed via simple Sugiyama-style top-down DAG layout (small N — under 30 stages typically — so a hand-rolled layout is fine; no new dep). Renders: + +- One `` per stage with: icon (camera | vision | unknown), label (override or resource name), status dot (color-coded), tint chip (filled when active), eye toggle (independent show/hide for graph-only filtering). +- One SVG `` over the node grid drawing arrows between connected nodes. +- Header with: "⚙ Overrides" button, "Use native colors" toggle, "Clear selection" button. + +Click semantics: +- Plain click on a node → `solo(id)`. +- Shift-click → `toggle(id)`. +- Click on background → `clear()`. + +**`StagePreview2D.svelte`** — Mounts iff `preview2D() !== undefined`. Reads the stage's outputs and renders accordingly: + +- For `image` stages: `getImages()` on `RefreshRates.pointclouds` cadence (a small misnomer, but reuses the existing rate; introducing `RefreshRates.images` is a follow-up). Returns `NamedImage[]`; rendered as labeled tiles in a vertical stack. If the camera's `mimeTypes` indicate a streamable format and the user has a stream-capable connection, fall back to the existing `StreamClient` approach (extracted from `Camera.svelte` into a shared utility). +- For `detections` stages: call `getDetectionsFromCamera(cameraName)` against the upstream camera (resolved from the DAG: the most-recently-active upstream camera in the active set; if none active, the configured `camera_name`). Render bounding boxes overlaid on that camera's current image (also via `getImages` for that camera). +- For `classifications` stages: `getClassificationsFromCamera(cameraName)`; render a ranked label list above the source image. +- For `objects`: not 2D — never the preview-2D candidate; rendered as 3D in the scene like today. + +Pane width is resizable via a drag handle; persisted to settings. + +**`PipelineOverrides.svelte`** — A new tab inside the existing `Settings` component. Three sections: + +1. **Visibility** — list of all stages with checkboxes. +2. **Outputs** — per-stage chip selector for `pointcloud | image | objects | detections | classifications`. Pre-checked from discovered outputs; unchecking forces an output off in this pipeline. +3. **Manual edges** — list of user-added `from → to` pairs with delete buttons; "Add edge" form with two pickers populated from the resource name set. + +A "Reset to derived" button clears all overrides for the current `partID`. + +### Hook modifications + +```ts +// existing +export const providePointclouds = (partID: () => string) => { ... } + +// modified +export interface PointcloudsOptions { + filter?: () => Set | undefined +} +export const providePointclouds = (partID: () => string, options: PointcloudsOptions = {}) => { + // inside enabledClients: + // if (options.filter && !options.filter().has(client.current.name)) continue + // rest unchanged +} +``` + +`providePointcloudObjects` gets the same shape. Existing call sites pass no options (no behavior change). + +The pipeline route's provider call is: + +```ts +providePointclouds(() => partID, { filter: () => pipelineResourceNames() }) +providePointcloudObjects(() => partID, { filter: () => pipelineResourceNames() }) +``` + +`pipelineResourceNames()` is a derived getter: the set of stage ids in the merged graph minus those hidden via override. + +### Scene rendering — filter and tint + +Two reactive effects in `+page.svelte`: + +1. **Visibility effect.** Watches `active()` and the world's entities. For each entity with a `Name` trait that matches a stage id: if the stage is in `active`, ensure the entity has no `Invisible` trait; otherwise add `Invisible`. Entities whose `Name` is not a stage id (e.g. arms, geometries) are left alone — the pipeline view doesn't hide non-pipeline entities by default. (Alternative considered: hide everything not in `active`. Rejected: drops the framed components that contextualize the pointclouds. The user can use the existing tree in the main app for that view.) +2. **Tint effect.** Watches `active()` + the "use native colors" setting. For each active entity, when native-colors is OFF: ensure a `PipelineTint(tint(id))` trait is present and the geometry's `color` attribute (if any) is removed; the existing `Color` trait reading then drives material color. When the entity becomes inactive or the toggle flips ON: remove the trait and let the original color render again. (The `color` attribute removal is one-way in Three.js — once deleted, we'd need to re-parse to restore. Plan: store the original color attribute on the entity in a `OriginalColors` trait at first apply, restore on tint removal. The existing PCD worker output already gives us the bytes to re-create the attribute without a re-fetch.) + +### Data flow + +``` +machine config (Struct) + │ + └─► usePartConfig ─┐ + │ +per-stage getProperties (TanStack, staleTime ∞) ─┤ + │ +overrides (localStorage, partID) ─┤ + ▼ + buildGraph + mergeOverrides → PipelineGraph + │ + ▼ + usePipeline context + ├─► PipelineGraph.svelte (renders) + ├─► usePointclouds.filter + ├─► usePointcloudObjects.filter + ├─► visibility effect (Invisible trait) + ├─► tint effect (PipelineTint trait) + └─► StagePreview2D (when 2D active) +``` + +## Pipeline schema (in-memory; not persisted as JSON) + +```ts +type StageId = string // resource name + +type StageOutput = + | 'pointcloud' + | 'image' + | 'objects' + | 'detections' + | 'classifications' + +interface Stage { + id: StageId + label: string + api: 'camera' | 'vision' + model: string + outputs: StageOutput[] + derivedFrom: 'config' | 'override' +} + +interface Edge { + from: StageId + to: StageId + derivedFrom: 'config' | 'override' +} + +interface Warning { + stageId: StageId | null + message: string +} + +interface PipelineGraph { + stages: Stage[] + edges: Edge[] + warnings: Warning[] +} +``` + +## Layout + +``` +┌─ header: part selector · refresh · "Pipeline overrides" ──────────────┐ +├─ graph (left, ~280px) ─┬─ scene (center, flex-1) ──┬─ preview-2d ────┤ +│ ▼ wrist-cam-left ● │ │ named-img: color│ +│ │ │ [3D point clouds] │ [tile] │ +│ ▼ wrist-l-cropped ● │ │ │ +│ │ │ [optional 2nd cloud │ named-img: depth│ +│ ▼ merged-cloud ◑ │ in compare mode, │ [tile] │ +│ │ │ different tint] │ │ +│ ▼ object-segmenter ◐ │ │ (only mounted │ +│ │ │ when preview2D │ +│ ⚙ Overrides │ │ is active) │ +└─────────────────────────┴────────────────────────────┴─────────────────┘ +``` + +## Error handling + +| Failure | Surface | Recovery | +|---|---|---| +| `getProperties` failure for a stage | Red status dot + tooltip; outputs default to "unknown" until override pins them | User opens overrides → set outputs manually | +| Pointcloud / image fetch failure | Stage status = error; scene/preview retains last-good or shows "no data yet" | Re-fetch on next refresh tick | +| Cyclic edges | Back-edge dropped; warning annotation on the receiving node | User uses overrides → "Hide edge" / "Add edge" to fix | +| Resource referenced in attributes but absent from resources list | Edge silently ignored; reported in `warnings[]` shown in overrides panel | User edits machine config or accepts | +| Detection stage's input camera not active | Preview pane warning: "input camera not active — graph lineage broken" | User shift-clicks the input camera into the active set | +| Stale override (references resource that no longer exists) | Override panel shows it as "unknown — remove" with a one-click remove | User clicks remove | + +## Testing + +**Unit (Vitest)** + +- `derive.ts`: + - Linear chain (camera → transform → segmenter) + - Fan-in (multiple cameras → join) + - Fan-out (one camera → multiple services) + - Unknown model with attribute references + - Cyclic config — verifies back-edge dropped + warning emitted + - Attribute-key variations (`src`, `source`, `camera`, `cameras`, `source_cameras`, `service`, `detector`, `vision`) + - Self-reference filtered + - Reference to non-existent resource → warning, no edge +- `override.ts`: + - Override wins over derived + - Stale override (referenced stage gone) emits a warning, doesn't crash + - Reset-to-derived clears all +- `tint.ts`: + - Deterministic per-id + - Reasonable hue distribution for N=20 + +**Component (Vitest + @testing-library/svelte)** + +- `PipelineGraph.svelte`: click → solo, shift-click → toggle, eye toggle → independent visibility. +- `StagePreview2D.svelte`: image stage renders tiles; detection stage renders bboxes given a fixture image + detection list. + +**E2E (Playwright)** + +- Open `/pipeline` against the existing fixture machine; assert the graph renders with expected nodes; solo a stage; assert the scene contains only that stage's points (count entities in the world that don't have `Invisible`). + +## Open Questions + +None blocking — implementation can begin. + +## Out-of-scope follow-ups + +1. `getPipelineMetadata` DoCommand convention so custom modules can declare lineage explicitly. +2. Per-stage `RefreshRates` (e.g. images at 100ms, pointclouds at 5s). +3. `transformPCD` to a common frame for compare mode if alignment problems surface. +4. Snapshot integration: capturing a pipeline frame and replaying it via `/snapshot`. +5. Server-side pipeline graph if a future backend wants to drive pipeline UX from a service. +6. Multi-part pipelines spanning machines. + +## Phased rollout + +1. **Phase 1 — Skeleton.** Route, providers shell, `derive.ts` + tests, `usePipeline`, `PipelineGraph` rendering nodes only. Unit tests green. +2. **Phase 2 — Spotlight scene.** Hook `filter` plumbing, `Invisible` + `PipelineTint` reactive effects, solo / toggle, header tint switch. Pointcloud + objects end-to-end. +3. **Phase 3 — Edges + cycles.** Attribute-value walker, edge layer in graph, cycle detection. +4. **Phase 4 — 2D preview.** `getImages` + stream fallback for image stages; detections / classifications overlays. +5. **Phase 5 — Override panel.** Visibility, outputs pinning, manual edges, persistence. + +Each phase is independently testable and demoable. From 9339fc9b3553c869d07e3c5892cd8a77c68c5740 Mon Sep 17 00:00:00 2001 From: HipsterBrown Date: Fri, 8 May 2026 14:06:34 -0400 Subject: [PATCH 2/9] docs(spec): address pipeline view review feedback - Scope tint effect to Points-trait entities only - Clarify clear-on-empty semantics (graph panel only, not canvas) - Add preview2D worked-examples table doubling as test fixtures - Add noise-rejection case to derive.ts test list - Call out the shared-refresh-rate UX wart for images --- .../specs/2026-05-08-pipeline-view-design.md | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-05-08-pipeline-view-design.md b/docs/superpowers/specs/2026-05-08-pipeline-view-design.md index d6bdad99f..138655d18 100644 --- a/docs/superpowers/specs/2026-05-08-pipeline-view-design.md +++ b/docs/superpowers/specs/2026-05-08-pipeline-view-design.md @@ -146,6 +146,17 @@ interface PipelineContext { Internally, `graph` is `$derived` from `usePartConfig` + per-stage `getProperties` queries + overrides. `active` is `$state` (a `SvelteSet`). `preview2D` is `$derived` from `active`: the most-recently-toggled active stage whose outputs include any of `image | detections | classifications` (and whose outputs do **not** include `pointcloud | objects`, OR whose `pointcloud`/`objects` is hidden via override). `status` aggregates per-stage TanStack queries. +**`preview2D` worked examples** (also serve as test fixtures): + +| Stage outputs (effective, post-override) | Active set (chronological) | `preview2D()` | +|---|---|---| +| `[pointcloud]` | `{ pcd-stage }` | `undefined` (3D-only; no preview pane) | +| `[image]` | `{ image-stage }` | `image-stage` | +| `[pointcloud, image]` | `{ stage }` | `undefined` (has a 3D output; renders in scene only — does **not** open preview) | +| `[pointcloud, image]` with `pointcloud` overridden off | `{ stage }` | `stage` (now effectively 2D-only) | +| `[detections]` then `[pointcloud]` shift-toggled | `{ det-stage, pcd-stage }` (det first) | `det-stage` (most-recent 2D-only candidate) | +| `[detections]` then another `[image]` | `{ det-stage, image-stage }` | `image-stage` (most-recently toggled wins) | + **`PipelineGraph.svelte`** — A `FloatingPanel`-shaped left-pane component (matches the existing tree's docking style). Layout computed via simple Sugiyama-style top-down DAG layout (small N — under 30 stages typically — so a hand-rolled layout is fine; no new dep). Renders: - One `` per stage with: icon (camera | vision | unknown), label (override or resource name), status dot (color-coded), tint chip (filled when active), eye toggle (independent show/hide for graph-only filtering). @@ -155,11 +166,11 @@ Internally, `graph` is `$derived` from `usePartConfig` + per-stage `getPropertie Click semantics: - Plain click on a node → `solo(id)`. - Shift-click → `toggle(id)`. -- Click on background → `clear()`. +- Click on the graph panel's empty area (not on a node, not on the SVG edge layer) → `clear()`. The Threlte canvas is not a clear target — orbiting/clicking the scene leaves the active set untouched. **`StagePreview2D.svelte`** — Mounts iff `preview2D() !== undefined`. Reads the stage's outputs and renders accordingly: -- For `image` stages: `getImages()` on `RefreshRates.pointclouds` cadence (a small misnomer, but reuses the existing rate; introducing `RefreshRates.images` is a follow-up). Returns `NamedImage[]`; rendered as labeled tiles in a vertical stack. If the camera's `mimeTypes` indicate a streamable format and the user has a stream-capable connection, fall back to the existing `StreamClient` approach (extracted from `Camera.svelte` into a shared utility). +- For `image` stages: `getImages()` on `RefreshRates.pointclouds` cadence (a small misnomer, but reuses the existing rate; introducing `RefreshRates.images` is a follow-up). Returns `NamedImage[]`; rendered as labeled tiles in a vertical stack. If the camera's `mimeTypes` indicate a streamable format and the user has a stream-capable connection, fall back to the existing `StreamClient` approach (extracted from `Camera.svelte` into a shared utility). **UX caveat:** because images and pointclouds share a refresh rate in v1, users with a slow `pointclouds` setting (e.g. 5s) will see equally slow image updates; users with a fast setting (e.g. 100ms) will spend bandwidth they may not want on `getImages` polls. Calling this out so it isn't a planning surprise — the stream fallback mitigates the bandwidth side when available. - For `detections` stages: call `getDetectionsFromCamera(cameraName)` against the upstream camera (resolved from the DAG: the most-recently-active upstream camera in the active set; if none active, the configured `camera_name`). Render bounding boxes overlaid on that camera's current image (also via `getImages` for that camera). - For `classifications` stages: `getClassificationsFromCamera(cameraName)`; render a ranked label list above the source image. - For `objects`: not 2D — never the preview-2D candidate; rendered as 3D in the scene like today. @@ -209,6 +220,8 @@ Two reactive effects in `+page.svelte`: 1. **Visibility effect.** Watches `active()` and the world's entities. For each entity with a `Name` trait that matches a stage id: if the stage is in `active`, ensure the entity has no `Invisible` trait; otherwise add `Invisible`. Entities whose `Name` is not a stage id (e.g. arms, geometries) are left alone — the pipeline view doesn't hide non-pipeline entities by default. (Alternative considered: hide everything not in `active`. Rejected: drops the framed components that contextualize the pointclouds. The user can use the existing tree in the main app for that view.) 2. **Tint effect.** Watches `active()` + the "use native colors" setting. For each active entity, when native-colors is OFF: ensure a `PipelineTint(tint(id))` trait is present and the geometry's `color` attribute (if any) is removed; the existing `Color` trait reading then drives material color. When the entity becomes inactive or the toggle flips ON: remove the trait and let the original color render again. (The `color` attribute removal is one-way in Three.js — once deleted, we'd need to re-parse to restore. Plan: store the original color attribute on the entity in a `OriginalColors` trait at first apply, restore on tint removal. The existing PCD worker output already gives us the bytes to re-create the attribute without a re-fetch.) + **Scope of the tint effect.** Applies only to entities that carry the `Points` trait (pointcloud + segmenter object clouds). Mesh/geometry entities (frames, arms, custom geometries) are not pipeline stages and are not tinted. This keeps the `Color`/`Colors`/`OriginalColors` interplay confined to the one consumer that actually needs it (the `Points` renderer); the existing `Color`-trait usage on meshes is unchanged. + ### Data flow ``` @@ -312,6 +325,7 @@ interface PipelineGraph { - Attribute-key variations (`src`, `source`, `camera`, `cameras`, `source_cameras`, `service`, `detector`, `vision`) - Self-reference filtered - Reference to non-existent resource → warning, no edge + - **Noise rejection:** an attributes Struct contains a string field whose value coincidentally matches a resource name but is not a dependency (e.g. a `description` or `label` field) — verifies a single edge is recorded if the value appears, and serves as a documented limitation; the override panel's "delete edge" handles real-world false positives - `override.ts`: - Override wins over derived - Stale override (referenced stage gone) emits a warning, doesn't crash From 033e372d782bb4563a4176fda7b2ecda2a3a003c Mon Sep 17 00:00:00 2001 From: HipsterBrown Date: Fri, 8 May 2026 14:14:56 -0400 Subject: [PATCH 3/9] docs(plan): pipeline view MVP implementation plan Covers spec phases 1-3 (skeleton, spotlight scene, edges). Phases 4 (2D preview) and 5 (overrides) deferred to follow-up plans. --- .../plans/2026-05-08-pipeline-view.md | 2056 +++++++++++++++++ 1 file changed, 2056 insertions(+) create mode 100644 docs/superpowers/plans/2026-05-08-pipeline-view.md diff --git a/docs/superpowers/plans/2026-05-08-pipeline-view.md b/docs/superpowers/plans/2026-05-08-pipeline-view.md new file mode 100644 index 000000000..21d272a0e --- /dev/null +++ b/docs/superpowers/plans/2026-05-08-pipeline-view.md @@ -0,0 +1,2056 @@ +# Pipeline View Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build the MVP slice of the pipeline view (spec phases 1–3): a `/pipeline` route that auto-derives perception stages from the live machine config, renders them as a DAG with directed edges, and "spotlights" selected stages in the existing 3D scene with per-stage color tinting. + +**Architecture:** New SvelteKit route at `/pipeline` with a trimmed providers shell. A new `src/lib/pipeline/` library owns derivation (pure functions), context (`usePipeline`), and UI (`PipelineGraph`, `StageNode`, `EdgeLayer`). Existing `usePointclouds` / `usePointcloudObjects` hooks gain an optional `filter` parameter and stamp every entity they spawn with a new `PipelineSource(stageId)` trait so the route can match scene entities to graph nodes. Visibility (`Invisible` trait) and tint (`PipelineTint` + `OriginalColors` traits) are reactive effects in the route page. + +**Tech Stack:** Svelte 5 runes, Threlte, Koota ECS, TanStack Query, `@viamrobotics/sdk`, `@viamrobotics/svelte-sdk`, Vitest, `@testing-library/svelte`, Playwright. + +**Spec:** `docs/superpowers/specs/2026-05-08-pipeline-view-design.md` + +**Out of scope (deferred to follow-up plans):** 2D preview pane (spec §6 / phase 4), override panel + persistence (spec §7 / phase 5). + +--- + +## File Structure + +**Created** + +- `src/lib/pipeline/types.ts` — shared types (`Stage`, `Edge`, `StageOutput`, `PipelineGraph`, `RGB`, `Warning`). +- `src/lib/pipeline/tint.ts` — deterministic stage-id → RGB mapping. +- `src/lib/pipeline/derive.ts` — pure functions building `PipelineGraph` from `(config, propertiesMap)`. +- `src/lib/pipeline/visibility.ts` — pure helper: given active set + entity list, compute which to mark `Invisible`. +- `src/lib/pipeline/usePipeline.svelte.ts` — Svelte context: graph, active set, status, tint, solo/toggle/clear API. +- `src/lib/pipeline/StageNode.svelte` — single graph node (icon, label, status dot, tint chip). +- `src/lib/pipeline/EdgeLayer.svelte` — SVG arrow layer between nodes. +- `src/lib/pipeline/layout.ts` — DAG layout (column-major topo levels) producing `{x,y}` per stage id. +- `src/lib/pipeline/PipelineGraph.svelte` — left-pane panel composing nodes + edges. +- `src/lib/pipeline/__tests__/derive.spec.ts` +- `src/lib/pipeline/__tests__/tint.spec.ts` +- `src/lib/pipeline/__tests__/layout.spec.ts` +- `src/lib/pipeline/__tests__/visibility.spec.ts` +- `src/lib/pipeline/__tests__/StageNode.spec.ts` +- `src/lib/pipeline/__tests__/PipelineGraph.spec.ts` +- `src/routes/pipeline/+page.svelte` — composition only; mounts providers, canvas, graph panel, effects. +- `src/routes/pipeline/+page.ts` — extracts `partID` from URL params. +- `e2e/pipeline.spec.ts` — Playwright smoke test. + +**Modified** + +- `src/lib/ecs/traits.ts` — add `PipelineSource(stageId: string)`, `PipelineTint({r,g,b})`, `OriginalColors(Uint8Array)`. +- `src/lib/hooks/usePointclouds.svelte.ts` — accept optional `filter`, stamp `PipelineSource` on spawn. +- `src/lib/hooks/usePointcloudObjects.svelte.ts` — same shape. +- `src/lib/components/Entities/Points.svelte` — read `PipelineTint` and prefer it over geometry's `color` attribute when present. +- `.gitignore` — add `.superpowers/`. + +--- + +## Conventions for every task + +- **TDD discipline.** Write the failing test first, run it to see the failure, write the smallest passing implementation, run again to see green, commit. +- **Run scope.** Use `pnpm test -- ` (Vitest) for the file under test, not the full suite, until the task closes. End each phase with `pnpm check && pnpm lint && pnpm test`. +- **Commit format.** Conventional Commits: `feat(pipeline): …`, `test(pipeline): …`, `refactor(hooks): …`. One commit per task unless a task explicitly says "split." +- **Imports.** Use `$lib/...` aliases (matches existing code). +- **Svelte 5 only.** Runes (`$state`, `$derived`, `$effect`, `$props`); no Svelte 4 syntax. No `` — snippets only. +- **No emojis** in code or commit messages. + +--- + +## Phase 1 — Skeleton + +### Task 0: Housekeeping + +**Files:** +- Modify: `.gitignore` + +- [ ] **Step 1:** Append `.superpowers/` on a new line in `.gitignore`. +- [ ] **Step 2:** Verify with `git status` that `.superpowers/` is now ignored. +- [ ] **Step 3:** Commit. + +```bash +git add .gitignore +git commit -m "chore: ignore .superpowers brainstorming workspace" +``` + +--- + +### Task 1: Pipeline types + +**Files:** +- Create: `src/lib/pipeline/types.ts` + +- [ ] **Step 1:** Write the file. Pure type module — no tests required. + +```ts +// src/lib/pipeline/types.ts + +export type StageId = string + +export type StageOutput = + | 'pointcloud' + | 'image' + | 'objects' + | 'detections' + | 'classifications' + +export type StageApi = 'camera' | 'vision' + +export interface Stage { + id: StageId + label: string + api: StageApi + model: string + outputs: StageOutput[] + derivedFrom: 'config' | 'override' +} + +export interface Edge { + from: StageId + to: StageId + derivedFrom: 'config' | 'override' +} + +export interface Warning { + stageId: StageId | null + message: string +} + +export interface PipelineGraph { + stages: Stage[] + edges: Edge[] + warnings: Warning[] +} + +export interface RGB { + r: number + g: number + b: number +} + +export type StageStatus = 'idle' | 'loading' | 'ok' | 'error' +``` + +- [ ] **Step 2:** `pnpm check` — must pass with no errors. +- [ ] **Step 3:** Commit. + +```bash +git add src/lib/pipeline/types.ts +git commit -m "feat(pipeline): add shared types" +``` + +--- + +### Task 2: Deterministic stage-id tint + +**Files:** +- Create: `src/lib/pipeline/tint.ts` +- Test: `src/lib/pipeline/__tests__/tint.spec.ts` + +- [ ] **Step 1: Write the failing test.** + +```ts +// src/lib/pipeline/__tests__/tint.spec.ts +import { describe, expect, it } from 'vitest' + +import { tintForStageId } from '../tint' + +describe('tintForStageId', () => { + it('returns r,g,b values in [0,1]', () => { + const rgb = tintForStageId('wrist-cam-left') + expect(rgb.r).toBeGreaterThanOrEqual(0) + expect(rgb.r).toBeLessThanOrEqual(1) + expect(rgb.g).toBeGreaterThanOrEqual(0) + expect(rgb.g).toBeLessThanOrEqual(1) + expect(rgb.b).toBeGreaterThanOrEqual(0) + expect(rgb.b).toBeLessThanOrEqual(1) + }) + + it('is deterministic for the same id', () => { + expect(tintForStageId('merged-cloud')).toEqual(tintForStageId('merged-cloud')) + }) + + it('produces different colors for different ids', () => { + const a = tintForStageId('wrist-cam-left') + const b = tintForStageId('wrist-cam-right') + expect(a).not.toEqual(b) + }) + + it('distributes hues across many ids', () => { + const ids = Array.from({ length: 20 }, (_, i) => `stage-${i}`) + const tints = new Set(ids.map((id) => JSON.stringify(tintForStageId(id)))) + // Expect close to N distinct colors (hash collisions tolerated up to ~10%) + expect(tints.size).toBeGreaterThanOrEqual(18) + }) +}) +``` + +- [ ] **Step 2:** Run `pnpm test -- src/lib/pipeline/__tests__/tint.spec.ts`. Expected: FAIL ("Cannot find module '../tint'"). + +- [ ] **Step 3: Write implementation.** + +```ts +// src/lib/pipeline/tint.ts +import type { RGB } from './types' + +const hashString = (s: string): number => { + let h = 2166136261 + for (let i = 0; i < s.length; i += 1) { + h ^= s.charCodeAt(i) + h = Math.imul(h, 16777619) + } + return h >>> 0 +} + +const hslToRgb = (h: number, s: number, l: number): RGB => { + const c = (1 - Math.abs(2 * l - 1)) * s + const hp = h * 6 + const x = c * (1 - Math.abs((hp % 2) - 1)) + let r = 0 + let g = 0 + let b = 0 + if (hp < 1) [r, g, b] = [c, x, 0] + else if (hp < 2) [r, g, b] = [x, c, 0] + else if (hp < 3) [r, g, b] = [0, c, x] + else if (hp < 4) [r, g, b] = [0, x, c] + else if (hp < 5) [r, g, b] = [x, 0, c] + else [r, g, b] = [c, 0, x] + const m = l - c / 2 + return { r: r + m, g: g + m, b: b + m } +} + +export const tintForStageId = (id: string): RGB => { + const hue = (hashString(id) % 360) / 360 + return hslToRgb(hue, 0.65, 0.55) +} +``` + +- [ ] **Step 4:** Run the test again. Expected: PASS. + +- [ ] **Step 5:** Commit. + +```bash +git add src/lib/pipeline/tint.ts src/lib/pipeline/__tests__/tint.spec.ts +git commit -m "feat(pipeline): deterministic stage-id tint helper" +``` + +--- + +### Task 3: Derive — stage list (no edges yet) + +`derive.ts` is built incrementally. Task 3 produces stages from config + properties map. Edges land in Task 15 (Phase 3). + +**Files:** +- Create: `src/lib/pipeline/derive.ts` +- Test: `src/lib/pipeline/__tests__/derive.spec.ts` + +- [ ] **Step 1: Write the failing test.** + +```ts +// src/lib/pipeline/__tests__/derive.spec.ts +import { describe, expect, it } from 'vitest' + +import { buildGraph } from '../derive' + +const config = { + components: [ + { name: 'wrist-cam-left', api: 'rdk:component:camera', model: 'webcam', attributes: {} }, + { + name: 'merged-cloud', + api: 'rdk:component:camera', + model: 'transform', + attributes: { source: 'wrist-cam-left' }, + }, + { name: 'arm-1', api: 'rdk:component:arm', model: 'ur5', attributes: {} }, + ], + services: [ + { + name: 'object-segmenter', + api: 'rdk:service:vision', + model: 'segmenter', + attributes: { camera_name: 'merged-cloud' }, + }, + ], +} + +const properties = new Map([ + ['wrist-cam-left', { supportsPcd: true, mimeTypes: ['image/jpeg'] }], + ['merged-cloud', { supportsPcd: true, mimeTypes: [] }], + [ + 'object-segmenter', + { classificationsSupported: false, detectionsSupported: false, objectPointCloudsSupported: true }, + ], +]) + +describe('buildGraph: stages', () => { + it('emits one stage per camera with relevant outputs', () => { + const graph = buildGraph(config, properties) + const ids = graph.stages.map((s) => s.id).sort() + expect(ids).toEqual(['merged-cloud', 'object-segmenter', 'wrist-cam-left']) + }) + + it('skips arms and other irrelevant components', () => { + const graph = buildGraph(config, properties) + expect(graph.stages.find((s) => s.id === 'arm-1')).toBeUndefined() + }) + + it('infers camera outputs from properties', () => { + const graph = buildGraph(config, properties) + const wrist = graph.stages.find((s) => s.id === 'wrist-cam-left')! + expect(wrist.outputs).toEqual(expect.arrayContaining(['pointcloud', 'image'])) + const merged = graph.stages.find((s) => s.id === 'merged-cloud')! + expect(merged.outputs).toEqual(['pointcloud']) + }) + + it('infers vision outputs from properties', () => { + const graph = buildGraph(config, properties) + const seg = graph.stages.find((s) => s.id === 'object-segmenter')! + expect(seg.outputs).toEqual(['objects']) + expect(seg.api).toBe('vision') + }) + + it('drops cameras with no relevant outputs', () => { + const graphProps = new Map([ + ['cam', { supportsPcd: false, mimeTypes: [] }], + ]) + const graph = buildGraph( + { + components: [ + { name: 'cam', api: 'rdk:component:camera', model: 'webcam', attributes: {} }, + ], + services: [], + }, + graphProps + ) + expect(graph.stages).toHaveLength(0) + }) + + it('returns empty edges in this milestone', () => { + const graph = buildGraph(config, properties) + expect(graph.edges).toEqual([]) + }) +}) +``` + +- [ ] **Step 2:** Run `pnpm test -- src/lib/pipeline/__tests__/derive.spec.ts`. Expected: FAIL. + +- [ ] **Step 3: Write implementation.** + +```ts +// src/lib/pipeline/derive.ts +import type { + Edge, + PipelineGraph, + Stage, + StageApi, + StageOutput, + Warning, +} from './types' + +export interface CameraProperties { + supportsPcd?: boolean + mimeTypes?: string[] +} + +export interface VisionProperties { + classificationsSupported?: boolean + detectionsSupported?: boolean + objectPointCloudsSupported?: boolean +} + +export type StageProperties = CameraProperties | VisionProperties + +export type PropertiesMap = ReadonlyMap + +interface ResourceConfig { + name: string + api?: string + model?: string + attributes?: Record +} + +interface MachineConfig { + components?: ResourceConfig[] + services?: ResourceConfig[] +} + +const apiOf = (res: ResourceConfig): StageApi | undefined => { + if (res.api?.includes(':camera')) return 'camera' + if (res.api?.includes(':vision')) return 'vision' + return undefined +} + +const cameraOutputs = (props: CameraProperties | undefined): StageOutput[] => { + const outputs: StageOutput[] = [] + if (props?.supportsPcd) outputs.push('pointcloud') + if (props?.mimeTypes && props.mimeTypes.length > 0) outputs.push('image') + return outputs +} + +const visionOutputs = (props: VisionProperties | undefined): StageOutput[] => { + const outputs: StageOutput[] = [] + if (props?.objectPointCloudsSupported) outputs.push('objects') + if (props?.detectionsSupported) outputs.push('detections') + if (props?.classificationsSupported) outputs.push('classifications') + return outputs +} + +const resourceToStage = ( + res: ResourceConfig, + properties: PropertiesMap +): Stage | undefined => { + const api = apiOf(res) + if (!api) return undefined + const props = properties.get(res.name) + const outputs = + api === 'camera' + ? cameraOutputs(props as CameraProperties | undefined) + : visionOutputs(props as VisionProperties | undefined) + if (outputs.length === 0) return undefined + return { + id: res.name, + label: res.name, + api, + model: res.model ?? '', + outputs, + derivedFrom: 'config', + } +} + +export const buildGraph = ( + config: MachineConfig, + properties: PropertiesMap +): PipelineGraph => { + const stages: Stage[] = [] + const warnings: Warning[] = [] + const edges: Edge[] = [] // edges land in phase 3 + + for (const res of [...(config.components ?? []), ...(config.services ?? [])]) { + const stage = resourceToStage(res, properties) + if (stage) stages.push(stage) + } + + return { stages, edges, warnings } +} +``` + +- [ ] **Step 4:** Run the test. Expected: PASS. + +- [ ] **Step 5:** Commit. + +```bash +git add src/lib/pipeline/derive.ts src/lib/pipeline/__tests__/derive.spec.ts +git commit -m "feat(pipeline): derive stages from machine config + properties" +``` + +--- + +### Task 4: `usePipeline` Svelte context + +**Files:** +- Create: `src/lib/pipeline/usePipeline.svelte.ts` + +`usePipeline` is mostly orchestration. Tested implicitly through component tests in Tasks 6 and 7. Pure-function bits (active-set transitions) should be testable but the API itself is small enough that direct unit testing isn't load-bearing. + +- [ ] **Step 1:** Write the file. + +```ts +// src/lib/pipeline/usePipeline.svelte.ts +import { SvelteSet } from 'svelte/reactivity' +import { getContext, setContext } from 'svelte' + +import type { PipelineGraph, RGB, StageId, StageStatus } from './types' + +import { tintForStageId } from './tint' + +const KEY = Symbol('pipeline-context') + +interface PipelineContext { + graph: () => PipelineGraph + active: () => ReadonlySet + tint: (id: StageId) => RGB + status: (id: StageId) => StageStatus + useNativeColors: () => boolean + setUseNativeColors: (value: boolean) => void + solo: (id: StageId) => void + toggle: (id: StageId) => void + clear: () => void +} + +export interface ProvidePipelineParams { + graph: () => PipelineGraph + status?: (id: StageId) => StageStatus +} + +export const providePipeline = (params: ProvidePipelineParams): PipelineContext => { + const active = new SvelteSet() + let useNativeColors = $state(false) + + const ctx: PipelineContext = { + graph: params.graph, + active: () => active, + tint: tintForStageId, + status: params.status ?? (() => 'idle'), + useNativeColors: () => useNativeColors, + setUseNativeColors: (value) => { + useNativeColors = value + }, + solo: (id) => { + active.clear() + active.add(id) + }, + toggle: (id) => { + if (active.has(id)) active.delete(id) + else active.add(id) + }, + clear: () => active.clear(), + } + + setContext(KEY, ctx) + return ctx +} + +export const usePipeline = (): PipelineContext => { + const ctx = getContext(KEY) + if (!ctx) throw new Error('usePipeline called outside providePipeline') + return ctx +} +``` + +- [ ] **Step 2:** `pnpm check`. Expected: PASS. + +- [ ] **Step 3:** Commit. + +```bash +git add src/lib/pipeline/usePipeline.svelte.ts +git commit -m "feat(pipeline): usePipeline context with active-set + tint API" +``` + +--- + +### Task 5: `StageNode` component + +**Files:** +- Create: `src/lib/pipeline/StageNode.svelte` +- Test: `src/lib/pipeline/__tests__/StageNode.spec.ts` + +- [ ] **Step 1: Write the failing test.** + +```ts +// src/lib/pipeline/__tests__/StageNode.spec.ts +import { render, screen } from '@testing-library/svelte' +import userEvent from '@testing-library/user-event' +import { describe, expect, it, vi } from 'vitest' + +import StageNode from '../StageNode.svelte' + +const baseStage = { + id: 'wrist-cam-left', + label: 'Wrist L', + api: 'camera' as const, + model: 'webcam', + outputs: ['pointcloud' as const, 'image' as const], + derivedFrom: 'config' as const, +} + +describe('StageNode', () => { + it('renders the label', () => { + render(StageNode, { + stage: baseStage, + active: false, + status: 'ok', + tint: { r: 1, g: 0, b: 0 }, + onclick: vi.fn(), + }) + expect(screen.getByText('Wrist L')).toBeInTheDocument() + }) + + it('marks itself as active via aria-pressed', () => { + render(StageNode, { + stage: baseStage, + active: true, + status: 'ok', + tint: { r: 1, g: 0, b: 0 }, + onclick: vi.fn(), + }) + expect(screen.getByRole('button', { name: /Wrist L/ })).toHaveAttribute( + 'aria-pressed', + 'true' + ) + }) + + it('fires onclick with shiftKey when shift-clicked', async () => { + const onclick = vi.fn() + const user = userEvent.setup() + render(StageNode, { + stage: baseStage, + active: false, + status: 'ok', + tint: { r: 1, g: 0, b: 0 }, + onclick, + }) + await user.keyboard('{Shift>}') + await user.click(screen.getByRole('button', { name: /Wrist L/ })) + await user.keyboard('{/Shift}') + expect(onclick).toHaveBeenCalledWith(expect.objectContaining({ shiftKey: true })) + }) +}) +``` + +- [ ] **Step 2:** Run `pnpm test -- src/lib/pipeline/__tests__/StageNode.spec.ts`. Expected: FAIL (component not found). + +- [ ] **Step 3: Write component.** + +```svelte + + + + +``` + +- [ ] **Step 4:** Run the test. Expected: PASS. + +- [ ] **Step 5:** Commit. + +```bash +git add src/lib/pipeline/StageNode.svelte src/lib/pipeline/__tests__/StageNode.spec.ts +git commit -m "feat(pipeline): StageNode component" +``` + +--- + +### Task 6: `PipelineGraph` panel (no edges) + +**Files:** +- Create: `src/lib/pipeline/PipelineGraph.svelte` +- Test: `src/lib/pipeline/__tests__/PipelineGraph.spec.ts` + +The panel renders a stack of `StageNode` instances in deterministic order, exposes a "Use native colors" toggle, and dispatches click events to `usePipeline`. Edges land in Task 14. + +- [ ] **Step 1: Write the failing test.** + +```ts +// src/lib/pipeline/__tests__/PipelineGraph.spec.ts +import { render, screen } from '@testing-library/svelte' +import userEvent from '@testing-library/user-event' +import { describe, expect, it } from 'vitest' + +import GraphHarness from './__fixtures__/GraphHarness.svelte' + +const stages = [ + { + id: 'wrist-cam-left', + label: 'Wrist L', + api: 'camera' as const, + model: 'webcam', + outputs: ['pointcloud' as const], + derivedFrom: 'config' as const, + }, + { + id: 'merged-cloud', + label: 'Merged', + api: 'camera' as const, + model: 'transform', + outputs: ['pointcloud' as const], + derivedFrom: 'config' as const, + }, +] + +describe('PipelineGraph', () => { + it('renders one node per stage', () => { + render(GraphHarness, { stages, edges: [], warnings: [] }) + expect(screen.getByRole('button', { name: /Wrist L/ })).toBeInTheDocument() + expect(screen.getByRole('button', { name: /Merged/ })).toBeInTheDocument() + }) + + it('plain click solos the stage', async () => { + const user = userEvent.setup() + const harness = render(GraphHarness, { stages, edges: [], warnings: [] }) + await user.click(screen.getByRole('button', { name: /Wrist L/ })) + expect(harness.component.activeIds()).toEqual(['wrist-cam-left']) + }) + + it('shift-click adds without clearing', async () => { + const user = userEvent.setup() + const harness = render(GraphHarness, { stages, edges: [], warnings: [] }) + await user.click(screen.getByRole('button', { name: /Wrist L/ })) + await user.keyboard('{Shift>}') + await user.click(screen.getByRole('button', { name: /Merged/ })) + await user.keyboard('{/Shift}') + expect(harness.component.activeIds().sort()).toEqual(['merged-cloud', 'wrist-cam-left']) + }) + + it('toggle native colors button is wired', async () => { + const user = userEvent.setup() + const harness = render(GraphHarness, { stages, edges: [], warnings: [] }) + expect(harness.component.useNativeColors()).toBe(false) + await user.click(screen.getByRole('button', { name: /native colors/i })) + expect(harness.component.useNativeColors()).toBe(true) + }) +}) +``` + +- [ ] **Step 2:** Create the test fixture component. + +```svelte + + + + + + +``` + +- [ ] **Step 3:** Run `pnpm test -- src/lib/pipeline/__tests__/PipelineGraph.spec.ts`. Expected: FAIL. + +- [ ] **Step 4: Write component.** + +```svelte + + + + +``` + +- [ ] **Step 5:** Run the test. Expected: PASS. + +- [ ] **Step 6:** Commit. + +```bash +git add src/lib/pipeline/PipelineGraph.svelte src/lib/pipeline/__tests__/PipelineGraph.spec.ts src/lib/pipeline/__tests__/__fixtures__/GraphHarness.svelte +git commit -m "feat(pipeline): PipelineGraph panel rendering nodes" +``` + +--- + +### Task 7: `/pipeline` route shell + +**Files:** +- Create: `src/routes/pipeline/+page.ts` +- Create: `src/routes/pipeline/+page.svelte` + +The page composes providers, the canvas, the graph panel, and the existing `Settings`/`RefreshRate` overlays. Property fetching is wired via `useResourceNames` + `createResourceQuery` (mirroring `usePointclouds`). No scene filter yet — that's Task 11. + +- [ ] **Step 1:** Add `+page.ts`. + +```ts +// src/routes/pipeline/+page.ts +import type { PageLoad } from './$types' + +export const ssr = false +export const prerender = false + +export const load: PageLoad = ({ url }) => { + const partID = url.searchParams.get('partID') ?? '' + return { partID } +} +``` + +- [ ] **Step 2:** Add `+page.svelte`. + +```svelte + + + +
+ + +
+ + + {#snippet children({ focus: _focus })} + + {/snippet} + + + +
+ +
+ + +
+
+``` + +- [ ] **Step 3:** `pnpm check`. Expected: PASS. (TypeScript may grumble about `providePointclouds` `options` param until Task 9; if blocking, temporarily remove the `options` arg here and restore in Task 9.) + +- [ ] **Step 4:** `pnpm dev` and visit `http://localhost:5173/pipeline?partID=`. Expected: empty graph (no derive-edges yet, no scene filter), Threlte canvas renders, Settings + RefreshRate overlays present. Confirm in a browser. + +- [ ] **Step 5:** Commit. + +```bash +git add src/routes/pipeline/+page.ts src/routes/pipeline/+page.svelte +git commit -m "feat(pipeline): /pipeline route with providers + graph panel" +``` + +--- + +## Phase 2 — Spotlight scene + +### Task 8: ECS traits for pipeline rendering + +**Files:** +- Modify: `src/lib/ecs/traits.ts` + +- [ ] **Step 1:** Append to `src/lib/ecs/traits.ts`: + +```ts +import { trait } from 'koota' +// (existing imports unchanged) + +/** The stage id (resource name) that produced this entity. Set by the pipeline-aware hooks. */ +export const PipelineSource = trait(() => '') + +/** Per-stage tint applied in pipeline view; rendered in place of geometry color. */ +export const PipelineTint = trait({ r: 0, g: 0, b: 0 }) + +/** Snapshot of a geometry's original `color` attribute bytes, used to restore tint→native. */ +export const OriginalColors = trait(() => new Uint8Array() as Uint8Array) +``` + +- [ ] **Step 2:** `pnpm check`. Expected: PASS. + +- [ ] **Step 3:** Commit. + +```bash +git add src/lib/ecs/traits.ts +git commit -m "feat(ecs): pipeline-rendering traits" +``` + +--- + +### Task 9: `usePointclouds` filter + `PipelineSource` stamp + +**Files:** +- Modify: `src/lib/hooks/usePointclouds.svelte.ts` +- Test: `src/lib/hooks/__tests__/usePointclouds.filter.spec.ts` + +The hook is large; this task only adds (a) an optional `filter` to `providePointclouds`, applied in `enabledClients`, and (b) a `traits.PipelineSource(name)` trait on every entity it spawns. No other behavior changes. The new test exercises the filter shape only — full hook integration is implicitly tested by the existing app. + +- [ ] **Step 1: Write the failing test.** + +```ts +// src/lib/hooks/__tests__/usePointclouds.filter.spec.ts +import { describe, expect, it } from 'vitest' + +import { applyFilter } from '$lib/hooks/usePointclouds.svelte' + +describe('usePointclouds applyFilter', () => { + const allow = new Set(['cam-a', 'cam-c']) + + it('returns true when no filter is given', () => { + expect(applyFilter('cam-a', undefined)).toBe(true) + }) + + it('returns true when name is in the filter', () => { + expect(applyFilter('cam-a', allow)).toBe(true) + }) + + it('returns false when name is missing from the filter', () => { + expect(applyFilter('cam-b', allow)).toBe(false) + }) +}) +``` + +- [ ] **Step 2:** Run `pnpm test -- src/lib/hooks/__tests__/usePointclouds.filter.spec.ts`. Expected: FAIL. + +- [ ] **Step 3: Modify the hook.** Add at the top of `usePointclouds.svelte.ts` (after imports): + +```ts +export const applyFilter = (name: string, filter: ReadonlySet | undefined): boolean => + filter === undefined || filter.has(name) + +export interface ProvidePointcloudsOptions { + filter?: () => ReadonlySet | undefined +} +``` + +Update the `providePointclouds` signature: + +```ts +export const providePointclouds = ( + partID: () => string, + options: ProvidePointcloudsOptions = {} +) => { + // ... existing code ... +} +``` + +In the `enabledClients` `$derived.by`, add the filter check alongside the existing checks. Replace this stretch: + +```ts +if ( + fetchedPropQueries && + client.current?.name && + interval !== RefetchRates.OFF && + disabledCameras[client.current?.name] !== true +) { + results.push(client as { current: CameraClient }) +} +``` + +with: + +```ts +const allow = options.filter?.() +if ( + fetchedPropQueries && + client.current?.name && + interval !== RefetchRates.OFF && + disabledCameras[client.current?.name] !== true && + applyFilter(client.current.name, allow) +) { + results.push(client as { current: CameraClient }) +} +``` + +In the spawn block, add `traits.PipelineSource(name)` to the entity traits: + +```ts +const entity = world.spawn( + ...hierarchy.parentTraits(name), + traits.Name(`${name} pointcloud`), + traits.BufferGeometry(geometry), + traits.Points, + traits.PipelineSource(name) +) +``` + +- [ ] **Step 4:** Run the test. Expected: PASS. + +- [ ] **Step 5:** `pnpm check && pnpm test -- usePointclouds`. Expected: existing tests stay green. + +- [ ] **Step 6:** Commit. + +```bash +git add src/lib/hooks/usePointclouds.svelte.ts src/lib/hooks/__tests__/usePointclouds.filter.spec.ts +git commit -m "feat(hooks): usePointclouds optional filter + PipelineSource trait" +``` + +--- + +### Task 10: `usePointcloudObjects` filter + `PipelineSource` stamp + +**Files:** +- Modify: `src/lib/hooks/usePointcloudObjects.svelte.ts` + +Same shape as Task 9. The test from Task 9 exports `applyFilter` from `usePointclouds`; this hook should import-and-reuse the same helper. + +- [ ] **Step 1:** At the top of `usePointcloudObjects.svelte.ts`: + +```ts +import { applyFilter } from './usePointclouds.svelte' + +export interface ProvidePointcloudObjectsOptions { + filter?: () => ReadonlySet | undefined +} +``` + +- [ ] **Step 2:** Update the signature: + +```ts +export const providePointcloudObjects = ( + partID: () => string, + options: ProvidePointcloudObjectsOptions = {} +) => { + // ... +} +``` + +- [ ] **Step 3:** In its `enabledClients` `$derived.by`, add the filter check (mirror Task 9). Then in **both** spawn blocks (the pointcloud entity around line ~204 and the geometry entity around line ~238), add `traits.PipelineSource(name)` to the entity traits — `name` here is the vision-service name from the enclosing loop. + +- [ ] **Step 4:** `pnpm check && pnpm test -- usePointcloudObjects`. Expected: PASS. + +- [ ] **Step 5:** Commit. + +```bash +git add src/lib/hooks/usePointcloudObjects.svelte.ts +git commit -m "feat(hooks): usePointcloudObjects optional filter + PipelineSource trait" +``` + +--- + +### Task 11: Pure visibility helper + +The visibility effect is the only place that talks to ECS, but the policy it implements (which entities should be `Invisible` given an active set) is pure and deserves a unit test before we wire it. + +**Files:** +- Create: `src/lib/pipeline/visibility.ts` +- Test: `src/lib/pipeline/__tests__/visibility.spec.ts` + +- [ ] **Step 1: Write the failing test.** + +```ts +// src/lib/pipeline/__tests__/visibility.spec.ts +import { describe, expect, it } from 'vitest' + +import { computeVisibility } from '../visibility' + +describe('computeVisibility', () => { + const entities = [ + { id: 1, source: 'wrist-cam-left' }, + { id: 2, source: 'merged-cloud' }, + { id: 3, source: 'object-segmenter' }, + { id: 4, source: 'arm-1' }, // not a stage; not a pipeline source + ] + + it('hides every pipeline-sourced entity not in active', () => { + const result = computeVisibility(entities, new Set(['merged-cloud']), new Set([ + 'wrist-cam-left', + 'merged-cloud', + 'object-segmenter', + ])) + expect(result.hide).toEqual([1, 3]) + expect(result.show).toEqual([2]) + }) + + it('leaves entities whose source is not a stage alone', () => { + const result = computeVisibility(entities, new Set(['merged-cloud']), new Set([ + 'wrist-cam-left', + 'merged-cloud', + 'object-segmenter', + ])) + expect(result.hide).not.toContain(4) + expect(result.show).not.toContain(4) + }) + + it('shows everything when active is empty', () => { + const result = computeVisibility(entities, new Set(), new Set([ + 'wrist-cam-left', + 'merged-cloud', + 'object-segmenter', + ])) + expect(result.hide).toEqual([]) + expect(result.show).toEqual([1, 2, 3]) + }) +}) +``` + +- [ ] **Step 2:** Run `pnpm test -- src/lib/pipeline/__tests__/visibility.spec.ts`. Expected: FAIL. + +- [ ] **Step 3: Write implementation.** + +```ts +// src/lib/pipeline/visibility.ts +import type { StageId } from './types' + +export interface PipelineEntity { + id: E + source: string +} + +export interface VisibilityResult { + show: E[] + hide: E[] +} + +export const computeVisibility = ( + entities: ReadonlyArray>, + active: ReadonlySet, + stageIds: ReadonlySet +): VisibilityResult => { + const show: E[] = [] + const hide: E[] = [] + for (const entity of entities) { + if (!stageIds.has(entity.source)) continue + if (active.size === 0 || active.has(entity.source)) show.push(entity.id) + else hide.push(entity.id) + } + return { show, hide } +} +``` + +Note the empty-active behavior: when no stage is selected, **everything** in the pipeline shows. This matches the user expectation that the page reads as "live monitor for the pipeline" when nothing is solo'd. + +- [ ] **Step 4:** Run the test. Expected: PASS. + +- [ ] **Step 5:** Commit. + +```bash +git add src/lib/pipeline/visibility.ts src/lib/pipeline/__tests__/visibility.spec.ts +git commit -m "feat(pipeline): pure visibility policy helper" +``` + +--- + +### Task 12: Wire visibility effect into the route + +**Files:** +- Modify: `src/routes/pipeline/+page.svelte` + +- [ ] **Step 1:** Add inside the ` + + +``` + +- [ ] **Step 4: Update `PipelineGraph.svelte`** to compute positions and render `EdgeLayer`. Replace the `
    ` block with: + +```svelte + {@const positions = layoutGraph(pipeline.graph())} + {@const orderedStages = [...stages].sort((a, b) => { + const pa = positions.get(a.id)! + const pb = positions.get(b.id)! + return pa.level - pb.level || pa.ordinal - pb.ordinal + })} + +
    + +
      + {#each orderedStages as stage (stage.id)} +
    1. + +
      2. + {/each} +
    +
    +``` + +(Use the same `StageNode` props as before.) + +Add at the top of the script: + +```ts + import { layoutGraph } from './layout' + import EdgeLayer from './EdgeLayer.svelte' +``` + +- [ ] **Step 5:** Run `pnpm test -- src/lib/pipeline/__tests__/PipelineGraph.spec.ts`. Expected: PASS (the previously-passing tests stay green; the new edge test now passes). + +- [ ] **Step 6:** Commit. + +```bash +git add src/lib/pipeline/EdgeLayer.svelte src/lib/pipeline/PipelineGraph.svelte src/lib/pipeline/__tests__/PipelineGraph.spec.ts +git commit -m "feat(pipeline): SVG edge layer with topo-level layout" +``` + +--- + +## Wrap-up + +### Task 18: Wire real `usePartConfig` into the route + +The route currently fakes a `partConfig` shape with empty attributes (Task 7 placeholder). Now that the derive code consumes attributes, swap the placeholder for the real reactive config exposed by `usePartConfig`. + +**Files:** +- Modify: `src/routes/pipeline/+page.svelte` + +- [ ] **Step 1:** Replace the `partConfig` placeholder. The `providePartConfig` call returns nothing usable directly; consume via `usePartConfig`: + +```ts + import { providePartConfig, usePartConfig } from '$lib/hooks/usePartConfig.svelte' + + providePartConfig(() => partID, () => undefined) + const partCfg = usePartConfig() + + const partConfig = $derived({ + components: partCfg.current.components ?? [], + services: (partCfg.current as any).services ?? [], + }) +``` + +(If `usePartConfig` is not exported, add the export in `src/lib/hooks/usePartConfig.svelte.ts` — this is a one-line `export` addition. Keep that change in this same task.) + +- [ ] **Step 2:** Smoke test against a live machine: `pnpm dev`, visit `/pipeline?partID=`. Confirm the graph reflects the actual config (cameras, vision services, transform edges). + +- [ ] **Step 3:** Commit. + +```bash +git add src/routes/pipeline/+page.svelte src/lib/hooks/usePartConfig.svelte.ts +git commit -m "feat(pipeline): consume real machine config for graph derivation" +``` + +--- + +### Task 19: Playwright smoke test + +**Files:** +- Create: `e2e/pipeline.spec.ts` + +- [ ] **Step 1:** Inspect `e2e/` for an existing fixture or test config (`ls e2e/`, `cat playwright.config.ts`). Pattern-match an existing test for the right `goto` URL form, mock setup, and partID handling. + +- [ ] **Step 2: Add the smoke test.** + +```ts +// e2e/pipeline.spec.ts +import { expect, test } from '@playwright/test' + +test.describe('/pipeline', () => { + test('renders the pipeline panel and at least one stage node', async ({ page }) => { + // Adjust partID + any auth bootstrap to match other e2e tests + await page.goto('/pipeline?partID=test') + await expect(page.getByRole('complementary')).toBeVisible() + await expect(page.getByRole('button', { name: /Pipeline|stage|camera/i }).first()).toBeVisible({ + timeout: 10_000, + }) + }) +}) +``` + +(If the fixture machine has no cameras configured, mark this test as `test.fixme` with a TODO and revisit when a fixture exists.) + +- [ ] **Step 3:** `pnpm test:e2e`. Expected: PASS or `fixme` if no fixture machine. + +- [ ] **Step 4:** Commit. + +```bash +git add e2e/pipeline.spec.ts +git commit -m "test(pipeline): playwright smoke for /pipeline route" +``` + +--- + +### Task 20: Final verification + changeset + +**Files:** +- Create: `.changeset/pipeline-view-mvp.md` + +- [ ] **Step 1:** `pnpm check`. Expected: 0 errors. +- [ ] **Step 2:** `pnpm lint`. Expected: clean (or only auto-fixable warnings — apply `pnpm lint --fix` if so). +- [ ] **Step 3:** `pnpm test`. Expected: full suite green. +- [ ] **Step 4:** Add a changeset: + +```markdown +--- +'@viamrobotics/visualization': minor +--- + +Add `/pipeline` route — auto-derives the perception pipeline DAG from the machine +config and spotlights selected stages in the existing 3D scene with per-stage +color tinting. +``` + +(Match the package name and bump kind to repo convention; check `.changeset/` for an existing template.) + +- [ ] **Step 5:** Commit. + +```bash +git add .changeset/pipeline-view-mvp.md +git commit -m "chore(changeset): pipeline view MVP" +``` + +- [ ] **Step 6:** Run `git log --oneline main..HEAD` to confirm the commit graph reads cleanly. + +--- + +## Summary of deferred work + +These remain out-of-scope for this plan and will be addressed in follow-up plans: + +- **2D preview pane** (`StagePreview2D.svelte`, `getImages` integration, detection / classification overlays) — spec phase 4. +- **Override panel + persistence** (`PipelineOverrides.svelte`, `override.ts`, localStorage per `partID`) — spec phase 5. +- `RefreshRates.images` decoupling from `RefreshRates.pointclouds`. +- `getPipelineMetadata` DoCommand convention for custom modules. +- Per-stage `transformPCD` to a common frame for compare-mode alignment (revisit only if alignment problems surface in practice). From cfcfda8ecf754dbd27bff15e399f24c153f798aa Mon Sep 17 00:00:00 2001 From: HipsterBrown Date: Fri, 8 May 2026 14:22:04 -0400 Subject: [PATCH 4/9] docs(plan): address pipeline plan review feedback - Add Task 6.5: widen PartConfig + thread filter props through SceneProviders (fixes the double-provide that defeated the filter and the missing services field) - Rewrite Task 7 route shell to use SceneProviders with filter props and to mount Settings / RefreshRate inside the Canvas snippet via domPortal (fixes the runtime crash from Settings outside Threlte context) - Drop OriginalColors from the new traits (Task 13's renderer-side override doesn't need it) - Gate the tint effect on entity.has(traits.Points) per spec scope - Note Struct.toJson() shape assumption for the attribute-value walker - Replace the now-obsolete Task 18 (real-config wiring) with a manual e2e gate - Rename e2e file to *.test.ts to match local convention --- .../plans/2026-05-08-pipeline-view.md | 231 ++++++++++++------ 1 file changed, 159 insertions(+), 72 deletions(-) diff --git a/docs/superpowers/plans/2026-05-08-pipeline-view.md b/docs/superpowers/plans/2026-05-08-pipeline-view.md index 21d272a0e..2c197cf76 100644 --- a/docs/superpowers/plans/2026-05-08-pipeline-view.md +++ b/docs/superpowers/plans/2026-05-08-pipeline-view.md @@ -35,13 +35,15 @@ - `src/lib/pipeline/__tests__/PipelineGraph.spec.ts` - `src/routes/pipeline/+page.svelte` — composition only; mounts providers, canvas, graph panel, effects. - `src/routes/pipeline/+page.ts` — extracts `partID` from URL params. -- `e2e/pipeline.spec.ts` — Playwright smoke test. +- `e2e/pipeline.test.ts` — Playwright smoke test (matches local `*.test.ts` convention). **Modified** -- `src/lib/ecs/traits.ts` — add `PipelineSource(stageId: string)`, `PipelineTint({r,g,b})`, `OriginalColors(Uint8Array)`. -- `src/lib/hooks/usePointclouds.svelte.ts` — accept optional `filter`, stamp `PipelineSource` on spawn. -- `src/lib/hooks/usePointcloudObjects.svelte.ts` — same shape. +- `src/lib/ecs/traits.ts` — add `PipelineSource(stageId: string)`, `PipelineTint({r,g,b})`. +- `src/lib/hooks/usePointclouds.svelte.ts` — accept optional `filter`, stamp `PipelineSource` on spawn, export `applyFilter`. +- `src/lib/hooks/usePointcloudObjects.svelte.ts` — same shape (re-uses `applyFilter`). +- `src/lib/hooks/usePartConfig.svelte.ts` — widen `PartConfig` to expose `attributes` on components and a typed `services[]`. +- `src/lib/components/SceneProviders.svelte` — accept optional `pointcloudsFilter` / `pointcloudObjectsFilter` props, threaded into the existing `providePointclouds` / `providePointcloudObjects` calls. - `src/lib/components/Entities/Points.svelte` — read `PipelineTint` and prefer it over geometry's `color` attribute when present. - `.gitignore` — add `.superpowers/`. @@ -834,13 +836,115 @@ git commit -m "feat(pipeline): PipelineGraph panel rendering nodes" --- +### Task 6.5: Widen `PartConfig` + thread filter props through `SceneProviders` + +Two prerequisite shared-code edits the route depends on. Bundled here so the route in Task 7 mounts cleanly. **No tests** in this task — it's wiring; behavior is validated by Task 9 / Task 12 / Task 18 against real config. + +**Files:** +- Modify: `src/lib/hooks/usePartConfig.svelte.ts` +- Modify: `src/lib/components/SceneProviders.svelte` + +- [ ] **Step 1: Widen `PartConfig`.** In `src/lib/hooks/usePartConfig.svelte.ts`, replace the `PartConfig` interface with: + +```ts +export interface PartConfigComponent { + name: string + api?: string + model?: string + attributes?: Record + frame?: Frame +} + +export interface PartConfigService { + name: string + api?: string + model?: string + attributes?: Record +} + +export interface PartConfig { + components: PartConfigComponent[] + services?: PartConfigService[] + fragment_mods?: { + fragment_id: string + // eslint-disable-next-line @typescript-eslint/no-explicit-any + mods: any[] + }[] +} +``` + +`Struct.toJson()` already returns these fields when the underlying config has them; we're just exposing them in the typed surface. + +Also export `usePartConfig` as a named export if it isn't already (the existing file may only export `providePartConfig`): + +```ts +export const usePartConfig = (): PartConfigContext => { + const ctx = getContext(key) + if (!ctx) throw new Error('usePartConfig called outside providePartConfig') + return ctx +} +``` + +(Check first: if `usePartConfig` already exists, leave it alone.) + +- [ ] **Step 2:** `pnpm check`. Expected: PASS — no consumers should regress because the new fields are optional. + +- [ ] **Step 3: Thread filter props through `SceneProviders`.** In `src/lib/components/SceneProviders.svelte`: + +```svelte + +``` + +(The `{ filter }` option arrives in Tasks 9 + 10. Check this task in **after** those tasks — see "Task ordering" below.) + +> **Task ordering note.** Tasks 9 and 10 add the `filter` option to the hook signatures. This task (6.5) is logically a *prerequisite* for Task 7's route shell, but its `SceneProviders` edit *uses* the option. Implement the prerequisite split this way: do Step 1 (widen `PartConfig`) now, then jump to Tasks 9 + 10 to add the `filter` plumbing, then return to do Step 3 (thread filter props through `SceneProviders`), then proceed to Task 7. The phase-2 ordering in the plan reflects narrative order; this is the build order. + +- [ ] **Step 4:** `pnpm check`. Expected: PASS. + +- [ ] **Step 5:** Commit (split into two commits — one per file — for clean review): + +```bash +git add src/lib/hooks/usePartConfig.svelte.ts +git commit -m "feat(hooks): widen PartConfig with attributes + services" + +git add src/lib/components/SceneProviders.svelte +git commit -m "feat(scene): thread pointcloud filter props through SceneProviders" +``` + +--- + ### Task 7: `/pipeline` route shell **Files:** - Create: `src/routes/pipeline/+page.ts` - Create: `src/routes/pipeline/+page.svelte` -The page composes providers, the canvas, the graph panel, and the existing `Settings`/`RefreshRate` overlays. Property fetching is wired via `useResourceNames` + `createResourceQuery` (mirroring `usePointclouds`). No scene filter yet — that's Task 11. +The page reuses `` for the full provider tree (frames, geometries, draw API, hierarchy, **and** the pointcloud hooks with our filter), and mounts `Settings` / `RefreshRate` inside the Canvas snippet via the existing `domPortal` pattern (matching `App.svelte`). Property fetching for graph derivation is wired via `useResourceNames` + `createResourceQuery` (mirroring `usePointclouds`). No scene filter behavior yet — that lands in Task 12. - [ ] **Step 1:** Add `+page.ts`. @@ -876,11 +980,10 @@ export const load: PageLoad = ({ url }) => { import Settings from '$lib/components/overlay/settings/Settings.svelte' import { provideWorld } from '$lib/ecs' import { provideEnvironment } from '$lib/hooks/useEnvironment.svelte' - import { providePartConfig } from '$lib/hooks/usePartConfig.svelte' + import { providePartConfig, usePartConfig } from '$lib/hooks/usePartConfig.svelte' import { createPartIDContext } from '$lib/hooks/usePartID.svelte' import { provideSettings } from '$lib/hooks/useSettings.svelte' - import { providePointclouds } from '$lib/hooks/usePointclouds.svelte' - import { providePointcloudObjects } from '$lib/hooks/usePointcloudObjects.svelte' + import { domPortal } from '$lib/portal' import { buildGraph, type PropertiesMap, type StageProperties } from '$lib/pipeline/derive' import PipelineGraph from '$lib/pipeline/PipelineGraph.svelte' import { providePipeline } from '$lib/pipeline/usePipeline.svelte' @@ -893,6 +996,7 @@ export const load: PageLoad = ({ url }) => { provideEnvironment() createPartIDContext(() => partID) providePartConfig(() => partID, () => undefined) + const partCfg = usePartConfig() const cameras = useResourceNames(() => partID, 'camera') const visionServices = useResourceNames(() => partID, 'vision') @@ -946,55 +1050,53 @@ export const load: PageLoad = ({ url }) => { return map }) - const partConfig = $derived(/* placeholder until derive uses it; pulled via usePartConfig */ { - components: cameras.current.map((c) => ({ - name: c.name, - api: 'rdk:component:camera', - model: '', - attributes: {}, - })), - services: visionServices.current.map((s) => ({ - name: s.name, - api: 'rdk:service:vision', - model: '', - attributes: {}, - })), + const partConfig = $derived({ + components: partCfg.current.components ?? [], + services: partCfg.current.services ?? [], }) const graph = $derived(buildGraph(partConfig, properties)) const pipelineCtx = providePipeline({ graph: () => graph }) - const filter = () => new Set(graph.stages.map((s) => s.id)) + const stageIdFilter = () => new Set(graph.stages.map((s) => s.id)) - providePointclouds(() => partID, { filter }) - providePointcloudObjects(() => partID, { filter }) + let root = $state.raw() -
    +
    - + {#snippet children({ focus: _focus })} + + +
    +
    + +
    + +
    {/snippet}     - -
    - -
    - -
    ``` -- [ ] **Step 3:** `pnpm check`. Expected: PASS. (TypeScript may grumble about `providePointclouds` `options` param until Task 9; if blocking, temporarily remove the `options` arg here and restore in Task 9.) +- [ ] **Step 3:** `pnpm check`. Expected: PASS. (TypeScript will grumble about the `pointcloudsFilter` / `pointcloudObjectsFilter` props on `SceneProviders` until Task 6.5 Step 3 is done — see the **Task ordering note** in Task 6.5. If you implemented in plan order, jump back and finish Task 6.5 Step 3 now.) -- [ ] **Step 4:** `pnpm dev` and visit `http://localhost:5173/pipeline?partID=`. Expected: empty graph (no derive-edges yet, no scene filter), Threlte canvas renders, Settings + RefreshRate overlays present. Confirm in a browser. +- [ ] **Step 4:** `pnpm dev` and visit `http://localhost:5173/pipeline?partID=`. Expected: graph populated with derived stages (no edges yet), Threlte canvas renders, Settings + RefreshRate overlays present. Confirm in a browser. Until Task 12 lands, the scene shows every pipeline stage's points; clicking a node has no scene effect yet (only graph state changes). - [ ] **Step 5:** Commit. @@ -1023,11 +1125,10 @@ export const PipelineSource = trait(() => '') /** Per-stage tint applied in pipeline view; rendered in place of geometry color. */ export const PipelineTint = trait({ r: 0, g: 0, b: 0 }) - -/** Snapshot of a geometry's original `color` attribute bytes, used to restore tint→native. */ -export const OriginalColors = trait(() => new Uint8Array() as Uint8Array) ``` +(`OriginalColors` from spec §5 is not added — Task 13 takes a renderer-side override path that doesn't need it. If a future renderer adopts `PipelineTint` and *does* need to mutate geometry, add the trait then.) + - [ ] **Step 2:** `pnpm check`. Expected: PASS. - [ ] **Step 3:** Commit. @@ -1380,6 +1481,9 @@ Also ensure `vertexColors` is disabled when `pipelineTint` is set: in the `$effe const useNative = pipelineCtx.useNativeColors() const active = pipelineCtx.active() for (const entity of list) { + // Spec §5 limits tint scope to Points-bearing entities. Other entity + // kinds (geometry boxes, frames) keep their native color. + if (!entity.has(traits.Points)) continue const source = entity.get(traits.PipelineSource) ?? '' const isActive = active.has(source) && !useNative if (isActive) { @@ -1547,7 +1651,7 @@ describe('buildGraph: edges via attribute-value matching', () => { - [ ] **Step 2:** Run the new tests. Expected: FAIL. -- [ ] **Step 3: Add walker to `derive.ts`.** Inside `buildGraph`, after the stage loop: +- [ ] **Step 3: Add walker to `derive.ts`.** Inside `buildGraph`, after the stage loop. The walker assumes `attributes` is the plain JSON shape produced by `Struct.toJson()` (objects, arrays, strings, numbers, booleans, null) — no `Map` / `Set` / class instances. Source: `usePartConfig.svelte.ts` does `config.current.toJson()` before exposing `current`, so the route's `partConfig.components[].attributes` is already in this shape. ```ts const stageIds = new Set(stages.map((s) => s.id)) @@ -1940,56 +2044,39 @@ git commit -m "feat(pipeline): SVG edge layer with topo-level layout" ## Wrap-up -### Task 18: Wire real `usePartConfig` into the route +### Task 18: End-to-end smoke against a live machine -The route currently fakes a `partConfig` shape with empty attributes (Task 7 placeholder). Now that the derive code consumes attributes, swap the placeholder for the real reactive config exposed by `usePartConfig`. +By this point, Task 7 already consumes the real `partConfig`, Task 14 wires the attribute walker, and Task 17 renders edges. This task is a manual verification gate, not new code. -**Files:** -- Modify: `src/routes/pipeline/+page.svelte` - -- [ ] **Step 1:** Replace the `partConfig` placeholder. The `providePartConfig` call returns nothing usable directly; consume via `usePartConfig`: - -```ts - import { providePartConfig, usePartConfig } from '$lib/hooks/usePartConfig.svelte' - - providePartConfig(() => partID, () => undefined) - const partCfg = usePartConfig() - - const partConfig = $derived({ - components: partCfg.current.components ?? [], - services: (partCfg.current as any).services ?? [], - }) -``` - -(If `usePartConfig` is not exported, add the export in `src/lib/hooks/usePartConfig.svelte.ts` — this is a one-line `export` addition. Keep that change in this same task.) - -- [ ] **Step 2:** Smoke test against a live machine: `pnpm dev`, visit `/pipeline?partID=`. Confirm the graph reflects the actual config (cameras, vision services, transform edges). - -- [ ] **Step 3:** Commit. - -```bash -git add src/routes/pipeline/+page.svelte src/lib/hooks/usePartConfig.svelte.ts -git commit -m "feat(pipeline): consume real machine config for graph derivation" -``` +- [ ] **Step 1:** `pnpm dev`. Open `http://localhost:5173/pipeline?partID=` against a fixture machine that has at least one transform-camera and one vision service. +- [ ] **Step 2:** Confirm: + - Graph nodes match the machine's cameras + vision services (no arms, motors, sensors). + - Edges from upstream cameras to transform cameras are drawn. + - Edges from cameras to vision services are drawn (when configured via `camera_name`-shaped attributes). + - Click a stage → only that stage's points render in scene with its tint color. + - Shift-click a second stage → both render with distinct tints. + - Toggle "Native colors" → original RGB returns. + - Edit machine config (in another tab) to add a stage; the graph updates without reload. +- [ ] **Step 3:** No commit; this is a manual check. If anything fails, stop and triage; do not advance. --- ### Task 19: Playwright smoke test **Files:** -- Create: `e2e/pipeline.spec.ts` +- Create: `e2e/pipeline.test.ts` -- [ ] **Step 1:** Inspect `e2e/` for an existing fixture or test config (`ls e2e/`, `cat playwright.config.ts`). Pattern-match an existing test for the right `goto` URL form, mock setup, and partID handling. +- [ ] **Step 1:** Inspect `e2e/` for an existing test (e.g. `cat e2e/arm.test.ts`) and `cat playwright.config.ts`. Pattern-match the `goto` URL form, fixture setup, and partID handling. - [ ] **Step 2: Add the smoke test.** ```ts -// e2e/pipeline.spec.ts +// e2e/pipeline.test.ts import { expect, test } from '@playwright/test' test.describe('/pipeline', () => { test('renders the pipeline panel and at least one stage node', async ({ page }) => { - // Adjust partID + any auth bootstrap to match other e2e tests + // Adjust partID + any auth bootstrap to match e2e/arm.test.ts await page.goto('/pipeline?partID=test') await expect(page.getByRole('complementary')).toBeVisible() await expect(page.getByRole('button', { name: /Pipeline|stage|camera/i }).first()).toBeVisible({ @@ -2006,7 +2093,7 @@ test.describe('/pipeline', () => { - [ ] **Step 4:** Commit. ```bash -git add e2e/pipeline.spec.ts +git add e2e/pipeline.test.ts git commit -m "test(pipeline): playwright smoke for /pipeline route" ``` From 37b46f78d60bb77c605c49d47c85ec2d84868c0a Mon Sep 17 00:00:00 2001 From: HipsterBrown Date: Fri, 8 May 2026 14:25:32 -0400 Subject: [PATCH 5/9] docs(plan): RefreshRate prop wiring + dashboard caveat - Provide id/label/onManualRefetch to RefreshRate (pointclouds + vision) - Note that Settings has no gear button on /pipeline because Dashboard is not mounted; the panel itself works once opened --- .../plans/2026-05-08-pipeline-view.md | 21 ++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/plans/2026-05-08-pipeline-view.md b/docs/superpowers/plans/2026-05-08-pipeline-view.md index 2c197cf76..925655a27 100644 --- a/docs/superpowers/plans/2026-05-08-pipeline-view.md +++ b/docs/superpowers/plans/2026-05-08-pipeline-view.md @@ -983,6 +983,8 @@ export const load: PageLoad = ({ url }) => { import { providePartConfig, usePartConfig } from '$lib/hooks/usePartConfig.svelte' import { createPartIDContext } from '$lib/hooks/usePartID.svelte' import { provideSettings } from '$lib/hooks/useSettings.svelte' + import { usePointclouds } from '$lib/hooks/usePointclouds.svelte' + import { usePointcloudObjects } from '$lib/hooks/usePointcloudObjects.svelte' import { domPortal } from '$lib/portal' import { buildGraph, type PropertiesMap, type StageProperties } from '$lib/pipeline/derive' import PipelineGraph from '$lib/pipeline/PipelineGraph.svelte' @@ -1081,9 +1083,20 @@ export const load: PageLoad = ({ url }) => { + {@const pointclouds = usePointclouds()} + {@const pointcloudObjects = usePointcloudObjects()}
    -
    - +
    + pointclouds.refetch()} + /> + pointcloudObjects.refetch()} + />
    @@ -1096,7 +1109,9 @@ export const load: PageLoad = ({ url }) => { - [ ] **Step 3:** `pnpm check`. Expected: PASS. (TypeScript will grumble about the `pointcloudsFilter` / `pointcloudObjectsFilter` props on `SceneProviders` until Task 6.5 Step 3 is done — see the **Task ordering note** in Task 6.5. If you implemented in plan order, jump back and finish Task 6.5 Step 3 now.) -- [ ] **Step 4:** `pnpm dev` and visit `http://localhost:5173/pipeline?partID=`. Expected: graph populated with derived stages (no edges yet), Threlte canvas renders, Settings + RefreshRate overlays present. Confirm in a browser. Until Task 12 lands, the scene shows every pipeline stage's points; clicking a node has no scene effect yet (only graph state changes). +- [ ] **Step 4:** `pnpm dev` and visit `http://localhost:5173/pipeline?partID=`. Expected: graph populated with derived stages (no edges yet), Threlte canvas renders, RefreshRate controls visible top-right. Until Task 12 lands, the scene shows every pipeline stage's points; clicking a node has no scene effect yet (only graph state changes). + +> **Settings UX caveat.** `Settings.svelte` opens its `FloatingPanel` via a gear button portalled into ``, which lives in the (deliberately omitted) `` overlay. That means the gear button doesn't render on this route. The Settings panel itself functions correctly once opened — but you may need to add a Pipeline-route-specific opener (e.g. a small button in the `PipelineGraph` header) in a follow-up if Settings access becomes important. Not a v1 blocker. - [ ] **Step 5:** Commit. From 6437569cb83431ffec031e1d174907738801210e Mon Sep 17 00:00:00 2001 From: HipsterBrown Date: Tue, 21 Jul 2026 09:54:01 -0400 Subject: [PATCH 6/9] feat(worldstate): pure reconcile diff for snapshot vs rendered set --- .../__tests__/reconcileWorldState.spec.ts | 60 +++++++++++++++++++ src/lib/hooks/reconcileWorldState.ts | 33 ++++++++++ 2 files changed, 93 insertions(+) create mode 100644 src/lib/hooks/__tests__/reconcileWorldState.spec.ts create mode 100644 src/lib/hooks/reconcileWorldState.ts diff --git a/src/lib/hooks/__tests__/reconcileWorldState.spec.ts b/src/lib/hooks/__tests__/reconcileWorldState.spec.ts new file mode 100644 index 000000000..d5cda6f11 --- /dev/null +++ b/src/lib/hooks/__tests__/reconcileWorldState.spec.ts @@ -0,0 +1,60 @@ +import { describe, expect, it } from 'vitest' + +import { reconcileWorldState } from '$lib/hooks/reconcileWorldState' + +describe('reconcileWorldState', () => { + it.each<{ + name: string + snapshot: Iterable + rendered: Iterable + toAdd: string[] + toRemove: string[] + }>([ + { + name: 'adds snapshot-only UUIDs and removes rendered-only UUIDs, keeping the intersection', + snapshot: ['B', 'C'], + rendered: ['A', 'B'], + toAdd: ['C'], + toRemove: ['A'], + }, + { + name: 'adds everything on initial mount (nothing rendered yet)', + snapshot: ['A', 'B'], + rendered: [], + toAdd: ['A', 'B'], + toRemove: [], + }, + { + name: 'is a no-op when snapshot and rendered match', + snapshot: ['A'], + rendered: ['A'], + toAdd: [], + toRemove: [], + }, + { + name: 'removes everything when the snapshot is empty', + snapshot: [], + rendered: ['A', 'B'], + toAdd: [], + toRemove: ['A', 'B'], + }, + { + name: 'accepts any iterable and de-duplicates the snapshot in toAdd', + snapshot: new Set(['A', 'B']), + rendered: new Set(['A']), + toAdd: ['B'], + toRemove: [], + }, + { + name: 'preserves snapshot iteration order in toAdd', + snapshot: ['C', 'A', 'B'], + rendered: [], + toAdd: ['C', 'A', 'B'], + toRemove: [], + }, + ])('$name', ({ snapshot, rendered, toAdd, toRemove }) => { + const result = reconcileWorldState(snapshot, rendered) + expect(result.toAdd).toEqual(toAdd) + expect(result.toRemove).toEqual(toRemove) + }) +}) diff --git a/src/lib/hooks/reconcileWorldState.ts b/src/lib/hooks/reconcileWorldState.ts new file mode 100644 index 000000000..ebd76d6c5 --- /dev/null +++ b/src/lib/hooks/reconcileWorldState.ts @@ -0,0 +1,33 @@ +export interface WorldStateReconciliation { + /** UUIDs present in the fresh snapshot but not yet rendered. */ + toAdd: string[] + /** UUIDs currently rendered but absent from the fresh snapshot. */ + toRemove: string[] +} + +/** + * Diffs a fresh world-state snapshot against the currently rendered entity set. + * Pure: callers own spawning/destroying and tombstone bookkeeping. Preserves + * snapshot iteration order in `toAdd` and rendered iteration order in `toRemove`. + * Duplicate UUIDs within either input collapse to a single entry, so a + * duplicated snapshot UUID appears at most once in `toAdd`. + */ +export const reconcileWorldState = ( + snapshotUUIDs: Iterable, + renderedUUIDs: Iterable +): WorldStateReconciliation => { + const snapshot = new Set(snapshotUUIDs) + const rendered = new Set(renderedUUIDs) + + const toAdd: string[] = [] + for (const uuid of snapshot) { + if (!rendered.has(uuid)) toAdd.push(uuid) + } + + const toRemove: string[] = [] + for (const uuid of rendered) { + if (!snapshot.has(uuid)) toRemove.push(uuid) + } + + return { toAdd, toRemove } +} From a1fc7d8e37a5160bce9263ddf2788f72f36d6b3f Mon Sep 17 00:00:00 2001 From: HipsterBrown Date: Tue, 21 Jul 2026 10:02:02 -0400 Subject: [PATCH 7/9] feat(worldstate): re-subscribe and reconcile on config-revision change An AlwaysRebuild world_state_store reconfigure swaps the backing resource instance under the same name, so client.current stays stable and neither the transform stream nor the cached snapshot query would otherwise notice. Read machine-status config revision at the provider level and thread it into each createWorldState instance to re-open the stream and reconcile the entity set against a fresh snapshot on every revision change, mirroring useFrames's existing revision-driven refetch pattern. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01BkDg3q9GxPnv4FUoA6rQ2K --- src/lib/hooks/useWorldState.svelte.ts | 56 ++++++++++++++++++++++++--- 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/src/lib/hooks/useWorldState.svelte.ts b/src/lib/hooks/useWorldState.svelte.ts index 8caff5b97..3c6fb0c6e 100644 --- a/src/lib/hooks/useWorldState.svelte.ts +++ b/src/lib/hooks/useWorldState.svelte.ts @@ -11,8 +11,10 @@ import { import { createResourceClient, createResourceQuery, + useMachineStatus, useResourceNames, } from '@viamrobotics/svelte-sdk' +import { untrack } from 'svelte' import { Matrix4 } from 'three' import { asFloat32Array, inMeters } from '$lib/buffer' @@ -20,6 +22,7 @@ import { createChunkLoader, type EntityChunk } from '$lib/chunking' import { drawTransform, updateMetadata } from '$lib/draw' import { hierarchy, traits, useWorld } from '$lib/ecs' import { isPointCloud } from '$lib/geometry' +import { reconcileWorldState } from '$lib/hooks/reconcileWorldState' import { metadataFromStruct } from '$lib/metadata' import { createPose, poseToMatrix } from '$lib/transform' @@ -32,6 +35,8 @@ type TransformEvent = TransformChangeEvent & { export const provideWorldStates = () => { const partID = usePartID() + const machineStatus = useMachineStatus(() => partID.current) + const revision = $derived(machineStatus.current?.config?.revision) const resourceNames = useResourceNames(() => partID.current, 'world_state_store') const clients = $derived( resourceNames.current.map(({ name }) => @@ -47,7 +52,7 @@ export const provideWorldStates = () => { const cleanups: (() => void)[] = [] for (const client of clients) { - cleanups.push(createWorldState(client)) + cleanups.push(createWorldState(client, () => revision)) } return () => { @@ -127,11 +132,16 @@ const decodeWorldStateChunk = (response: unknown, fallbackStart: number): Entity return { start, positions, colors, opacities, done } } -const createWorldState = (client: { current: WorldStateStoreClient | undefined }) => { +const createWorldState = ( + client: { current: WorldStateStoreClient | undefined }, + revision: () => string | undefined +) => { const { invalidate } = useThrelte() const world = useWorld() const relationships = useRelationships() + const UNRECONCILED = Symbol('unreconciled') + const entities = new Map() // UUIDs the stream has removed; guards against a stale initial snapshot or a // self-heal fetch re-creating an entity the server has already deleted. @@ -253,7 +263,10 @@ const createWorldState = (client: { current: WorldStateStoreClient | undefined } } } - let initialized = false + // Tracks which config revision the snapshot has been reconciled against, so the + // snapshot effect runs once per revision (initial mount + each reconfigure) rather + // than on every incidental query settle. + let reconciledRevision: string | undefined | typeof UNRECONCILED = UNRECONCILED let flushScheduled = false let rafId = 0 let pendingEvents: TransformEvent[] = [] @@ -270,6 +283,17 @@ const createWorldState = (client: { current: WorldStateStoreClient | undefined } }) ) + // Force a fresh snapshot on reconfigure. The createResourceQuery key omits the + // config revision, so a same-name rebuild would otherwise be served stale cache. + $effect(() => { + const rev = revision() + if (rev === undefined) return + if (reconciledRevision === UNRECONCILED) return // initial mount handled below + untrack(() => { + void listUUIDs.refetch() + }) + }) + const applyEvents = (events: TransformEvent[]) => { for (const event of events) { if (event.changeType === TransformChangeType.ADDED) { @@ -302,19 +326,33 @@ const createWorldState = (client: { current: WorldStateStoreClient | undefined } $effect(() => { if (!getTransformQueries) return - if (initialized) return if (getTransformQueries.some((query) => query?.isLoading)) return + const rev = revision() + if (reconciledRevision === rev) return + const transforms = getTransformQueries .flatMap((query) => query?.data) .filter((transform) => transform !== undefined) - for (const transform of transforms) { + const byUUID = new Map(transforms.map((t) => [t.uuidString, t])) + const { toAdd, toRemove } = reconcileWorldState(byUUID.keys(), entities.keys()) + + for (const uuid of toRemove) { + destroyEntity(uuid) + } + + for (const uuid of toAdd) { + const transform = byUUID.get(uuid) + if (!transform) continue + // The fresh snapshot is authoritative: a UUID it reports as present must not + // stay tombstoned from an earlier removal, or spawnEntity would refuse it. + removedUUIDs.delete(uuid) spawnEntity(transform) } invalidate() - initialized = true + reconciledRevision = rev }) /** @@ -343,6 +381,12 @@ const createWorldState = (client: { current: WorldStateStoreClient | undefined } } $effect(() => { + // Re-subscribe on a config-revision change: an AlwaysRebuild reconfigure + // swaps the backing resource instance under the same name, ending the old + // gRPC stream cleanly. `client.current` is a stable reference so this effect + // would not otherwise re-fire; reading `revision()` gives it that dependency. + revision() + if (!client.current) return const controller = new AbortController() From 70ad08e0b1c0da6817d12ee1019ca252b906174c Mon Sep 17 00:00:00 2001 From: HipsterBrown Date: Tue, 21 Jul 2026 10:12:12 -0400 Subject: [PATCH 8/9] fix(worldstate): reconcile against fresh snapshot, not the revision edge The reconcile effect was reading revision() tracked, so on a reconfigure it fired synchronously in the same flush as the refetch effect - before listUUIDs.refetch() (a background refetch) had landed new data. It diffed against the still-stale snapshot (a no-op) and latched reconciledRevision to the new revision, permanently blocking the real diff once the fresh snapshot arrived. Read revision untracked in the reconcile effect so it fires on snapshot-data settling instead of the revision flip; the refetch effect continues to own the revision edge. Also: hoist the UNRECONCILED sentinel to module scope (it doesn't need per-instance allocation), and sharpen comments on the accepted mount-ordering redundant-refetch case and the snapshot-vs-stream tombstone override. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01BkDg3q9GxPnv4FUoA6rQ2K --- src/lib/hooks/useWorldState.svelte.ts | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/src/lib/hooks/useWorldState.svelte.ts b/src/lib/hooks/useWorldState.svelte.ts index 3c6fb0c6e..d40cdeb5f 100644 --- a/src/lib/hooks/useWorldState.svelte.ts +++ b/src/lib/hooks/useWorldState.svelte.ts @@ -33,6 +33,8 @@ type TransformEvent = TransformChangeEvent & { transform: TransformWithUUID } +const UNRECONCILED = Symbol('unreconciled') + export const provideWorldStates = () => { const partID = usePartID() const machineStatus = useMachineStatus(() => partID.current) @@ -140,8 +142,6 @@ const createWorldState = ( const world = useWorld() const relationships = useRelationships() - const UNRECONCILED = Symbol('unreconciled') - const entities = new Map() // UUIDs the stream has removed; guards against a stale initial snapshot or a // self-heal fetch re-creating an entity the server has already deleted. @@ -285,6 +285,13 @@ const createWorldState = ( // Force a fresh snapshot on reconfigure. The createResourceQuery key omits the // config revision, so a same-name rebuild would otherwise be served stale cache. + // This effect owns the revision edge (reads it tracked); the reconcile effect + // below deliberately reads revision untracked so it reacts to the snapshot + // settling instead of racing this refetch. + // + // A rare mount ordering (snapshot settles before machineStatus loads) can cause + // one redundant, idempotent refetch on the first real revision; accepted over a + // guard that would risk skipping a genuine reconfigure. $effect(() => { const rev = revision() if (rev === undefined) return @@ -328,7 +335,12 @@ const createWorldState = ( if (!getTransformQueries) return if (getTransformQueries.some((query) => query?.isLoading)) return - const rev = revision() + // Read revision untracked: the refetch effect owns the revision edge and kicks + // off listUUIDs.refetch(); this effect must fire on the FRESH snapshot settling + // (getTransformQueries recomputing), not on the revision flip — otherwise it + // reconciles against the pre-refetch snapshot and latches the revision, blocking + // the real snapshot when it lands. + const rev = untrack(() => revision()) if (reconciledRevision === rev) return const transforms = getTransformQueries @@ -345,8 +357,10 @@ const createWorldState = ( for (const uuid of toAdd) { const transform = byUUID.get(uuid) if (!transform) continue - // The fresh snapshot is authoritative: a UUID it reports as present must not - // stay tombstoned from an earlier removal, or spawnEntity would refuse it. + // The fresh snapshot is authoritative for presence: a UUID it reports as + // present must not stay tombstoned, or spawnEntity would refuse it. Note this + // can override a newer stream REMOVED for the same UUID (snapshot wins) — an + // accepted MVP boundary; see the design doc's snapshot-vs-stream race note. removedUUIDs.delete(uuid) spawnEntity(transform) } From 1d2408847dd4d413c4990f439403c0cdcdc31ab7 Mon Sep 17 00:00:00 2001 From: HipsterBrown Date: Tue, 21 Jul 2026 10:15:49 -0400 Subject: [PATCH 9/9] docs(changeset): worldstate reconfigure recovery Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01BkDg3q9GxPnv4FUoA6rQ2K --- .changeset/wss-reconfigure-resubscribe.md | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 .changeset/wss-reconfigure-resubscribe.md diff --git a/.changeset/wss-reconfigure-resubscribe.md b/.changeset/wss-reconfigure-resubscribe.md new file mode 100644 index 000000000..2af1c2914 --- /dev/null +++ b/.changeset/wss-reconfigure-resubscribe.md @@ -0,0 +1,5 @@ +--- +'@viamrobotics/motion-tools': patch +--- + +Fix world state entities going stale after a machine reconfigure