Skip to content

Remove obsolete Gateway connection records and per-Sandbox routes - #16365

Open
agentydragon-agent wants to merge 10 commits into
agentydragon:develfrom
agentydragon-agent:gateway-route-removal
Open

agentydragon-agent wants to merge 10 commits into
agentydragon:develfrom
agentydragon-agent:gateway-route-removal

Conversation

@agentydragon-agent

@agentydragon-agent agentydragon-agent commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Merge only after #16345's receiver pin and live Runtime cycle are verified, #16361's Gateway image runs on every replica, AND every #16362 caller process has rolled to the stable Gateway Service.

Remove Gateway connection Lease/route-epoch allocation and EndpointSlice publication, and the Sandbox Controller's per-Sandbox control-Service creation/cache/watch. Gateway replicas continue their independent zero-epoch Runtime observations and calls; Runtime fences Sandbox UID, workload Pod UID and boot ID. This completes the route cutover without changing Dispatcher delivery ownership or automatically replaying uncertain calls. Existing legacy route objects are left inert for the rollback window.

Stacked on #16361 and #16362; review only 30932e9, 7db713f and b1e8cc9. All PRs target upstream devel from the requested fork; GitHub native stacks do not support that cross-fork chain. Rebase out parent commits after they land. Required permissions and obsolete accepted settings remain until this Gateway and Controller image is pinned and the old rollback window closes; no new settings key is added.

Rollout and rollback

Before merge, verify all App, Notification, Controller, Gateway/Dispatcher and central sampler caller processes have restarted with a stable runner-gateway.{namespace}.svc:8080 target. Inspect live Pods and mounted/decoded configuration, not just the current ConfigMaps. Confirm no external caller still targets <sandbox>-control. #16361 documents the live Pod/KubeVirt/Firecracker Runtime provenance checks; a pin alone or Firecracker RAM restore does not prove an upgraded Runtime.

Rollback by reverting this PR and restoring the route-publishing Gateway and control-Service-creating Controller image pins; keep stable-Service callers and #16345's receiver. Once Flux applies it, verify the Gateway/Controller rollouts, legacy route publication, Controller connected/bootstrapped readiness, idle suspend/resume and quiesce, retained History and a subsequent Session input. To roll back further, restore all per-Sandbox caller configurations and wait for those processes to roll before restoring the old exclusive Gateway. Do not roll Runtime back underneath zero-epoch watches. Reconcile uncertain mutations with their existing command ID, never a fresh automatic replay.

Validation

The browser failures were caused by the scripted Runtime rejecting zero-epoch WatchState requests. The fixture now accepts the receiver protocol and retains Sandbox/incarnation and peer authentication checks. The latest 40 indexed devel results for each browser target and the Gateway budget suite passed (raw GetTargetHistory, the query behind bbapi target history; CLI unavailable in this image); that index includes cache reuse.

Focused RBE invocation 7c298eca-e3b6-4abf-8ad3-4e10ec4f7125 passed //agentplane/app:test_thread_browser (all 16 shards), //agentplane/app:test_thread_window_browser, //agentplane/runner_gateway/internal/gateway:gateway_test and :queue_runner_integration_test, including the changed fixture library lint. The budget failure was a premature continuity assertion: the recovery probe had yielded a running code-mode cell while the sampler channel reconnected. The scenario now follows that same cell with wait, retaining native-session continuity, exact-once marker and next-input assertions. Focused invocation 8124457c-97ce-404e-8d3f-97d436a824c9 passed //agentplane/system_tests/suites/mcp:test_gateway_budget (348.6s) and the changed scenario library lint. No mutation replay, flake rerun, skipped test or timeout increase. These focused runs used the pre-rebase heads on 9a955bd3. After those fixes, both browser targets and the Gateway budget suite also passed in PR CI 18a3d6e1-f93d-5953-9973-e8cd47f22473 on the pre-rebase head. Full CI after rebasing onto 6e591f38 is pending; the range-diff confirms no patch changes. pre-commit run --from-ref origin/devel --to-ref HEAD passed after the rebase.

Required k3s suites: //agentplane/system_tests/suites:test_pod, :test_session_deletion, :test_firecracker. Named scenarios cover bootstrap (test_a_preset_launches_an_initialized_editable_codex_thread), idle suspend (test_operator_resumes_an_idle_pod_without_granting_its_identity_lifecycle_control), quiesce/History/next input (test_two_harness_threads_keep_their_context_after_sandbox_suspend_resume), deletion fencing (test_session_command_cannot_cross_deletion, test_managed_cli_history_identity_uid_scope) and Firecracker (test_firecracker_preserves_live_guest). Central Service-routing/recovery coverage uses //agentplane/system_tests/suites/mcp:test_services, :test_gateway_crash, :test_history_outage, :test_sampler_roll, :test_gateway_budget. Deterministic two-replica and stale-boot/incarnation coverage is in the Gateway tests; k3s does not claim a deployed two-replica Gateway scenario. Flux/Cilium objects have no system test; staging is their first cluster.

Deliberately left out

Lease/EndpointSlice/control-Service RBAC and legacy epoch proto-field/receiver-tolerance removal, gated on this PR's image pins and rollback window; deletion of inert route objects; the Dispatcher's ownership Lease; EnvironmentService extraction. No Runtime-dials-Gateway mode, live-migration compatibility path, service-SVID/auth allowlist migration, Identity admission, Viper, Policy error mapping or TLS-setting edits.

Questions for Claude

The receiver/live-Runtime gate was verified by Claude on staging at 08:25 UTC on 2026-11-14. Please confirm the completed #16361 Gateway and #16362 caller-process rollout before merge, including external callers. I have verified the repository runner-disk pin and have not changed live staging Sandboxes.

@agentydragon-agent
agentydragon-agent force-pushed the gateway-route-removal branch 2 times, most recently from e41e0a7 to 28bad5f Compare November 14, 2026 07:23
@agentydragon
agentydragon enabled auto-merge (squash) November 14, 2026 07:25
@agentydragon-agent
agentydragon-agent force-pushed the gateway-route-removal branch 2 times, most recently from e439c74 to 40bc637 Compare November 14, 2026 07:41
agentydragon added a commit that referenced this pull request Nov 14, 2026
<!-- claude-projects-attribution: {"github_login":"agentydragon"} -->
_Requested by **Rai** · [project
thread](https://claude.ai/code/project/chan_01RbvEzrs8Yi8YhNnHoSLtMy?thread=cmsg_01RbvEzrs8Yi8YhNnHoSLtMy2SHUu9XsCD6ej526Bhqzxa)_

Doc-only update of `agentplane/plans/task_dag.md` (and one link in
`plans/environment_rules.md`) for the last day's plans and decisions.

**Added**
- **Released 2026-11-14:** each track the operator released today, with
the quoted release, its node and its PRs' state at 07:40 UTC.
- **Sandbox-free central Sessions:** orders the existing sections 5, 13
and 14 nodes for a central Session with no Sandbox attached, from the
three #15436 decisions through the zero-Environment path and its
follow-ons. New decision node `CODE_MODE_HOST_DESIGN` for whether to
keep Codex's pinned JavaScript host or own a V8 host; it is recorded as
not a prerequisite, since the host runs in the sampler Pod and needs no
executor.
- **Section 17, two binaries and the Go layout:** the agreed
Environments/Sessions split with its decided cards (automatic wake,
directories, identity re-check, no budget for explicit resume), a stage
table mapping the gh-pages Go plan onto DAG nodes, and nodes for the
stage 3 moves, keep-alive holds, stage 2 layout fixes and stage 4 local
extractions.

**Updated**
- 2026-10-30 sprint priorities marked stale; no new priorities are
stated.
- `NOTIFICATION_SERVICE_SPLIT` carries the operator's 01:11–01:12
direction (inbox service plus GitHub actor; subscription seam open).
- `SESSION_TOOL_SURFACE_CONVERGENCE_DESIGN`: direction agreed at 01:10;
interface review open.
- `DISPATCHER_PERSISTENCE_MODEL_DESIGN`: topology selected; delivery
ownership and the Dispatcher's ownership Lease remain open.
- `RUNNER_GATEWAY_RENAME`: the Environment Gateway name is dropped; the
node now covers deployed names after the split.
- `PRIVILEGED_BACKEND_AUTH_DESIGN`, `APP_AGENT_TOKEN_DECISION`,
`ENVIRONMENT_LIFECYCLE_RULES`, `ENVIRONMENT_IDENTITY_BIND_SESSION` and
`REFACTOR_COHERENCE_AUDIT` carry their merged and in-review PRs.

**Deleted as finished:** `ENVIRONMENT_RULE_TYPE_DESIGN` (selected
04:58), `ENVIRONMENT_IDENTITY_RESUME` (#16346 merged),
`TRANSPORT_PACKAGE_RENAME` (the package lives only at `net/peertls`).

**Left out on purpose**
- Section 12 is untouched: the open Lease stack (#16361, #16362, #16365)
edits it. Section 17 refers only to `RUNNER_CONNECTION_LEASE_CLEANUP`,
which exists in both versions; whichever lands second rebases.
- `plans/environment_rules.md` status line: #16351 edits it.
- No priority changes and no new dispatch.

**Checks:** a local script parsed every Mermaid block (no unknown IDs or
cycles introduced), node headings (no duplicates) and local anchors; the
remaining findings, such as links to `central_session_contract.md` from
open #15436, predate this PR. `pre-commit run --files` on both files
passes.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01AYUnw2srgM3L5WxRM4C2Kt)_

Co-authored-by: Claude <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Visual review for b1e8cc98

Request a manual rerun from the CI run page.

Warning

12 targets used the latest devel baseline (a68e0de5) because the PR base published none. Diffs there may include devel drift, not just this PR's changes.

No visual changes among the 12 affected Bazel test targets. Open visual review.

@agentydragon-agent
agentydragon-agent force-pushed the gateway-route-removal branch 2 times, most recently from d31c0d7 to 08047af Compare November 14, 2026 08:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant