feat(core): derive geyser slot statuses from a per-slot lifecycle - #778
Open
cds-amal wants to merge 7 commits into
Open
feat(core): derive geyser slot statuses from a per-slot lifecycle#778cds-amal wants to merge 7 commits into
cds-amal wants to merge 7 commits into
Conversation
Contributor
Greptile SummaryThe PR centralizes Geyser and WebSocket slot-status generation in a per-slot lifecycle registry.
Confidence Score: 5/5The PR appears safe to merge because no blocking failure remains in the eligible follow-up review scope. No blocking failure remains. Important Files Changed
Reviews (3): Last reviewed commit: "docs(core): the slot diagram as mermaid,..." | Re-trigger Greptile |
cds-amal
marked this pull request as draft
August 25, 2026 05:13
- Every slot-status emission now comes from one transition relation:
- `announce` emits `CreatedBank`
- `produce` emits `Processed`
- `confirm` emits `Confirmed`
- `root` emits `Rooted`
- `warp` emits `Dead` for the slot the clock abandons and
`CreatedBank` for the one it lands on
Warp announcement is guarded so a slot is announced at most once.
Every transition requires its exact predecessor stage, so no status
can be skipped or repeated.
- Block production follows the lifecycle order:
- emit the slot's block data
- produce and confirm the slot
- root whatever is due
- announce the next slot last
A consumer therefore sees every slot announced before its data, and
its data before its confirmation.
- Startup announces the open slot and sends `EndOfStartup` before the
RPC listeners bind. Nothing external can therefore emit block data for
a slot a plugin is not tracking yet.
- Both clock-warp handlers resolve the lifecycle after moving the clock.
A network reset forgets every slot and announces its new genesis.
- `UpdateSlotStatus` now carries Agave's `SlotStatus`, since
`CreatedBank` and `Dead` exist only there.
- The registry lives in `surfnet/slot_lifecycle.rs`, with the slot table
as its test oracle: one test per deciding cell. SVM-level tests read
the geyser stream as a consumer sees it and check:
- announce before data
- data before confirmation
- statuses in order and exactly once
- a warp yielding exactly `Dead`, then `CreatedBank`
Co-authored-by: Snehendu Roy <roysnehendupersonal@gmail.com>
Co-authored-by: Micaiah Reid <micaiahreid@gmail.com>
- Add the spec as a second encoding of the slot table, following the startup state machine's precedence: - expected emissions per `(state, event)` cell - expected successor registry per cell - reads of the machine only through its public accessors The maintenance procedure stays the same: state the cell in the spec first, change the machine second, and let the sweep fail while they disagree. - Sweep the full alphabet over a bounded slot domain: - visit every reachable registry state times every event - compare the machine's emissions exactly against the spec - compare the successor registry exactly against the spec - assert that the whole bounded state space was reached This checks the transition systems cell for cell, discharging implementation-refines-model for the bounded sequential core rather than leaving it as an argument. - Sweep production's call grammar separately, mirroring the operations that drive the registry: - close a block, as in `confirm_current_block` - warp, as in `warp_clock` - reset, as in `reset_network` At every reachable point, assert the invariant underlying E1 and L1: exactly one slot is `Announced`, and it is the open slot. - Keep interleaving properties in the Promela models; these sweeps cover the sequential transition core and its production grammar.
- Make the spec's encoding the table itself in `slot_lifecycle/spec.rs`:
- `PER_SLOT` holds one grid-aligned row per `(state, event)` cell
- the sweeps interpret those rows directly
- a totality test keeps the table complete and names any missing cell
This makes a missing cell explicit, which the old match's
exhaustiveness check did not surface as clearly.
- Keep warp and clear as set-level rules:
- warp routes its announce step through the table's `announce` cell
- clear remains outside the per-slot transition table
This prevents the two encodings from drifting on what announcing
means. The machine itself never reads the table; spec and
implementation remain independent encodings, or the sweeps would
prove nothing.
- Render the documentation from the same spec rows:
- `slot-lifecycle.md` carries authored prose around generated table
and edge-diagram blocks
- the module includes it directly into rustdoc
- a test detects stale generated blocks and directs the developer to
`cargo surfpool-update-slot-spec`
- that alias regenerates the documentation
One generic renderer iterates the rows, keeping the apparatus to a
fraction of the startup spec's per-table render functions.
- Give this machine the full treatment because geyser plugins consume
its emission sequences and the table has outgrown eyeball totality.
Smaller registries keep a named test per cell and hand-written
rustdoc.
…statuses Three changes to how the slot lifecycle meets its callers, from reviewing the warp paths: - A backward clock warp is a reorg: every slot the new timeline rewrites dies (Dead, in slot order) instead of vanishing silently, and the destination is re-announced. At-most-once holds per bank; a reorg makes a new bank, so time travel to the current slot reads as Dead then CreatedBank rather than as a repeated status. - Rooting is a threshold, not arithmetic: root_through(r) roots every confirmed slot at or below r from the registry's own record, so a history with gaps (a clock warp) roots exactly what it confirmed and strands nothing. The genesis guard in confirm_current_block goes away: a threshold below everything on record roots nothing. - emit_slot_statuses mirrors CreatedBank, Confirmed, Rooted, and Dead to slotsUpdatesSubscribe, so the geyser and ws streams cannot diverge on lifecycle events; warp emissions previously reached geyser only. Block production keeps sending Frozen itself, since only it knows the block's transaction stats. The bespoke ws sends in confirm_current_block are gone. - CreatedBank names its parent from the registry (the highest slot on record below), so a warp destination links to the chain tip instead of a slot that never existed. Other statuses carry no parent, as agave's notifier does. - Both clock command handlers call one warp_clock writer method, which owns the capture-write-resolve order the two arms previously duplicated, one of them with a dead first write of absolute_slot. The spec table gains a set-level RootThrough event routed through the table's root cell, and the reachability sweeps hold the machine to the new rules; the production grammar's state space is now linear in its slot bound, since threshold rooting keeps the steady-state registry at two slots.
…lot spec - define backward warps as reorgs - describe rooting as a threshold drained from the registry's record - document histories the registry cannot see: persistent-mode restarts start empty, and network resets forget slots without terminal status - regenerate the slot lifecycle diagram - drop the review-notes model pointer; the exhaustive sweeps are the checkable artifact that ships
- define warps at or below root as legal discontinuities, not reconciliation - let rooted history disappear without Dead; CreatedBank at the landing signals the new timeline - require consumers that see CreatedBank for a rooted slot to drop state at or above it and resync - split unconditional per-bank guarantees from cross-timeline guarantees that hold only until an operator warps across them - document the contract in the slot spec, warp_clock, and SlotLifecycle::warp - leave the machine unchanged; the registry already implements this contract
- Emit the machine's edges as a mermaid state diagram instead of a
preformatted text block:
- `render_diagram` maps `(absent)` and forgotten onto the start and
end pseudo-states and generates one edge per advancing cell
- the two warp rules stay appended beside the table-driven edges,
with a floating "any stage" node carrying the set-level warp back
- Adopt the startup spec's render pipeline, with one twist: the fence
is itself spec-generated, so its `BEGIN MERMAID` markers nest inside
the `GENERATED: diagram` region and regenerate with it:
- `cargo surfpool-render-slot-diagrams` renders the fence to
`src/surfnet/diagrams/machine-edges.svg`, pinned to the fnv1a hash
of its source
- `the_diagrams_match_their_renderings` fails on a stale render, so
CI needs no mermaid toolchain
- a new `crates/core/build.rs` splices the SVG in place of the fence
into `$OUT_DIR/slot-lifecycle.rustdoc.md`, which the module now
includes; the source file keeps the fence GitHub renders natively
- Render labels as SVG text with a quote-free font stack: rustdoc
applies smart punctuation to text inside the inlined style block,
so a quoted font name arrives curly-quoted and the browser falls
back to a wider face, clipping every foreignObject label at its
measured edge. SVG text overflows visibly instead, and an unquoted
stack survives the pipeline; the mechanism is spelled beside the
config in the render test.
- Duplicate the splice script and the diagram test helpers from the
startup crate rather than exporting its test-only module; the copies
point at their originals, and a third spec doc is the trigger to
extract a shared crate.
cds-amal
marked this pull request as ready for review
August 25, 2026 13:51
cds-amal
marked this pull request as draft
August 25, 2026 13:51
cds-amal
marked this pull request as ready for review
August 25, 2026 13:51
cds-amal
force-pushed
the
state/slot_lifecycle
branch
from
August 25, 2026 16:42
6c50a7e to
2a1e73a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1. What
This PR refactors #747 starting with its bones (the status set, warp handling, and wiring points) and moves the scattered emission sites behind one state machine, so the ordering guarantees are properties of the machine, not its call sites.
Every slot-status emission now comes from one transition relation, a per-slot lifecycle registry in
surfnet/slot_lifecycle/:announceemitsCreatedBank,produceemitsProcessed, andconfirmemitsConfirmedroot_through(r)roots every confirmed slot at or belowr, in slot order, from the registry's own recordDeadfor the abandoned open slot andCreatedBankfor the destination; a backward warp is a reorg, emittingDeadfor every rewritten slot, in slot order, before re-announcing the destinationEach transition requires its exact predecessor stage, so a status cannot be skipped or repeated within a bank. Block production follows lifecycle order (block data, then produce/confirm, then root, then announce the next slot), so consumers see every slot announced before its data and its data before confirmation.
UpdateSlotStatuscarries Agave'sSlotStatus, sinceCreatedBankandDeadexist only there.2. Consumer-visible changes
Both streams now derive from the same lifecycle, which makes a few wire-visible changes.
WS and geyser agree on lifecycle statuses
slotsUpdatesSubscribenow mirrorsCreatedBank,OptimisticConfirmation,Root, andDeadfrom the same emission point as geyser. Warp statuses, which previously reached geyser only, now reach both.Frozenstays with block production, which alone knows the block's transaction stats. Within a block,CreatedBankfor the next slot therefore arrives afterFrozenandOptimisticConfirmation, rather than between them.Rooting follows the registry
A warp gap drains cleanly: every confirmed slot still roots.
After a persistent-mode restart, the fresh registry knows nothing about prior-run slots, so neither stream emits roots for them. Previously ws
Rootfired unconditionally while geyserRooteddid not.CreatedBankcarries the registry's parentThe parent is the highest slot on record below the new slot: the registry's chain tip. A warp destination therefore links to a real slot instead of
slot - 1. If nothing below the destination remains on record, it is parentless.Other statuses carry no parent, matching Agave's notifier.
Deep backward warps replace the timeline
A backward warp may land at or below the root line; time travel to the current epoch does exactly this. Those rooted slots have already left the registry, so the landing re-announces one with a parentless
CreatedBank.That is the discontinuity signal: a consumer that sees
CreatedBankfor a slot it previously saw rooted drops state at or above that slot and resyncs.Time travel is a cheatcode, so calling it suspends the cross-timeline guarantees that rooted slots are final and slots increase. The per-bank guarantees still hold through the warp: announce before data, data before confirmation, statuses in order and once.
3. How the contract is checked
The lifecycle has an independent machine and spec, following the startup state machine's precedent:
slot_lifecycle/mod.rsimplements the machineslot_lifecycle/spec.rsencodes the transition table, one row per(state, event)cell; warp, root-through, and clear are set-level rules routed through those same cellsThe machine never reads the spec, so the two remain independent encodings.
The machine matches the spec
An exhaustive sweep visits every reachable registry state across every event. For each transition, it compares the machine's emissions and successor state against the spec cell.
A totality test separately ensures that every
(state, event)cell exists.Production preserves the open-slot invariant
A second sweep drives the registry through production's call grammar: close a block, warp, reset.
At every reachable point, exactly one slot must be
Announced, and it must be the open slot. This rules out both an orphaned announced slot and an open slot whose data could arrive before its announcement.The wire contract is tested as consumers see it
SVM-level tests observe both streams and check the external guarantees: announce before data, data before confirmation, statuses in order and once per bank, and warp statuses mirrored to ws.
The docs come from the spec
The rustdoc table and edge diagram in
slot-lifecycle.mdrender from the spec rows. A test flags stale generated blocks;cargo surfpool-update-slot-specregenerates them. This is the startup spec's diagram pipeline, applied to the slot spec.