Skip to content

feat(core): derive geyser slot statuses from a per-slot lifecycle - #778

Open
cds-amal wants to merge 7 commits into
solana-foundation:mainfrom
cds-rs:state/slot_lifecycle
Open

feat(core): derive geyser slot statuses from a per-slot lifecycle#778
cds-amal wants to merge 7 commits into
solana-foundation:mainfrom
cds-rs:state/slot_lifecycle

Conversation

@cds-amal

@cds-amal cds-amal commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

1. What

This PR refactors #747 starting with its bones (the status set, warp handling, and wiring points) and moves the scattered emission sites behind one state machine, so the ordering guarantees are properties of the machine, not its call sites.

Every slot-status emission now comes from one transition relation, a per-slot lifecycle registry in surfnet/slot_lifecycle/:

  • announce emits CreatedBank, produce emits Processed, and confirm emits Confirmed
  • rooting is a threshold: root_through(r) roots every confirmed slot at or below r, in slot order, from the registry's own record
  • a forward warp emits Dead for the abandoned open slot and CreatedBank for the destination; a backward warp is a reorg, emitting Dead for every rewritten slot, in slot order, before re-announcing the destination

Each transition requires its exact predecessor stage, so a status cannot be skipped or repeated within a bank. Block production follows lifecycle order (block data, then produce/confirm, then root, then announce the next slot), so consumers see every slot announced before its data and its data before confirmation. UpdateSlotStatus carries Agave's SlotStatus, since CreatedBank and Dead exist only there.

2. Consumer-visible changes

Both streams now derive from the same lifecycle, which makes a few wire-visible changes.

WS and geyser agree on lifecycle statuses

slotsUpdatesSubscribe now mirrors CreatedBank, OptimisticConfirmation, Root, and Dead from the same emission point as geyser. Warp statuses, which previously reached geyser only, now reach both.

Frozen stays with block production, which alone knows the block's transaction stats. Within a block, CreatedBank for the next slot therefore arrives after Frozen and OptimisticConfirmation, rather than between them.

Rooting follows the registry

A warp gap drains cleanly: every confirmed slot still roots.

After a persistent-mode restart, the fresh registry knows nothing about prior-run slots, so neither stream emits roots for them. Previously ws Root fired unconditionally while geyser Rooted did not.

CreatedBank carries the registry's parent

The parent is the highest slot on record below the new slot: the registry's chain tip. A warp destination therefore links to a real slot instead of slot - 1. If nothing below the destination remains on record, it is parentless.

Other statuses carry no parent, matching Agave's notifier.

Deep backward warps replace the timeline

A backward warp may land at or below the root line; time travel to the current epoch does exactly this. Those rooted slots have already left the registry, so the landing re-announces one with a parentless CreatedBank.

That is the discontinuity signal: a consumer that sees CreatedBank for a slot it previously saw rooted drops state at or above that slot and resyncs.

Time travel is a cheatcode, so calling it suspends the cross-timeline guarantees that rooted slots are final and slots increase. The per-bank guarantees still hold through the warp: announce before data, data before confirmation, statuses in order and once.

3. How the contract is checked

The lifecycle has an independent machine and spec, following the startup state machine's precedent:

  • slot_lifecycle/mod.rs implements the machine
  • slot_lifecycle/spec.rs encodes the transition table, one row per (state, event) cell; warp, root-through, and clear are set-level rules routed through those same cells

The machine never reads the spec, so the two remain independent encodings.

The machine matches the spec

An exhaustive sweep visits every reachable registry state across every event. For each transition, it compares the machine's emissions and successor state against the spec cell.

A totality test separately ensures that every (state, event) cell exists.

Production preserves the open-slot invariant

A second sweep drives the registry through production's call grammar: close a block, warp, reset.

At every reachable point, exactly one slot must be Announced, and it must be the open slot. This rules out both an orphaned announced slot and an open slot whose data could arrive before its announcement.

The wire contract is tested as consumers see it

SVM-level tests observe both streams and check the external guarantees: announce before data, data before confirmation, statuses in order and once per bank, and warp statuses mirrored to ws.

The docs come from the spec

The rustdoc table and edge diagram in slot-lifecycle.md render from the spec rows. A test flags stale generated blocks; cargo surfpool-update-slot-spec regenerates them. This is the startup spec's diagram pipeline, applied to the slot spec.

@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR centralizes Geyser and WebSocket slot-status generation in a per-slot lifecycle registry.

  • Adds ordered announce, produce, confirm, root, warp, and reset transitions.
  • Routes block production, startup, and clock warps through the lifecycle.
  • Adds an independent transition specification, exhaustive reachability tests, generated documentation, and diagram tooling.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains in the eligible follow-up review scope.

No blocking failure remains.

Important Files Changed

Filename Overview
crates/core/src/surfnet/slot_lifecycle/mod.rs Introduces the per-slot lifecycle registry and ordered transitions for normal progression, rooting, warps, and resets.
crates/core/src/surfnet/svm.rs Integrates lifecycle emissions with block production, clock warps, Geyser plugins, and WebSocket slot-update subscribers.
crates/core/src/runloops/mod.rs Moves startup lifecycle notifications before RPC binding and routes clock updates through the centralized warp helper.
crates/core/src/surfnet/slot_lifecycle/spec.rs Defines an independent transition-table specification used as the lifecycle test oracle and documentation source.
crates/core/src/surfnet/slot_lifecycle/reachability_tests.rs Exhaustively compares reachable machine states with the specification and checks the production open-slot invariant.
crates/core/build.rs Generates rustdoc-ready lifecycle documentation by replacing Mermaid regions with checked-in SVG renderings.

Reviews (3): Last reviewed commit: "docs(core): the slot diagram as mermaid,..." | Re-trigger Greptile

@cds-amal
cds-amal marked this pull request as draft August 25, 2026 05:13
- Every slot-status emission now comes from one transition relation:
  - `announce` emits `CreatedBank`
  - `produce` emits `Processed`
  - `confirm` emits `Confirmed`
  - `root` emits `Rooted`
  - `warp` emits `Dead` for the slot the clock abandons and
    `CreatedBank` for the one it lands on

  Warp announcement is guarded so a slot is announced at most once.
  Every transition requires its exact predecessor stage, so no status
  can be skipped or repeated.

- Block production follows the lifecycle order:
  - emit the slot's block data
  - produce and confirm the slot
  - root whatever is due
  - announce the next slot last

  A consumer therefore sees every slot announced before its data, and
  its data before its confirmation.

- Startup announces the open slot and sends `EndOfStartup` before the
  RPC listeners bind. Nothing external can therefore emit block data for
  a slot a plugin is not tracking yet.

- Both clock-warp handlers resolve the lifecycle after moving the clock.
  A network reset forgets every slot and announces its new genesis.

- `UpdateSlotStatus` now carries Agave's `SlotStatus`, since
  `CreatedBank` and `Dead` exist only there.

- The registry lives in `surfnet/slot_lifecycle.rs`, with the slot table
  as its test oracle: one test per deciding cell. SVM-level tests read
  the geyser stream as a consumer sees it and check:
  - announce before data
  - data before confirmation
  - statuses in order and exactly once
  - a warp yielding exactly `Dead`, then `CreatedBank`

  Co-authored-by: Snehendu Roy <roysnehendupersonal@gmail.com>
  Co-authored-by: Micaiah Reid <micaiahreid@gmail.com>
- Add the spec as a second encoding of the slot table, following the
  startup state machine's precedence:
  - expected emissions per `(state, event)` cell
  - expected successor registry per cell
  - reads of the machine only through its public accessors

  The maintenance procedure stays the same: state the cell in the spec
  first, change the machine second, and let the sweep fail while they
  disagree.

- Sweep the full alphabet over a bounded slot domain:
  - visit every reachable registry state times every event
  - compare the machine's emissions exactly against the spec
  - compare the successor registry exactly against the spec
  - assert that the whole bounded state space was reached

  This checks the transition systems cell for cell, discharging
  implementation-refines-model for the bounded sequential core rather
  than leaving it as an argument.

- Sweep production's call grammar separately, mirroring the operations
  that drive the registry:
  - close a block, as in `confirm_current_block`
  - warp, as in `warp_clock`
  - reset, as in `reset_network`

  At every reachable point, assert the invariant underlying E1 and L1:
  exactly one slot is `Announced`, and it is the open slot.

- Keep interleaving properties in the Promela models; these sweeps cover
  the sequential transition core and its production grammar.
- Make the spec's encoding the table itself in `slot_lifecycle/spec.rs`:
  - `PER_SLOT` holds one grid-aligned row per `(state, event)` cell
  - the sweeps interpret those rows directly
  - a totality test keeps the table complete and names any missing cell

  This makes a missing cell explicit, which the old match's
  exhaustiveness check did not surface as clearly.

- Keep warp and clear as set-level rules:
  - warp routes its announce step through the table's `announce` cell
  - clear remains outside the per-slot transition table

  This prevents the two encodings from drifting on what announcing
  means. The machine itself never reads the table; spec and
  implementation remain independent encodings, or the sweeps would
  prove nothing.

- Render the documentation from the same spec rows:
  - `slot-lifecycle.md` carries authored prose around generated table
    and edge-diagram blocks
  - the module includes it directly into rustdoc
  - a test detects stale generated blocks and directs the developer to
    `cargo surfpool-update-slot-spec`
  - that alias regenerates the documentation

  One generic renderer iterates the rows, keeping the apparatus to a
  fraction of the startup spec's per-table render functions.

- Give this machine the full treatment because geyser plugins consume
  its emission sequences and the table has outgrown eyeball totality.
  Smaller registries keep a named test per cell and hand-written
  rustdoc.
…statuses

Three changes to how the slot lifecycle meets its callers, from
reviewing the warp paths:

- A backward clock warp is a reorg: every slot the new timeline
  rewrites dies (Dead, in slot order) instead of vanishing silently,
  and the destination is re-announced. At-most-once holds per bank; a
  reorg makes a new bank, so time travel to the current slot reads as
  Dead then CreatedBank rather than as a repeated status.

- Rooting is a threshold, not arithmetic: root_through(r) roots every
  confirmed slot at or below r from the registry's own record, so a
  history with gaps (a clock warp) roots exactly what it confirmed and
  strands nothing. The genesis guard in confirm_current_block goes
  away: a threshold below everything on record roots nothing.

- emit_slot_statuses mirrors CreatedBank, Confirmed, Rooted, and Dead
  to slotsUpdatesSubscribe, so the geyser and ws streams cannot
  diverge on lifecycle events; warp emissions previously reached
  geyser only. Block production keeps sending Frozen itself, since
  only it knows the block's transaction stats. The bespoke ws sends in
  confirm_current_block are gone.

- CreatedBank names its parent from the registry (the highest slot on
  record below), so a warp destination links to the chain tip instead
  of a slot that never existed. Other statuses carry no parent, as
  agave's notifier does.

- Both clock command handlers call one warp_clock writer method, which
  owns the capture-write-resolve order the two arms previously
  duplicated, one of them with a dead first write of absolute_slot.

The spec table gains a set-level RootThrough event routed through the
table's root cell, and the reachability sweeps hold the machine to the
new rules; the production grammar's state space is now linear in its
slot bound, since threshold rooting keeps the steady-state registry at
two slots.
…lot spec

- define backward warps as reorgs

- describe rooting as a threshold drained from the registry's record

- document histories the registry cannot see: persistent-mode restarts
  start empty, and network resets forget slots without terminal status
- regenerate the slot lifecycle diagram

- drop the review-notes model pointer; the exhaustive sweeps are the
  checkable artifact that ships
- define warps at or below root as legal discontinuities, not
  reconciliation

- let rooted history disappear without Dead; CreatedBank at the landing
  signals the new timeline

- require consumers that see CreatedBank for a rooted slot to drop state
  at or above it and resync

- split unconditional per-bank guarantees from cross-timeline guarantees
  that hold only until an operator warps across them

- document the contract in the slot spec, warp_clock, and
  SlotLifecycle::warp

- leave the machine unchanged; the registry already implements this
  contract
- Emit the machine's edges as a mermaid state diagram instead of a
  preformatted text block:
  - `render_diagram` maps `(absent)` and forgotten onto the start and
    end pseudo-states and generates one edge per advancing cell
  - the two warp rules stay appended beside the table-driven edges,
    with a floating "any stage" node carrying the set-level warp back

- Adopt the startup spec's render pipeline, with one twist: the fence
  is itself spec-generated, so its `BEGIN MERMAID` markers nest inside
  the `GENERATED: diagram` region and regenerate with it:
  - `cargo surfpool-render-slot-diagrams` renders the fence to
    `src/surfnet/diagrams/machine-edges.svg`, pinned to the fnv1a hash
    of its source
  - `the_diagrams_match_their_renderings` fails on a stale render, so
    CI needs no mermaid toolchain
  - a new `crates/core/build.rs` splices the SVG in place of the fence
    into `$OUT_DIR/slot-lifecycle.rustdoc.md`, which the module now
    includes; the source file keeps the fence GitHub renders natively

- Render labels as SVG text with a quote-free font stack: rustdoc
  applies smart punctuation to text inside the inlined style block,
  so a quoted font name arrives curly-quoted and the browser falls
  back to a wider face, clipping every foreignObject label at its
  measured edge. SVG text overflows visibly instead, and an unquoted
  stack survives the pipeline; the mechanism is spelled beside the
  config in the render test.

- Duplicate the splice script and the diagram test helpers from the
  startup crate rather than exporting its test-only module; the copies
  point at their originals, and a third spec doc is the trigger to
  extract a shared crate.
@cds-amal
cds-amal marked this pull request as ready for review August 25, 2026 13:51
@cds-amal
cds-amal marked this pull request as draft August 25, 2026 13:51
@cds-amal
cds-amal marked this pull request as ready for review August 25, 2026 13:51
@cds-amal
cds-amal force-pushed the state/slot_lifecycle branch from 6c50a7e to 2a1e73a Compare August 25, 2026 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant