Fix storage decode policy - #28
Draft
GCdePaula wants to merge 3 commits into
Draft
Conversation
Restore the five ledger files from 0561d53 and update statuses: F7 records PR #26 as the field-extension half of WP5 (the invalidation contract remains open, owned by the WS-redesign track), including the interim rebuild-on-disconnect consumer rule from the hint branch. Plan of record for the six in-flight efforts: PR #26 (done), this ledger restore, the WS protocol redesign (generation id, pull-based discontinuity detection), the storage decode-policy reconciliation, the fee exponentiation LUT, and the dump/Application API redesign. Captures the agreed directions, the open decisions (freeze-vs-exact fee table, boundaries-as-events vs per-row context, the mmap working-image fork), and the libdex constraints to surface to Bart (fsync posture, canonical-bytes layout, DirectInput timestamp duality). Indexed from CLAUDE.md/AGENTS.md.
convert.rs promised saturating conversions so corrupted DB rows could not crash the process — the opposite of docs/invariants.md's fail-loud policy, which bans saturating decode of contract-impossible data because a fabricated plausible value externalizes as state divergence (a signed batch, a feed event, a wrong recovery pivot) while a loud crash is recoverable by design. A 116-call-site audit found every non-test width-conversion site is contract-bound (schema CHECKs, triggers, or writer-side types), so the width converters now panic on out-of-range values. The three principled exceptions keep saturating, now explicitly: - clock conversions (to_unix_ms/now_unix_ms) — wall-clock is environmental, never an invariant (review F8); from_unix_ms still fail-louds on sign, which every u64-clock-sourced writer guarantees - untrusted/config-sourced SQL bounds via the new saturating_query_bound (WS from_offset cursors, page limits) where clamping to i64::MAX preserves comparison semantics exactly - the two fee-policy reads in query_batch_policy, now a documented local clamp over legal admin-set ranges (tested in admin.rs) Also from the audit: two adjacent saturating_adds on contract-bound counters (next batch nonce, safe-input index) became checked_add, and safe_accepted_batches.nonce gained its missing CHECK (nonce >= 0).
Track 1 is merged. Track 3 updated (subscribe with generation, server replies current, stale rejected, invalidation disconnects with a roll-back-and-recover error and bumps the generation), recorded as the confirmed consumer-facing contract, with the crash-path caveat that the farewell error is best-effort and stale-generation rejection at resubscribe stays load-bearing. Added a dated next-steps block. Track 3 grows the historical-replay: two paginated finalized-only endpoints (input-box order with provenance; ordered L2-tx order capped at the gold boundary, generation-free) composing race-free with the subscription, whose catch-up depth must exceed the finalized threshold's lag (static-cap-vs-dynamic flagged, dynamic preferred). Track 6 records machine-emulator PR #398 (durable rename_stored/remove_stored) as design input. New Track 7 queues the context-engineering review of CLAUDE/AGENTS/docs/READMEs with the blog's rules distilled in-tree. Campaign schedule replaces next-steps. Defer fee LUT (exact-floor if built; log-space fees may go defunct) Track 3 updated: generation semantics (persisted, bumped in-cascade, carried in DumpInfo across rebuilds), two finalized-only replay endpoints capped at the gold boundary, a WS v2 handshake with typed control events, and the depth guarantee that replaces the static catch-up cap (everything above gold always serveable); making replay+subscribe a closed, race-free pair. Track 6: adopts the working-image model (open/flush/clone_image/ delete_image/canonical_file_in_image), keeps commit and revert sequencer-side, keeps I13 with an explicit app/sequencer durability split validated by CM PR #398, and folds in the SafeInputRecord cleanup and the DirectInput timestamp duality decision.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Storage decode policy
convert.rspromised saturating conversions to keep corrupted DB rows from crashing the process, the opposite ofdocs/invariants.md's fail-loud policy. A 116-call-site audit found every non-test width-conversion site contract-bound, so the width converters now panic on out-of-range values instead of fabricating plausible ones. Deliberate saturation survives only where the full range is legal: clock conversions (F8), the new explicitly-namedsaturating_query_boundfor WS cursors/limits, and the fee-policy clamps inquery_batch_policy. Bonus from the audit: twosaturating_adds on contract-bound counters becamechecked_add, andsafe_accepted_batches.noncegained its missingCHECK (nonce >= 0).Housekeeping
Restore
docs/review/. The squash that merged the cockroach branch dropped the review ledger while AGENTS.md/CLAUDE.md kept pointing at it. Restored with statuses updated: F7 (feed invalidation, high) stays open with PR #26 recorded as WP5's field-extension half, and the interim mirror rule (treat any socket drop as a discontinuity, rebuild the soft suffix) is now written down in-tree.Record the 2026-07 coordination tracks. New
docs/plans/(indexed from CLAUDE.md/AGENTS.md) holding the plan of record for the in-flight efforts: feed & replay protocol redesign (generation id, historical replay endpoints), dump/Application API redesign, the LLM-docs review campaign, and the deferred fee LUT (exact-floor decided, pending a design decision that may retire log-space fees).