Problem
hyp backfill dedups against committed part_ids (scanExistingPartIds) but is blind to the
spool. If rows are sitting unflushed in the spool (captured live, not yet committed) when
backfill runs, it materializes its own copy of those messages, then the spool flushes its copy
later → two rows with the same id.
Not a tight race — the window is however long rows sit unflushed (debounce interval), and it's
deterministic given overlapping unflushed rows at backfill time.
Fix
- Operational: flush the spool before backfill.
- Real fix: have the materializer also scan spooled rows, not just committed ones.
Done when
Running hyp backfill with unflushed overlapping rows in the spool produces no same-id
duplicates.
Problem
hyp backfilldedups against committedpart_ids (scanExistingPartIds) but is blind to thespool. If rows are sitting unflushed in the spool (captured live, not yet committed) when
backfill runs, it materializes its own copy of those messages, then the spool flushes its copy
later → two rows with the same id.
Not a tight race — the window is however long rows sit unflushed (debounce interval), and it's
deterministic given overlapping unflushed rows at backfill time.
Fix
Done when
Running
hyp backfillwith unflushed overlapping rows in the spool produces no same-idduplicates.