Skip to content

Don't report shutdown DB interrupts as processing errors#3441

Open
ouziel-slama wants to merge 2 commits into
developfrom
fix-shutdown-interrupt-sentry-noise
Open

Don't report shutdown DB interrupts as processing errors#3441
ouziel-slama wants to merge 2 commits into
developfrom
fix-shutdown-interrupt-sentry-noise

Conversation

@ouziel-slama

Copy link
Copy Markdown
Member

Problem

Sentry issue API-SERVER-8G: Error processing message: ParseTransactionError('interrupted') in production.

ParseTransactionError('interrupted') is an apsw.InterruptError (whose str() is exactly "interrupted") wrapped at blocks.py:260. It is triggered by a db.interrupt() that aborts an in-flight SQLite query.

Chain

  1. RawMempoolParser shares the same DB connection as the watcher (RawMempoolParser(self.db)).
  2. On shutdown, BlockchainWatcher.stop()mempool_parser.stop()db.interrupt() on that shared connection.
  3. If the watcher is mid-parse inside receive_message (receive_rawblock/receive_rawtxparse_tx), the query is aborted → InterruptError("interrupted")ParseTransactionError("interrupted").
  4. receive_multipart logged it as an error, sent it to Sentry, and re-raised to force a "fail loud" restart — of a process that was already stopping.

This is benign teardown noise: no block/tx is corrupted (the supervisor restarts catch_up() from scratch anyway).

Fix

Guard the receive_multipart exception handler with stop_event: during shutdown, log the interrupt at debug and return quietly instead of paging Sentry / re-raising. The fail-loud path is preserved for genuine processing failures (when stop_event is not set).

Tests

  • New test_receive_multipart_swallows_interrupt_during_shutdown: with stop_event set, a ParseTransactionError("interrupted") is neither re-raised nor sent to Sentry.
  • Existing test_receive_multipart_message_processing_error still asserts the fail-loud re-raise when not shutting down.

Release notes updated in release-notes/release-notes-v11.1.1.md.

Ouziel added 2 commits June 16, 2026 22:29
When the server stops, RawMempoolParser.stop() calls db.interrupt() on the
connection shared with the BlockchainWatcher. If the watcher is mid-parse,
the in-flight SQLite query is aborted as apsw.InterruptError, surfacing as
ParseTransactionError("interrupted") in receive_multipart. This benign
teardown race was logged as an error, sent to Sentry, and re-raised to force
a "fail loud" restart of an already-stopping process.

Guard the handler with stop_event: during shutdown, log the interrupt at
debug and return quietly instead of paging Sentry. The fail-loud path is
preserved for genuine processing failures (stop_event not set).
@codecov

codecov Bot commented Jun 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.22%. Comparing base (3f08503) to head (50b77c2).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3441      +/-   ##
===========================================
- Coverage    91.38%   91.22%   -0.17%     
===========================================
  Files          114      114              
  Lines        16296    16299       +3     
===========================================
- Hits         14892    14868      -24     
- Misses        1404     1431      +27     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants