Refactor ChainQuery to be more flexible #2139
Draft
evanlinjin wants to merge 19 commits intobitcoindevkit:masterfrom
Draft
Refactor ChainQuery to be more flexible #2139evanlinjin wants to merge 19 commits intobitcoindevkit:masterfrom
ChainQuery to be more flexible #2139evanlinjin wants to merge 19 commits intobitcoindevkit:masterfrom
Conversation
Open
3 tasks
evanlinjin
force-pushed
the
refactor/sans-io-mtp-chain-query
branch
2 times, most recently
from
6faf825 to
c1ad161
Compare
evanlinjin
force-pushed
the
refactor/sans-io-mtp-chain-query
branch
from
c1ad161 to
bdd7608
Compare
replace `CanonicalIter` with sans-io `CanonicalizationTask`
Introduces new `CanonicalizationTask`, which implements the canonicalization process
through a request/response pattern, that allow us to remove the
`ChainOracle` dependency in the future.
The `CanonicalizationTask` handles direct/transitive anchor determination, also tracks
already confirmed anchors to avoid redundant queries. After all the `CanonicalizationRequest`'s
have been resolved, the `CanonicalizationTask` can be finalized returning the final `CanonicalView`.
It batches all the anchors, which require a chain query, for a given transaction into a single
`CanonicalizationRequest`, instead of having multiple requests for each one.
- Add new `CanonicalizationTask`, relying on
`Canonicalization{Request|Response}` for chain queries. It
- Replaces the old `CanonicalIter` usage, with new
`CanonicalizationTask`.
BREAKING CHANGE: It replaces direct `ChainOracle` querying in canonicalization process, with
the new request/response pattern by `CanonicalizationTask`.
The new API introduces a sans-io behavior, separating the canonicalization logic from `I/O` operations, it should be used as follows: 1. Create a new `CanonicalizationTask` with a `TxGraph`, by calling: `graph.canonicalization_task(params)` 2. Execute the canonicalization process with a chain oracle (e.g `LocalChain`, which implements `ChainOracle` trait), by calling: `chain.canonicalize(task, chain_tip)` - Replace `CanonicalView::new()` constructor with internal `CanonicalView::new()` for use by `CanonicalizationTask` - Remove `TxGraph::try_canonical_view()` and `TxGraph::canonical_view()` methods - Add `TxGraph::canonicalization_task()` method to create canonicalization tasks - Add `LocalChain::canonicalize()` method to process tasks and return `CanonicalView`'s - Update `IndexedTxGraph` to delegate canonicalization to underlying `TxGraph` BREAKING CHANGE: Remove `CanonicalView::new()` and `TxGraph::canonical_view()` methods in favor of task-based approach
- Adds `CanonicalReason`, `ObservedIn`, and `CanonicalizationParams` to `canonical_task.rs` module, instead of using the ones from `canonical_iter.rs`. - Removes the `canonical_iter.rs` file and its module declaration. BREAKING CHANGE: `CanonicalIter` and all its exports are removed
…icalizationTask` Introduce a new `ChainQuery` trait in `bdk_core` that provides an interface for query-based operations against blockchain data. This trait enables sans-IO patterns for algorithms that need to interact with blockchain oracles without directly performing I/O. The `CanonicalizationTask` now implements this trait, making it more composable and allowing the query pattern to be reused for other blockchain query operations. - Add `ChainQuery` trait with associated types for Request, Response, Context, and Result - Implement `ChainQuery` for `CanonicalizationTask` with `BlockId` as context BREAKING CHANGE: `CanonicalizationTask::finish()` now requires a `BlockId` parameter Co-Authored-By: Claude <noreply@anthropic.com>
Make `ChainRequest`/`ChainResponse` generic over block identifier types to enable reuse beyond BlockId. Move `chain_tip` into `ChainRequest` for better encapsulation and simpler API. - Make `ChainRequest` and `ChainResponse` generic types with `BlockId` as default - Add `chain_tip` field to `ChainRequest` to make it self-contained - Change `ChainQuery` trait to use generic parameter `B` for block identifier type - Remove `chain_tip` parameter from `LocalChain::canonicalize()` method - Rename `ChainQuery::Result` to `ChainQuery::Output` for clarity BREAKING CHANGE: - `ChainRequest` now has a `chain_tip` field and is generic over block identifier type - `ChainResponse` is now generic with default type parameter `BlockId` - `ChainQuery` trait now takes a generic parameter `B = BlockId` - `LocalChain::canonicalize()` no longer takes a `chain_tip` parameter Co-authored-by: Claude <noreply@anthropic.com> refactor(chain): make `LocalChain::canonicalize()` generic over `ChainQuery` Allow any type implementing `ChainQuery` trait instead of requiring `CanonicalizationTask` specifically. Signed-off-by: Leonardo Lima <oleonardolima@users.noreply.github.com>
- Unify both `unprocessed_anchored_txs` and `pending_anchored_txs` in a single `unprocessed_anchored_txs` queue. - Changes the `unprocessed_anchored_txs from `Iterator` to `VecDeque`. - Removes the `pending_anchored_txs` field and it's usage. - Collects all `anchored_txs` upfront instead of lazy iteration.
- Add new `CanonicalStage` enum for tracking the different canonicalization phases/stages. - Add new `try_advance()` method for stage progression. - Add new `is_transitive()` helper to `CanonicalReason`. - Change internal `confirmed_anchors` to `direct_anchors` for better clarity. - Update the `resolve_query()` to handle staged-based processing. Co-authored-by: Claude <noreply@anthropic.com>
Inline all stage-processing logic into `next_query()`, removing the separate `try_advance()` method, `process_*_txs()` helpers, and `is_finished()` from the `ChainQuery` trait. Add `AssumedTxs` as an explicit first stage and `CanonicalStage::advance()` for centralized stage transitions. Document the `ChainQuery` protocol contract.
…`Canonical<A, P>` Separate concerns by splitting `CanonicalizationTask` into two phases: 1. `CanonicalTask` determines which transactions are canonical and why (`CanonicalReason`), outputting `CanonicalTxs<A>`. 2. `CanonicalViewTask` resolves reasons into `ChainPosition`s (confirmed vs unconfirmed), outputting `CanonicalView<A>`. Make `Canonical<A, P>`, `CanonicalTx<P>`, and `FullTxOut<P>` generic over the position type so the same structs serve both phases. Add `LocalChain::canonical_view()` convenience method for the common two-step pipeline. Renames: `CanonicalizationTask` -> `CanonicalTask`, `CanonicalizationParams` -> `CanonicalParams`, `canonicalization_task()` -> `canonical_task()`, `FullTxOut::chain_position` -> `FullTxOut::pos`. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…Query::tip()` The chain tip is constant for the lifetime of a query, so it belongs on the trait rather than being redundantly copied into every request. `ChainRequest` is now a type alias for `Vec<B>`. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ChainResponse` These types only ever used `BlockId`, so the generic parameter added unnecessary complexity. All three are now hardcoded to `BlockId`. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Assumed transactions bypass the `AnchoredTxs` stage and are marked
canonical immediately with `CanonicalReason::Assumed`. Previously,
`view_task()` only queued anchor checks for transitive txs, so directly
assumed txs (`Assumed { descendant: None }`) were never checked and
always resolved to `Unconfirmed` even when they had confirmed anchors.
Queue all `Assumed` txs for anchor checks in `view_task()` and look up
`direct_anchors` for both `Assumed` variants in `finish()`.
Fixes bitcoindevkit#2088
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…onical_view_task.rs` Move shared types (`CanonicalTx`, `Canonical`, `CanonicalView`, `CanonicalTxs`) and convenience methods into `canonical.rs`. Keep only the phase-2 task (`CanonicalViewTask`) in `canonical_view_task.rs`. Also rename `FullTxOut` to `CanonicalTxOut` and move it to `canonical.rs`. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- ignore `canonical_task.rs` test module in coverage. - reuse `canonical_view` in `test_tx_graph_conflicts.rs`
- add new `test_canonical_view_task.rs` to handle different scenarios of chain position resolution. - fixes the assumed canonical txs chain position resolution, especially for transitively assumed canonical transactions, where there's an anchored/confirmed descendant.
Rename `ChainQuery` to `ChainTask`.
Replace `ChainRequest`/`ChainResponse` with a poll-based `ChainTask`
trait. The driver calls `poll()` in a loop, resolving block-height
queries as needed. A generic parameter `B` (default `BlockHash`) lets
tasks receive richer block data (e.g. `Header`) from the chain source.
Add per-transaction and tip MTP fields to `CanonicalView`, computed
optionally via `CanonicalViewTask::with_mtp()`.
Add `BlockQueries` helper for tracking request/resolve state, and
`LocalChain::canonical_view{,_with_mtp}()` convenience methods.
evanlinjin
force-pushed
the
refactor/sans-io-mtp-chain-query
branch
from
bdd7608 to
45bf03a
Compare
`canonicalize` previously drove any `ChainTask`, which was misleading. Rename it to `run_task` and reuse `canonicalize`/`canonicalize_with_mtp` for the convenience methods that actually produce a `CanonicalView` (formerly `canonical_view`/`canonical_view_with_mtp`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #2139 +/- ##
==========================================
- Coverage 77.36% 77.29% -0.08%
==========================================
Files 35 31 -4
Lines 9035 5945 -3090
Branches 0 267 +267
==========================================
- Hits 6990 4595 -2395
+ Misses 2045 1277 -768
- Partials 0 73 +73
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This builds on top of #2038.
ChainQuerycan fetch any block-data type (BlockHash,Header).CanonicalViewTaskcan fetch headers to calculate MTP values.CanoncialTx,CanonicalTxOuthas new MTP fields.Notes to the reviewers
WIP
Changelog notice
WIP
Checklists
All Submissions:
New Features: