test(vector): assert routing invariant, not exact HNSW recall (#773)#774
Merged
Conversation
The field-routing tests asserted the presence of every expected doc
(e.g. contains(&1) && contains(&2)) on a randomized HNSW graph. That is
an exact-recall assertion, which is environment-sensitively flaky: the
ubuntu-latest x86_64 CI job intermittently returned only the nearest doc
(e.g. {1}), failing the assertion, while aarch64/Windows passed.
These are routing tests: the deterministic guarantee is that a query
routed to a field returns only that field's docs (no cross-field leak),
not that approximate search recalls every one. Assert that invariant
instead — non-empty result that is a subset of the targeted field — and,
for the search-all case, that both fields are represented via the
query-identical docs that are reliably returned.
Closes #773
This was referenced Jun 3, 2026
ci: free disk space before embeddings-all builds to stop intermittent "No space left on device"
#775
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
vector_field_routing_test.rsasserted the presence of every expected doc (e.g.ids.contains(&1) && ids.contains(&2)) on a randomized HNSW graph. That is an exact-recall assertion, which is environment-sensitively flaky: theTest laurus (ubuntu-latest, x86_64-unknown-linux-gnu, stable)CI job intermittently returned only the nearest doc ({1}) and failed atvector_field_routing_test.rs:138, while the aarch64 and Windows jobs of the same run passed. It is not reproducible locally (75/75 passes here), consistent with HNSW build-RNG/connectivity non-determinism — the anti-patternlessons.md(2026-05-29) already documents.All five tests in the file shared this pattern, so all were latent flakes; only
test_query_vector_fields_routeshappened to fire.Change
These are routing tests. The deterministic guarantee is that a query routed to a field returns only that field's docs (no cross-field leakage) — not that approximate search recalls every one. So:
contains, keeps the no-leak guarantee).test_no_fields_searches_allasserts both fields are represented, anchored on the docs identical to the query (doc 1inimg_vec,doc 3intxt_vec, similarity 1.0, reliably returned) instead of requiring all four docs.The observed failure case
{1}now passes deterministically ({1} ⊆ {1,2}and non-empty), so this resolves the flake without weakening the routing guarantee.Verification
cargo fmt --check— cleancargo clippy -p laurus --test vector_field_routing_test -- -D warnings— cleanCloses #773