fix: Double embedding pass on fresh index - #260
Merged
Conversation
Fresh index builds embedded every file's chunks in the per-file loop and then embedded the whole corpus again at the end. Only incremental builds need the per-file vectors, so the fresh path now does a single batched encode. Cold index time on the speed benchmark drops from 518 ms to 383 ms; READMEs, plot data, and the cold plot are updated to match. The installer printed IntegrationType members through an f-string, which renders as IntegrationType.MCP on Python 3.11+. Use the enum value. Claude-Session: https://claude.ai/code/session_01JtE5efYw5bBqbbwhkkuXv2
Keep main's per-file embedding in the loop and reuse those vectors for fresh builds instead of encoding the whole corpus a second time. This removes the extra branch, so no complexity suppression is needed. Per-file encoding is also slightly faster than one repo-wide batch, because the tokenizer pads each batch to its longest text. Cold index time on the speed benchmark is now 344 ms; NDCG@10 on the quality benchmark is unchanged (0.8524 -> 0.8527). Claude-Session: https://claude.ai/code/session_01JtE5efYw5bBqbbwhkkuXv2
Codecov Report✅ All modified and coverable lines are covered by tests.
🚀 New features to boost your workflow:
|
Confidence Score: 4/5The functional indexing fix appears safe to merge, with a non-blocking benchmark-provenance concern in the refreshed performance claims. Vector parts remain aligned with chunks on both fresh and incremental builds, and installer identifiers satisfy the new enum access; the only accepted issue is that the published comparison mixes newly measured Semble timings with baselines from an earlier run. Files Needing Attention: benchmarks/results/speed-5fcc8a0a7b89.json Reviews (1): Last reviewed commit: "benchmarks: speed results for 5fcc8a0a7b..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
stephantul
approved these changes
Sep 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes a double embedding pass on fresh indexes. We had a guard for checking if there was a previous index on the second call, but not on the first one, which is now added. This speeds up Semble by ~34%, so the benchmarks/claims are also updated.
Also found a small visual bug on newer Python versions with the installer enum printing which is fixed.