fix(serve): resolve missing revisions via on-demand refetch#411
Merged
Conversation
The serve query service only fetched at startup, so a request for a commit
that landed afterwards (or on a ref the default refspec doesn't cover) failed
with an opaque `400 git error: ... Missing unknown <sha>`. Two compounding
causes:
- resolveSha reported success for a full 40-char SHA that isn't in the local
clone: JGit's Repository.resolve() parses the hex without an object-DB lookup,
and `git rev-parse <full-sha>` echoes it straight back. The miss therefore
surfaced later at checkout instead of at resolution.
- The service never refetched, so a just-landed commit could not be served
without a restart.
Fixes:
- Validate object existence in resolveSha for both git engines (JGit
RevWalk.parseCommit; subprocess `rev-parse --verify <rev>^{commit}`), raising
a new retryable MissingRevisionException when the commit is absent.
- ImpactedTargetsService performs one on-demand `git fetch` + retry when a
revision is missing, serialized and double-checked via a lock so concurrent
requests for the same new commit fetch at most once.
Genuinely-unknown revisions still return 400, now with a clear
"revision '<x>' is missing from the local clone" message.
Adds unit + integration coverage across both git engines and the service retry
paths, including an end-to-end JGit repro of a commit that only resolves after
a fetch.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
tinder-maxwellelliott
added a commit
that referenced
this pull request
Jul 7, 2026
… JGit fetch failures (#413) * fix(serve): stop --trackDeps over-reporting impacted targets on cache hits The serve query service returned inflated /impacted_targets sets whenever it was started with --trackDeps and served a `from` revision from the per-SHA hash cache. TargetHash is a data class whose generated equals() includes the `deps` field, but TargetHash.fromJson (used to read cache/JSON entries) only parses type#hash~directHash and always leaves deps=null. A cache-miss `to` (freshly generated, deps populated under --trackDeps) compared against a cache-hit `from` (deserialized, deps=null) therefore reported every dep-carrying target as changed even when its hash was identical. C1->C2 (both freshly generated) was correct; C2->C3 / C2->C4 (with `from` served from cache) were not. The CLI never hit this because both sides are deserialized, hence symmetric. Fix: compare targets by their hash identity (type#hash~directHash) rather than the full TargetHash in computeSimpleImpactedTargets and computeAllDistances -- `deps` is auxiliary data used only to derive distance metrics, not part of a target's impactedness. Adds regression tests for both the impacted and distance paths. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(serve): fall back to native git when a JGit fetch fails JGit 5.13 cannot fetch some clone shapes native git handles fine -- notably shallow (--depth) and partial (--filter=blob:none) clones, whose thin packs are delta-compressed against objects absent from the clone ("Missing delta base <sha>"). With the on-demand refetch added in #411, such a fetch now surfaces to callers as a 400. JGitClient.fetch() now retries a failed in-process fetch with the native `git` binary (at --gitPath) before giving up, and logs a one-time warning at startup if the workspace is a shallow or partial clone. JGit stays the default for resolve/checkout. If native git also fails, both causes are combined so neither is hidden. --gitEngine=subprocess still skips the in-process attempt entirely. Adds JGitClientTest coverage (fallback-disabled surfaces the JGit error; both-fail names the native fallback; a shallow clone can still fetch a just-landed commit) and documents the behavior in the serve README section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(serve): add serve integration harness + cron CI workflow Adds tools/serve_harness.py, a standalone integration harness that exercises `bazel-diff serve` end to end against a live git:// remote (git daemon) across both git engines x full/shallow/partial clones and the on-demand refetch path -- coverage the in-process E2ETest cannot reach. It builds //cli:bazel-diff, drives the real binary over HTTP, and asserts on the deterministic rule+source impacted subset (generated-file targets are filtered as noisy). This harness is what surfaced the --trackDeps over-reporting bug fixed earlier in this branch. Wired into CI as .github/workflows/serve-harness.yml, cron-only for now (12h schedule + workflow_dispatch) until it proves stable on runners; promote to push/PR gating afterward. The harness gained a --bazel flag so CI can point it at bazelisk. It is intentionally not a py_binary (a nested `bazel run` would deadlock on the output-base lock). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
bazel-diff servequery service returned an opaque400for commits it should have been able to serve:Two compounding causes:
resolveShadidn't verify the commit exists. JGit'sRepository.resolve("<40-hex>")parses the hex into anObjectIdwithout an object-DB lookup, andgit rev-parse <full-sha>echoes any well-formed SHA straight back. So a full SHA that isn't in the local clone "resolved" successfully, and the real failure (MissingObjectException) was deferred to the latercheckout— surfacing as a confusing checkout error.Fix
resolveShafor both git engines, raising a new retryableMissingRevisionException:RevWalk.parseCommit(objectId)(also peels annotated tags to their commit).rev-parse --verify <rev>^{commit}instead of a barerev-parse.ImpactedTargetsService.resolveBoth: if either revision is missing, run onegit fetchand retry. Serialized and double-checked via afetchLock, so a burst of requests for the same just-landed commit triggers at most one fetch.Resulting behavior
400 … Missing unknown …400(opaque)400after one refetch, clear message:revision '<x>' is missing from the local cloneThe
--gitEngine=subprocesspath had the same latent bug and is fixed the same way.Tests
All green via
bazel test:JGitClientTest— absent full SHA →MissingRevisionException; end-to-end repro (resolveShaSeesCommitOnlyAfterFetch: a commit lands in an origin after clone, is rejected locally, then resolves post-fetch()); non-commit object → hard error.GitClientTest(subprocess) — absent full SHA →MissingRevisionException.ImpactedTargetsServiceTest— refetch-then-succeeds; double-check skips the redundant fetch; persistent miss propagates after exactly one fetch.Ran
//cli:{GitClientTest,JGitClientTest,ImpactedTargetsServiceTest,BazelDiffServerTest,HashServiceTest,ServeCommandTest}(6/6 pass) and applied//cli/format(ktfmt).Notes for reviewers
GitClientExceptionis nowopensoMissingRevisionExceptioncan extend it; the HTTP layer's existingcatch (GitClientException)→400mapping still covers it.bazel-diff serveHTTP query service (#29) #393 / issue RFC: Bazel Query Service #29).🤖 Generated with Claude Code