ci(test): make the Codecov coverage upload a required signal - #85
Closed
bketelsen wants to merge 1 commit into
Closed
ci(test): make the Codecov coverage upload a required signal#85bketelsen wants to merge 1 commit into
bketelsen wants to merge 1 commit into
Conversation
codecov.yml gates pull requests on project and patch coverage statuses, but those statuses only exist once the Unit Tests job's coverage profile actually reaches Codecov. The upload step both asked the action not to fail on error (fail_ci_if_error: false) and suppressed the step's own failure (continue-on-error: true), so an upload or configuration failure left the job green with no coverage statuses to report — the gate could silently disappear. Set fail_ci_if_error: true and drop the step-level continue-on-error so an upload failure fails the Unit Tests job. The tokenless OIDC upload (use_oidc, id-token: write) and every codecov.yml threshold are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UigLsmd17TVDuaaSN4yfdU
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
codecov.ymlgates pull requests onprojectandpatchcoverage statuses, but those statuses only come into existence once theUnit Testsjob's coverage profile actually reaches Codecov.fail_ci_if_error: falsetold the action to swallow upload errors, andcontinue-on-error: truesuppressed the step's own failure. A failed or misconfigured upload therefore leftUnit Testsgreen with no coverage statuses reported — the coverage gate could vanish silently while CI stayed all-checks-passing.fail_ci_if_error: trueand remove the step-levelcontinue-on-errorso an upload or configuration failure fails theUnit Testsjob.use_oidc: trueplus the job-scopedid-token: writepermission), as is the action's pinned SHA.codecov.ymlis not touched:project(target: auto,threshold: 1%) andpatch(target: 70%,threshold: 5%) thresholds are byte-for-byte identical..github/workflows/test.yml, and touches only the Codecov upload step and its explanatory comment.What this uncovered
The change did its job on its very first run, and the answer is worse than a hypothetical: the upload is not merely fragile, it is already failing on every run, including
main.Codecov rejects the upload outright:
This is not a regression introduced here. The identical 404 appears in the
Unit Testsjob of the most recentTestsrun onmain(run 33028261901) —
and that run is recorded as
success, because the two suppressions thispull request removes hid it. Consequences of the status quo:
codecov/projectandcodecov/patchstatuses incodecov.ymlhave therefore never been posted, so every threshold in that file is currently inert. The coverage gate exists only on paper.Repository not foundis a Codecov-side account/activation error, not aworkflow defect — nothing in this repository can fix it. An operator must
activate
frostyard/firnon Codecov (and confirm the org's tokenless OIDCupload is enabled for it) before this pull request can go green. Once that is
done, re-running this job is the verification: the upload succeeds and the two
coverage statuses appear on this pull request for the first time.
Merging this branch before that operator action would turn a silently-missing
signal into a hard CI failure on every subsequent pull request. That trade is
the maintainer's call, which is why this stays a draft.
Test plan
actionlint .github/workflows/test.yml— exit 0, no findings (go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 .github/workflows/test.yml)make verify— passed:go mod tidy -diff,gofmt,golangci-lint 2.13.1(0 issues.),go vet ./...,go test ./...allokmake ci—==> ci passed(adds coverage,go test -race -short ./internal/..., and thelinux/amd64+linux/arm64cross-arch builds)git diff origin/main -- codecov.ymlis empty — the thresholds are unchanged07406bf):Repository gatepass,Security Scanpass,E2E Harness Syntaxpass,Unit Testsfail — failing exactly atUpload coverage to Codecovwith the 404 above, which is the intended new behavior firing against a genuinely broken upload.codecov/projectandcodecov/patchreported — blocked: requires the Codecov-side repository activation described above. Not achievable from this repository.Risk classification
High.
.github/workflows/**is a protected boundary inpolicies/agent-governance.json(workflow-and-permissions,decision: review-required,minimum_risk_tier: high), and the repository classifies by highest-applicable tier. The change only tightens a check —never_relaxlistsrequired-checksandcoverage-checks, and this moves in the permitted direction — but it converts a currently-and-permanently failing upload into a hard CI failure, so it must not merge before the Codecov activation lands.