Skip to content

chore(ci): harden the release supply chain - #128

Draft
perryqh wants to merge 1 commit into
mainfrom
chore/release-supply-chain
Draft

chore(ci): harden the release supply chain#128
perryqh wants to merge 1 commit into
mainfrom
chore/release-supply-chain

Conversation

@perryqh

@perryqh perryqh commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Why

We distribute codeowners via DotSlash. Reading verify_artifact() settled what actually protects the binaries: DotSlash checks the size and BLAKE3/SHA-256 digest recorded in the DotSlash file, and nothing else. There is no signature verification anywhere in the tool.

So for the DotSlash path the digest is the mechanism, and it is only as trustworthy as the file holding it. That reframes the work around how that file is generated and consumed.

The main fix is documentation

README.md told users to download the DotSlash file from a release and run it. If you re-fetch the DotSlash file next to the binary every time, the digest verifies nothing — you're trusting whatever the release currently claims, and an attacker who can modify the release swaps both.

DotSlash's own motivation doc is explicit: "the provenance of a DotSlash file is paramount" and DotSlash files "are meant to live in source control." The README now says to commit the file and upgrade via a reviewed diff, and explains why.

Pipeline changes

  • Draft → publish. The release is created with --draft and published only after every asset lands. Prerequisite for immutable releases: once published, gh release upload fails with Cannot upload assets to an immutable release (attest-build-provenance#734).
  • Build provenance via actions/attest-build-provenance, attested after packaging so the covered digest matches what users download.
  • exclude-http-provider on the DotSlash config. Generating against a draft bakes a permanently dead provider URL into the published file. A draft has no git tag, so GitHub serves assets under an untagged-<hash> path and the generator records that verbatim. Publishing rewrites the real URLs, but the DotSlash file is already uploaded — and under immutable releases it cannot be replaced. Found by running the pipeline, not by reading it (see Testing).
  • DotSlash assets matched by exact name instead of regex. ^codeowners-mac is prefix-only and unanchored, and the generator takes the first match over dict order. The generated DotSlash file is itself named codeowners, so on a re-run a prefix pattern could select it as the binary.
  • All third-party actions pinned to commit SHAs, plus .github/dependabot.yml so the pins don't go stale. Only codeql.yml stays on @main, which already carries a documented zizmor: ignore. cross pinned to 0.2.5.
  • Ad-hoc re-sign after lipo, per Apple DTS guidance, with codesign --verify so a regression fails in CI rather than for a user.
  • Idempotent re-runs. Reuse an existing draft rather than creating a duplicate (a draft creates no tag, so GitHub does not enforce tag uniqueness), --clobber on uploads, and a concurrency group that queues on main rather than cancelling mid-release.
  • Fixed if: success() && ${{ ... }} on the DotSlash job — mixing a bare expression with an interpolation doesn't evaluate as written.

Testing

Local: clippy, cargo fmt --check, and the full test suite pass (this PR touches no Rust).

The release path doesn't run outside main, so I ran the whole pipeline twice in a throwaway public repo with Immutable releases enabled:

  • dotslash-publish-release does read assets from a draft release. This was the main open question.
  • All 8 jobs pass in order: check/test/lintsrelease (draft) → both uploads + attest → generate-dotslash-filespublish-release.
  • The published release reports draft: false, immutable: true with all four assets, confirming the ordering is what makes immutable releases workable.
  • gh attestation verify passes against the published macOS tarball.
  • The generated DotSlash file executes: dotslash codeowners --versioncodeowners 0.3.6. That covers the consumer path end to end — provider resolution, size and BLAKE3 enforcement, unpack, run. It also confirms the post-lipo re-sign is valid, since unsigned code is SIGKILLed on Apple silicon rather than failing quietly.
  • The second run had a prior release present, so --generate-notes and the resolve-draft-by-pending-tag path were exercised with history rather than on an empty repo.

The first run is what surfaced the untagged- provider bug: it shipped a DotSlash file whose primary provider 404'd on all four platforms. After exclude-http-provider, the regenerated file carries exactly one github-release provider per platform.

Still unverified: the re-run-after-partial-failure path. The idempotency logic is written for it, but a green run never takes that branch.

Known gap: Developer ID signing

macOS binaries are ad-hoc signed only. Ad-hoc signing satisfies Apple silicon's requirement that all code be signed, but it asserts nothing about who built the binary and does not satisfy Gatekeeper — so a browser download (which sets com.apple.quarantine, unlike DotSlash or curl) is blocked. The README now documents this and the xattr -d workaround.

Real Developer ID signing plus notarization is the fix, and Gusto/gusto-cli already does it from a Linux runner via rcodesign. Porting that here needs a decision about where the signing credential lives, since this is a public repo outside the Gusto org. Tracked separately from this PR.

Requires a manual step

Enable Settings → General → Immutable releases after merge. The draft→publish restructure exists to make that safe; before this change, enabling it would have broken the next version bump.

@perryqh

perryqh commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

Applied the review findings in 81a4041:

  • Idempotent draft creation. Reuse an existing draft rather than creating a duplicate. A draft creates no git tag, so GitHub does not enforce tag uniqueness for drafts, and gh's lookup returns the first draft release that has tagName as its pending tag — reachable via "Re-run all jobs" after a failed upload.
  • Concurrency group. Queue on main (cancelling mid-release would strand a partial draft), cancel elsewhere via cancel-in-progress: ${{ github.ref != 'refs/heads/main' }} so PR feedback stays fast.
  • --clobber on all three asset uploads.
  • Pinned cargo install cross --version 0.2.5 — the largest unpinned input left in the release path, and what an unpinned install resolves to today, so no behavior change. Deliberately not --locked: cross 0.2.5's lockfile predates current rustc and this build cannot be exercised outside main.
  • README now states that releases up to and including v0.3.4 predate attestation, so gh attestation verify reporting no match for them is expected rather than a tampering signal. Avoided naming a future version, which would go stale if the next bump is a minor.
  • Dropped the redundant codesign -dv.

I verified the idempotency shell logic against all three states (no release / existing draft / existing published release), including that set -e does not abort on the failed lookup inside the condition.

Not applied, deliberately:

  • actionlint in CI — I could not download it to verify it passes against the current workflows, and adding a gate I have not run is how you get a red main. Worth a follow-up.
  • The grep "\\+version" fragility in version detection — a [dependencies.foo] section with a version key would extract the wrong string and cut a bogus release. Not currently triggerable (Cargo.toml has no such sections), and it is the riskiest line in the file to touch in a PR whose release path CI cannot exercise. Follow-up issue.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the release supply chain for codeowners distribution via DotSlash by making the DotSlash file’s provenance reviewable and by strengthening the CI release workflow (draft-first publishing, provenance attestations, and pinned actions).

Changes:

  • Updated README guidance to commit the DotSlash file in-repo and explain why that’s necessary for digest verification to be meaningful.
  • Restructured the release workflow to create draft releases first, upload/attest packaged assets, generate DotSlash files, then publish the release.
  • Pinned third-party GitHub Actions to commit SHAs and added Dependabot configuration to keep those pins updated.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
README.md Updates DotSlash usage and adds guidance for independently verifying release provenance.
.github/workflows/dotslash-config.json Switches DotSlash generator selection from regex matching to exact asset names.
.github/workflows/ci.yml Implements draft→upload/attest→dotslash→publish flow; adds concurrency control; pins actions to SHAs.
.github/workflows/audit.yml Pins actions to SHAs for the security audit workflow.
.github/dependabot.yml Adds Dependabot updates for GitHub Actions SHA pins.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread README.md
```sh
gh attestation verify codeowners-mac.tar.gz \
--repo rubyatscale/codeowners-rs \
--signer-workflow rubyatscale/codeowners-rs/.github/workflows/ci.yml

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this against a live attestation rather than the docs, and the current README form is the correct one — the suggested change would break verification.

gh's own help text specifies the format as owner/repo/path, not a repo-relative path:

--signer-workflow string   Enforce that the workflow that signed the attestation
                           matches the provided value
                           ([host/]<owner>/<repo>/<path>/<to>/<workflow>)

I ran both forms against a real attested codeowners-mac.tar.gz from a full pipeline run in a throwaway repo:

# README's form — passes
gh attestation verify codeowners-mac.tar.gz \
  --repo OWNER/REPO --signer-workflow OWNER/REPO/.github/workflows/ci.yml
# exit 0

# suggested form — fails
gh attestation verify codeowners-mac.tar.gz \
  --repo OWNER/REPO --signer-workflow .github/workflows/ci.yml
# Error: verifying with issuer "sigstore.dev"
# exit 1

The reason is that --signer-workflow is matched against the SAN in the Fulcio signing certificate, which is a full URI identifying the workflow including its repository — it is not a path lookup within --repo. So the owner/repo/ prefix is load-bearing, and --repo does not supply it.

Leaving the README as-is. Thanks for the flag though — this one is worth having pinned down explicitly, since the failure mode is a verification command that looks stricter but silently never matches.

@perryqh
perryqh marked this pull request as ready for review August 26, 2026 22:44
@perryqh
perryqh requested a review from a team as a code owner August 26, 2026 22:44
@perryqh
perryqh marked this pull request as draft August 26, 2026 23:36
Binaries are distributed via DotSlash, which verifies only the `size` and
BLAKE3 `digest` recorded in the DotSlash file -- there is no signature check
anywhere in the tool. The digest is the whole mechanism, and it is only as
trustworthy as the file holding it. This reworks how that file is produced
and consumed, and closes the gaps around it.

README:

- Commit the DotSlash file and upgrade via a reviewed diff. Re-fetching it
  next to the binary means trusting whatever the release currently claims,
  so the digest verifies nothing. This is DotSlash's own guidance.
- Document `gh attestation verify`, and note that ad-hoc signing does not
  satisfy Gatekeeper for browser downloads.

Pipeline:

- Create the release as a draft, publish only once every asset has landed.
  Required for immutable releases, which reject uploads to a published
  release.
- Attest build provenance after packaging, so the digest the attestation
  covers matches what users download.
- Pin third-party actions to commit SHAs; add dependabot to keep the pins
  fresh. Pin `cross` to 0.2.5.
- Match DotSlash assets by exact name. An unanchored prefix regex could
  select the generated DotSlash file itself as a binary on a re-run.
- Set `exclude-http-provider`. Generating against a draft bakes an
  `untagged-<hash>` asset URL into the published file; it 404s once the real
  tag exists and cannot be corrected under immutable releases.
- Re-sign after `lipo` so the universal binary carries one coherent ad-hoc
  signature, and verify rather than assume.
- Reuse an existing draft instead of creating a duplicate, `--clobber` on
  uploads, and a concurrency group that queues on main.

Known gap: macOS Developer ID signing and notarization. Ad-hoc signing
asserts nothing about who built the binary; provenance currently comes from
the attestations. Tracked separately.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@perryqh
perryqh force-pushed the chore/release-supply-chain branch from 7d3c19b to 1f17a23 Compare August 26, 2026 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

2 participants