Repository navigation
feat(codeartifact): support the cargo (crates.io) registry format - #8370
Conversation
PR Summary by QodoAdd Cargo registry and crates.io support to CodeArtifact
AI Description
Diagram
High-Level Assessment
Files changed (19)
|
Code Review by Qodo
1.
|
a4b479e to
73fe462
Compare
CodeArtifact's Rust/Cargo repositories had no native support at all: no sidecar, no wire protocol, no external connection. Adds it following the same shape as the existing Maven/npm/pypi/NuGet container-backed formats. KellnrSidecarManager/KellnrSidecarClient run a real Kellnr container per cargo-enabled repository and speak its native HTTP API for publish, download, and the sparse index. CodeArtifactCargoController implements the real `cargo` sparse-registry wire protocol (publish, download, sparse index lookup, config.json) and forwards Kellnr's own publish rejections (e.g. republishing an existing version, which the Cargo Book documents as permanently disallowed) verbatim rather than swallowing them into an opaque error. Adds the `public:crates-io` external connection via CargoExternalConnectionClient (real index.crates.io/static.crates.io wire calls) and CodeArtifactCargoExternalConnectionImporter, reusing the existing claim/fence/retention-origin machinery the same way NuGet's own importer does. Cargo's sparse index covers every version of a crate name in one file and a real cargo client always consults it first, so unlike the other formats' importers this one also merges real upstream-only version lines into the local index response (mergeExternalVersions), a read-only proxy step with no retention write of its own, so an upstream-only crate is actually visible to a client's dependency resolution before anything is retained. Fixes against the live Kellnr/crates.io protocols found while addressing review: KellnrSidecarClient now implements listPackages (its absence made every ListPackages call 500 once cargo joined the container-backed formats); sparse-index bucket paths are lowercased per the Cargo Book's own spec (crates.io's own index is case-sensitive about this even though a crate's real name isn't); the admin account behind Kellnr's internal publish token is given a fresh random password per container rather than Kellnr's own shared default; a stale cached publish token (e.g. after a silent container restart) is retried once rather than 401ing forever; both publish-body parsing and the external sparse-index fetch are now bounded against an oversized allocation; a renamed dependency's alias and real name are mapped correctly between the index and publish shapes, and features2/links/rust_version now survive a retained crate's synthesized metadata; the Kellnr image is pinned to the exact version this was built and tested against rather than a floating major tag. Known gaps, called out in docs/services/codeartifact.md: no repository-to-repository upstream importer yet for cargo (every other native format already has one), and no yank/owner/search routes.
73fe462 to
03e2440
Compare
|
🎉 This PR is included in version 2.5.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Summary
CodeArtifact's Rust/Cargo repositories currently have no native support at all: no sidecar, no
wire protocol, no external connection. This adds it, following the same shape as the existing
Maven/npm/pypi/NuGet container-backed formats.
KellnrSidecarManager/KellnrSidecarClientrun a real Kellnrcontainer per cargo-enabled repository and speak its native HTTP API for publish, download, and
the sparse index.
CodeArtifactCargoControllerimplements the realcargosparse-registry wire protocol (publish,download, sparse index lookup,
config.json), and forwards Kellnr's own publish rejections (e.g.republishing an existing version, which the Cargo Book documents as permanently disallowed)
verbatim rather than swallowing them into an opaque error.
public:crates-ioexternal connection is wired up viaCargoExternalConnectionClient(realindex.crates.io/static.crates.iowire calls) andCodeArtifactCargoExternalConnectionImporter,reusing the same claim/fence/retention-origin machinery NuGet's own importer already uses. Cargo's
sparse index covers every version of a crate name in one file, and a real
cargoclient alwaysconsults it before anything else, so unlike the other formats' importers this one also merges
real upstream-only version lines into the local index response — a read-only proxy step with no
retention write of its own — so an upstream-only crate is actually visible to dependency
resolution before anything is retained.
Known gaps, called out in
docs/services/codeartifact.md: no repository-to-repository upstreamimporter yet for cargo (every other native format already has one), and no yank/owner/search
routes. Both are natural follow-ups, not bundled into this PR.
Update after review: fixed
KellnrSidecarClientmissing alistPackagesoverride (made everyListPackagescall 500 once cargo joined the container-backed formats), sparse-index bucket pathsnot being lowercased per the Cargo Book's own spec (confirmed live: a mixed-case crate like
Inflectorgenuinely 404s at its original-case path against the realindex.crates.io), the adminaccount behind Kellnr's internal publish token sitting at Kellnr's own shared
admin/admindefault instead of a fresh per-container random password, a cached publish token never recovering
from a silent container restart, two unbounded-allocation spots (the publish body's own declared
metadata length, and the external sparse-index fetch), a renamed dependency's alias/real-name fields
swapped between the index and publish shapes,
features2/links/rust_versionbeing dropped froma retained crate's synthesized metadata, and the Kellnr image floating on an unpinned major tag.
Type of change
fix:)feat:)feat!:orfix!:)AWS Compatibility
Verified against the real
cargowire protocol (Cargo Book'sregistry-web-api/publishingpages) and against the real, live
index.crates.io/static.crates.ioproduction endpointsdirectly (sparse-index bucketing, line shape, download route, and a live
cksum-vs-downloaded-bytescheck), not just Kellnr's own behavior. AWS's own
external-connection.htmldocumentspublic:crates-ioas a supported external connection;CodeArtifactServicealready had thepublic:crates-io -> cargomapping scaffolded in before this PR.Checklist
./mvnw testpasses locally