feat(tools): add automated tool updater - #12359
Merged
Merged
Conversation
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.OpenSSF Scorecard
Scanned Files
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a repository-wide mechanism to centrally pin, validate, and automatically refresh external CLI tool versions and SHA-256 checksums via a YAML manifest and a new Go-based updater, then wires it into Make and GitHub Actions.
Changes:
- Adds
build/tools.yamlas the canonical tool manifest and commits the generatedbuild/tools.generated.mkinclude consumed by existingmake install-<tool>targets. - Introduces
cmd/tool-updaterplusinternal/toolinghelpers to validate the manifest, check release sources, and refresh checksums/versions (including syncing Terraform consumer files). - Adds a scheduled
update-toolsGitHub Actions workflow to runmake update-toolsand open/update a PR using a GitHub App token.
Reviewed changes
Copilot reviewed 40 out of 40 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/recipes/terraform/version.go | Updates comments to reflect tools manifest as the source of truth for Terraform pinning. |
| pkg/recipes/terraform/version_test.go | Updates test comments to match the new compatibility-file/updater flow. |
| Makefile | Reorders includes so tool metadata is available before build logic consumes it. |
| internal/tooling/updater.go | Implements version/checksum refresh logic and HTTP retrieval with retry/backoff. |
| internal/tooling/updater_test.go | Adds tests for version bumping, checksum refresh, and shared checksum-file caching. |
| internal/tooling/README.md | Documents updater usage and the tools manifest schema/behavior. |
| internal/tooling/manifest.go | Adds manifest schema, validation, template expansion, and file sync utilities. |
| internal/tooling/manifest_test.go | Adds repository-manifest parsing and helper-function tests. |
| docs/contributing/contributing-code/contributing-code-shell-and-make/README.md | Documents how contributors should manage external CLI pins via make update-tools. |
| cmd/tool-updater/main.go | Adds CLI entrypoint for manifest update and Make include regeneration. |
| build/tools.yaml | Adds the canonical tool manifest with sources, templates, assets, and checksums. |
| build/tools.mk | Switches to consuming generated metadata and adds make update-tools + generator rules. |
| build/tools.generated.mk | Adds committed generated Make metadata for tool versions and platform checksums. |
| build/scripts/install-yq.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-terraform.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-stern.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-shellcheck.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-oras.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-kubectl.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-kind.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-k3d.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-jq.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-helm.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-golangci-lint.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-dlv.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-dapr.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/scripts/install-bicep.sh | Updates comments to reflect tool data source as the manifest/generated include. |
| build/generate.mk | Removes unused CONTROLLER_TOOLS_VERSION variable. |
| build/build.mk | Updates Terraform version wiring comments to reflect tool-metadata include loading. |
| .github/workflows/validate-installers.yaml | Updates workflow comments to point at build/tools.yaml as pin source. |
| .github/workflows/validate-bicep.yaml | Updates workflow comments to point at build/tools.yaml as pin source. |
| .github/workflows/update-tools.yaml | Adds scheduled/manual workflow to run updater and open/update a PR via GitHub App token. |
| .github/workflows/unit-tests.yaml | Updates workflow comments to point at build/tools.yaml as pin source. |
| .github/workflows/lint.yaml | Updates workflow comments to point at build/tools.yaml as pin source. |
| .github/workflows/functional-test-noncloud.yaml | Updates workflow comments to point at build/tools.yaml as pin source. |
| .github/workflows/functional-test-cloud.yaml | Updates workflow comments to point at build/tools.yaml as pin source. |
| .github/workflows/copilot-setup-steps.yml | Expands path triggers and updates comments to reference the manifest-based pins. |
| .github/workflows/build.yaml | Updates workflow comments to point at build/tools.yaml as pin source. |
| .github/actions/create-kind-cluster/action.yaml | Updates composite action comment to reference manifest-based pinning. |
| .devcontainer/post-create.sh | Updates devcontainer post-create comments to reference manifest-based pinning. |
DariuszPorowski
force-pushed
the
fp/whole-takin-lime
branch
3 times, most recently
from
July 9, 2026 20:36
51f876b to
7d6e18b
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #12359 +/- ##
==========================================
- Coverage 54.06% 54.05% -0.02%
==========================================
Files 765 765
Lines 50830 50830
==========================================
- Hits 27480 27474 -6
- Misses 20761 20764 +3
- Partials 2589 2592 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
DariuszPorowski
force-pushed
the
fp/whole-takin-lime
branch
3 times, most recently
from
July 9, 2026 23:05
fe5d5af to
3896335
Compare
DariuszPorowski
enabled auto-merge
July 9, 2026 23:17
Functional Tests - corerp-noncloud180 tests 178 ✅ 1h 21m 42s ⏱️ Results for commit 0ab6a6d. ♻️ This comment has been updated with latest results. |
Functional Tests - upgrade-noncloud3 tests 3 ✅ 3m 35s ⏱️ Results for commit 0ab6a6d. ♻️ This comment has been updated with latest results. |
1 task
DariuszPorowski
force-pushed
the
fp/whole-takin-lime
branch
from
July 13, 2026 17:37
3896335 to
379ddff
Compare
willdavsmith
self-requested a review
July 13, 2026 18:20
DariuszPorowski
force-pushed
the
fp/whole-takin-lime
branch
from
July 13, 2026 23:34
379ddff to
90597c2
Compare
Functional Tests - ucp-cloud4 tests 4 ✅ 33s ⏱️ Results for commit 90597c2. ♻️ This comment has been updated with latest results. |
- Implemented a new updater for managing tool versions and checksums in the manifest. - Added functionality to retrieve the latest version and checksum from various sources. - Created tests to ensure the updater correctly refreshes versions and checksums. - Introduced a new HTTP client for making requests to version sources. - Updated the Terraform version handling to synchronize with the manifest. - Added tests to verify the correctness of version and checksum updates. - Ensured that comments and documentation reflect the new functionality and structure. Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
…retrieval Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
…on tests Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
…s-pr.sh Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
DariuszPorowski
force-pushed
the
fp/whole-takin-lime
branch
from
July 29, 2026 18:48
c039c49 to
178b0d5
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 44 out of 44 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (2)
internal/tooling/manifest_test.go:29
- TestLoadRepositoryManifest verifies build/tools.generated.mk matches build/tools.yaml, but it doesn’t assert that the Terraform compatibility file (.terraform-version) stays in sync with the manifest’s terraform tool version. Since other code/docs treat build/tools.yaml as canonical, add a check here to prevent drift between the manifest and .terraform-version.
.github/workflows/copilot-setup-steps.yml:181 - The Terraform setup comment still says the version defaults to .terraform-version, but make now sources TERRAFORM_VERSION from the generated tool metadata include (synced from build/tools.yaml). Update the comment to avoid implying that workflows read .terraform-version directly.
- name: Install Terraform
# Version defaults to .terraform-version (overridable via TERRAFORM_VERSION)
# and is compiled into the rad binary; checksums pinned in build/tools.yaml.
run: make install-terraform
…ksums while preserving formatting feat(tests): add test for updateManifestYAML to ensure only changed values are rewritten fix(updater): improve error handling in writeResponse for copy operation fix(script): implement safer git push with push_branch function Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 44 out of 44 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (3)
build/tools.mk:23
TOOL_UPDATER_GOOS/TOOL_UPDATER_GOARCH/TOOL_UPDATER_BINARYare defined with immediate (:=) expansion, which runsgo envwhile parsing the Makefile even for unrelated targets (e.g.make help). Use recursive (=) assignment sogo envis only executed when the updater variables are actually needed.
internal/tooling/README.md:93- The updater rewrites versions/checksums in-place and currently requires the existing YAML scalar values to be plain (unquoted) scalars (see
applyScalarEditserror path). Document this constraint so contributors don’t accidentally quoteversion/checksumfields and breakmake update-tools.
.github/scripts/update-tools-pr.sh:66 git commitwill fail with exit code 1 when the script is run with no staged changes (e.g. a local rerun). Add a cheap guard aftergit add -Ato exit cleanly when there’s nothing to commit.
git checkout -B "${PR_BRANCH}"
git add -A
git commit --signoff -m "${PR_TITLE}"
gh auth setup-git
push_branch
lakshmimsft
reviewed
Jul 29, 2026
lakshmimsft
reviewed
Jul 29, 2026
… to global.terraform.version Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
…ocking on signing prompt Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Radius functional test overviewClick here to see the test run details
Test Status⌛ Building Radius and pushing container images for functional tests... |
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 45 out of 45 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (3)
internal/tooling/manifest_test.go:50
- This test mutates the real build/tools.yaml by string-replacing "update: false". That makes the test fragile: if the repository manifest no longer contains that exact substring (or uses a different formatting), the test fails for reasons unrelated to unknown-field handling. Use a small inline YAML fixture with an intentionally misspelled field instead of depending on the repo manifest contents.
internal/tooling/manifest_test.go:360 - This test assumes manifest.Tools[0] is a checksum-bearing tool with a non-nil Platforms map. If the repository manifest is reordered (or starts with a checksumSource.type: none tool), this will panic or stop testing the intended behavior. Pick the first checksum-bearing tool explicitly (or use a minimal fixture YAML) so the test remains stable as tools.yaml evolves.
.github/workflows/copilot-setup-steps.yml:180 - This comment is now misleading: TERRAFORM_VERSION no longer defaults from .terraform-version at Make parse time (it comes from build/tools.generated.mk, generated from build/tools.yaml). .terraform-version is now a compatibility file kept in sync by the updater. Please update the comment to match the new source of truth.
# Version defaults to .terraform-version (overridable via TERRAFORM_VERSION)
# and is compiled into the rad binary; checksums pinned in build/tools.yaml.
lakshmimsft
approved these changes
Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This pull request adds an automated updater for Radius's external CLI tool versions and SHA-256 checksums.
The change:
build/tools.yamlas the canonical tool manifest and commits the generatedbuild/tools.generated.mkMake include.tool-updaterGo command with manifest validation, release-source checks, checksum refreshes, and Terraform version synchronization..github/workflows/update-tools.yamlworkflow, which opens or updates a PR using the Dependabot-manager GitHub App credentials.make install-<tool>targets and uses a stablebin/tool-updaterpath to avoid Windows temporary-executable elevation blocks.internal/tooling/README.md.Type of change
Fixes: #issue_number
Contributor checklist
eng/design-notes/in this repository, if new APIs are being introduced.Validation
go test ./internal/tooling ./cmd/tool-updatergo vet ./internal/tooling ./cmd/tool-updateractionlint