-
Notifications
You must be signed in to change notification settings - Fork 10
feat: Add reusable workflow to sync branches and tags #158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
bthomee
wants to merge
10
commits into
main
Choose a base branch
from
bthomee/sync
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 5 commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
6937960
feat: Add reusable workflow to sync branches and tags
bthomee 06e22f0
Fix unbound variable error
bthomee ec579aa
Address review feedback
bthomee 89388bf
Address review feedback
bthomee 470b9bc
Get an fresh app token in each job so passing it around isn't needed …
bthomee e99b445
Merge branch 'main' into bthomee/sync
bthomee ee02f97
Address review feedback
bthomee 3d7dc73
Address review feedback
bthomee 272bf7c
Gate on XRPLF repo
bthomee 1f20bc1
Change logic to not run on fork PRs
bthomee File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,134 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # Synchronize important branches from a source repository into a destination repository. Missing | ||
| # branches are always created. By default branches are fast-forwarded only, while a diverged branch | ||
| # aborts the run. Force mode overwrites diverged branches with a force-push (using | ||
| # --force-with-lease). Dry-run mode reports what would change without pushing anything. | ||
| # | ||
| # Configuration is supplied via environment variables: | ||
| # | ||
| # Required: | ||
| # GH_TOKEN Token with read access to the source repository and write access to the | ||
| # destination. A public repository ignores the token as long as it is valid. When | ||
| # running this script locally, your personal access token must have at least the | ||
| # "repo", "read:org" (for branches in organization repos), and "workflow" scopes. | ||
| # SOURCE_REPO Source repository, as "owner/repo" (github.com is assumed). | ||
| # DEST_REPO Destination repository, as "owner/repo" (github.com is assumed). | ||
| # BRANCHES_GLOB Glob matching the branches to synchronize. A plain branch name is a glob that | ||
| # matches only itself; a wider glob may match several branches, all of which are | ||
| # synchronized. | ||
| # | ||
| # Optional: | ||
| # DRY_RUN "true" to report changes without pushing. | ||
| # (default: false) | ||
| # FORCE "true" to overwrite diverged destination branches. | ||
| # (default: false) | ||
| # | ||
| # Exit status: | ||
| # 0 All matched branches synchronized successfully (or a clean dry run was performed). | ||
| # non-zero Synchronization failed, and the run stopped at the first branch that could not be | ||
| # synchronized. Failures that git detects (an unreachable remote, a missing branch, a | ||
| # rejected push) exit with git's own status and message. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| # shellcheck source=sync-common-functions.sh | ||
| source "${BASH_SOURCE[0]%/*}/sync-common-functions.sh" | ||
|
|
||
| # --- Configuration ----------------------------------------------------------------------------- | ||
|
|
||
| set_common_defaults | ||
| BRANCHES_GLOB="${BRANCHES_GLOB:-}" | ||
|
|
||
| # Internal git ref/remote names used within the working repository. | ||
| BRANCH_REF="refs/heads" | ||
| SOURCE_REF="refs/remotes/source" | ||
| DEST_REF="refs/remotes/dest" | ||
| SOURCE_REMOTE="source" | ||
| DEST_REMOTE="dest" | ||
| REMOTE_REF_PREFIX="${BRANCH_REF}" | ||
|
|
||
| # --- Helpers ----------------------------------------------------------------------------------- | ||
|
|
||
| validate_config() { | ||
| validate_common_config | ||
| [ -n "${BRANCHES_GLOB}" ] || die "BRANCHES_GLOB must be a non-empty branch name or glob." | ||
| } | ||
|
|
||
| # Fetch the branches matching the glob from both remotes into local tracking refs. The '+' prefix | ||
| # force-updates the tracking refs so they always reflect each remote. | ||
| fetch_branches() { | ||
| echo "Fetching source branch(es): ${BRANCHES_GLOB}" | ||
| git fetch --quiet "${SOURCE_REMOTE}" \ | ||
| "+${BRANCH_REF}/${BRANCHES_GLOB}:${SOURCE_REF}/${BRANCHES_GLOB}" | ||
|
|
||
| # A glob that matches no refs makes 'git fetch' succeed having fetched nothing, so a mistyped | ||
| # BRANCHES_GLOB would otherwise go undetected. | ||
| [ -n "$(git for-each-ref "${SOURCE_REF}/")" ] || | ||
| die "Source branch(es) '${BRANCHES_GLOB}' not found in '${SOURCE_REPO}'." | ||
|
|
||
| # A branch that does not yet exist on the destination is expected: it is created below. A literal | ||
| # branch name that is missing makes this fetch fail (a glob matching nothing succeeds), so | ||
| # failures are ignored here; a genuine problem such as an unreachable remote or an invalid token | ||
| # surfaces when pushing. | ||
| echo "Fetching destination branch(es): ${BRANCHES_GLOB}" | ||
| git fetch --quiet "${DEST_REMOTE}" \ | ||
| "+${BRANCH_REF}/${BRANCHES_GLOB}:${DEST_REF}/${BRANCHES_GLOB}" || true | ||
| } | ||
|
|
||
| # Synchronize a single already-fetched ref onto the destination. | ||
| sync_ref() { | ||
| local ref="$1" | ||
|
|
||
| # Create branches that do not yet exist on the destination. The refspec is not forced, so if a | ||
| # racing push created the branch between our fetch and now, the push is rejected rather than | ||
| # clobbering the destination. | ||
| if ! git show-ref --verify --quiet "${DEST_REF}/${ref}"; then | ||
| echo "Creating: ${ref}" | ||
| push_ref "${ref}" | ||
| return | ||
| fi | ||
|
|
||
| # Overwrite diverged branches when force is enabled. The --force-with-lease flag guards against | ||
| # clobbering destination commits that appeared after our fetch. | ||
| if [ "${FORCE}" = "true" ]; then | ||
| echo "Overwriting: ${ref}" | ||
| push_ref "${ref}" --force-with-lease | ||
| return | ||
| fi | ||
|
|
||
| # Check out the destination branch and fast-forward it to the source. | ||
| echo "Syncing: ${ref}" | ||
| git checkout --quiet -B "${ref}" "${DEST_REF}/${ref}" | ||
| git merge --ff-only "${SOURCE_REF}/${ref}" || | ||
| die "Branch '${ref}' has diverged and cannot be fast-forwarded; use FORCE to overwrite it." | ||
| git push "${PUSH_OPTS[@]+"${PUSH_OPTS[@]}"}" "${DEST_REMOTE}" "HEAD:${BRANCH_REF}/${ref}" | ||
| } | ||
|
|
||
| # Synchronize every ref that was fetched under SOURCE_REF (a glob may match several). | ||
| sync_branches() { | ||
| # A dry run adds --dry-run to every push so nothing is actually written. The options are expanded | ||
| # as "${PUSH_OPTS[@]+"${PUSH_OPTS[@]}"}" to avoid the "unbound variable" error that an empty | ||
| # array triggers under 'set -u' on older versions of Bash. | ||
| PUSH_OPTS=() | ||
| if [ "${DRY_RUN}" = "true" ]; then | ||
| echo "Dry run: no branches will be pushed." | ||
| PUSH_OPTS+=("--dry-run") | ||
| fi | ||
|
|
||
| local ref | ||
| while IFS= read -r ref; do | ||
| sync_ref "${ref}" | ||
| done < <(git for-each-ref --format='%(refname:lstrip=3)' "${SOURCE_REF}/") | ||
| } | ||
|
|
||
| # --- Main -------------------------------------------------------------------------------------- | ||
|
|
||
| main() { | ||
| validate_config | ||
| setup_working_repo | ||
| fetch_branches | ||
| sync_branches | ||
| } | ||
|
|
||
| main "$@" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| # shellcheck shell=bash | ||
| # | ||
| # Common helpers shared by sync-branches.sh and sync-tags.sh. | ||
|
|
||
| # Emit an error and exit. | ||
| die() { | ||
| echo "::error::$1" >&2 | ||
| exit 1 | ||
| } | ||
|
|
||
| # Apply the defaults shared by all sync scripts, so that only the variables that differ from the | ||
| # default need to be set. The workflow passes DRY_RUN explicitly; FORCE is only ever set when running | ||
| # a script directly, as the workflow deliberately does not expose force-pushing. | ||
| set_common_defaults() { | ||
| DRY_RUN="${DRY_RUN:-false}" | ||
| FORCE="${FORCE:-false}" | ||
| } | ||
|
|
||
| # Push the given source ref to the destination. Reads the SOURCE_REF and REMOTE_REF_PREFIX globals | ||
| # (the caller sets REMOTE_REF_PREFIX to "refs/heads" or "refs/tags") and the PUSH_OPTS global (e.g. | ||
| # --dry-run). Any extra arguments (e.g. --force-with-lease) are inserted before the refspec. | ||
| # Args: <ref> [git-push-option...] | ||
| push_ref() { | ||
| local ref="$1" | ||
| shift | ||
| git push "$@" "${PUSH_OPTS[@]+"${PUSH_OPTS[@]}"}" "${DEST_REMOTE}" \ | ||
| "${SOURCE_REF}/${ref}:${REMOTE_REF_PREFIX}/${ref}" | ||
| } | ||
|
|
||
| # Add a remote pointing at a github.com repository and authenticate to it with GH_TOKEN. The token | ||
| # is passed via an HTTP Authorization header rather than embedded in the remote URL. Embedding would | ||
| # persist the credential in .git/config and, because git prints the remote URL in its error | ||
| # messages, risk leaking it into CI logs. Passing it as an 'extraheader' keeps git's output limited | ||
| # to 'https://github.com/<repo>'. The header is scoped to the single repository rather than to | ||
| # github.com as a whole, so that the token for one remote is never sent to the other. The 'tr -d' | ||
| # command strips the line breaks GNU base64 inserts (it wraps at 76 columns, and an encoded token | ||
| # exceeds that) so the header stays on one line. A public repository ignores a valid token, so the | ||
| # same header is used for both. | ||
| # Args: <remote-name> <owner/repo> | ||
| add_authenticated_remote() { | ||
| local remote="$1" repo="$2" | ||
| local url="https://github.com/${repo}" | ||
| git remote add "${remote}" "${url}" | ||
| git config --local "http.${url}/.extraheader" \ | ||
| "AUTHORIZATION: basic $(printf 'x-access-token:%s' "${GH_TOKEN}" | base64 | tr -d '\n')" | ||
| } | ||
|
|
||
| # Validate configuration shared by all sync scripts. | ||
| validate_common_config() { | ||
| [ -n "${GH_TOKEN:-}" ] || die "GH_TOKEN is required but not set." | ||
| [ -n "${SOURCE_REPO:-}" ] || die "SOURCE_REPO is required but not set." | ||
| [ -n "${DEST_REPO:-}" ] || die "DEST_REPO is required but not set." | ||
|
|
||
| case "${DRY_RUN}" in | ||
| true | false) ;; | ||
| *) die "DRY_RUN must be 'true' or 'false', got '${DRY_RUN}'." ;; | ||
| esac | ||
|
|
||
| case "${FORCE}" in | ||
| true | false) ;; | ||
| *) die "FORCE must be 'true' or 'false', got '${FORCE}'." ;; | ||
| esac | ||
| } | ||
|
|
||
| # Initialize a fresh working repository in a temporary directory and wire up the authenticated | ||
| # remotes. | ||
| setup_working_repo() { | ||
| WORKDIR="$(mktemp -d)" | ||
| trap 'rm -rf "${WORKDIR}"' EXIT | ||
| cd "${WORKDIR}" | ||
|
|
||
| git init --quiet | ||
| # Both remotes carry the token; a public source simply ignores it as long as the token is valid. | ||
| add_authenticated_remote "${SOURCE_REMOTE}" "${SOURCE_REPO}" | ||
| add_authenticated_remote "${DEST_REMOTE}" "${DEST_REPO}" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # Synchronize tags from a source repository into a destination repository. Missing tags are always | ||
| # created. By default existing tags that match on both sides are a no-op, while a diverged tag (same | ||
| # name, different object) aborts the run. Force mode overwrites diverged tags with a force-push | ||
| # (using --force-with-lease). Dry-run mode reports what would change without pushing anything. | ||
| # | ||
| # This script is intended to run after the branches have been synchronized (see sync-branches.sh), | ||
| # so that tagged commits are already reachable from a branch in the destination before their tags | ||
| # arrive. | ||
| # | ||
| # Configuration is supplied via environment variables: | ||
| # | ||
| # Required: | ||
| # GH_TOKEN Token with read access to the source repository and write access to the | ||
| # destination. A public repository ignores the token as long as it is valid. When | ||
| # running this script locally, your personal access token must have at least the | ||
| # "repo", "read:org" (for tags in organization repos), and "workflow" scopes. | ||
| # SOURCE_REPO Source repository, as "owner/repo" (github.com is assumed). | ||
| # DEST_REPO Destination repository, as "owner/repo" (github.com is assumed). | ||
| # | ||
| # Optional: | ||
| # DRY_RUN "true" to report changes without pushing. | ||
| # (default: false) | ||
| # FORCE "true" to overwrite diverged destination tags. | ||
| # (default: false) | ||
| # | ||
| # Exit status: | ||
| # 0 All tags synchronized successfully (or a clean dry run was performed). | ||
| # non-zero Synchronization failed, and the run stopped at the first tag that could not be | ||
| # synchronized. Failures that git detects (an unreachable remote, a rejected push) exit | ||
| # with git's own status and message. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| # shellcheck source=sync-common-functions.sh | ||
| source "${BASH_SOURCE[0]%/*}/sync-common-functions.sh" | ||
|
|
||
| # --- Configuration ----------------------------------------------------------------------------- | ||
|
|
||
| set_common_defaults | ||
|
|
||
| # Internal git ref/remote names used within the working repository. | ||
| TAG_REF="refs/tags" | ||
| SOURCE_REF="refs/remotes/source-tags" | ||
| DEST_REF="refs/remotes/dest-tags" | ||
| SOURCE_REMOTE="source" | ||
| DEST_REMOTE="dest" | ||
| REMOTE_REF_PREFIX="${TAG_REF}" | ||
|
|
||
| # --- Helpers ----------------------------------------------------------------------------------- | ||
|
|
||
| # Fetch all tags from both remotes into local tracking refs. --no-tags suppresses git's automatic | ||
| # tag-following, which would otherwise populate refs/tags/* locally as a side effect. That side | ||
| # effect can cause the destination fetch to short-circuit (all objects already locally reachable via | ||
| # refs/tags/*), leaving refs/remotes/dest-tags/ empty and making every tag appear as new. | ||
| fetch_tags() { | ||
| echo "Fetching tags from source." | ||
| git fetch --no-tags --quiet "${SOURCE_REMOTE}" "+${TAG_REF}/*:${SOURCE_REF}/*" | ||
|
|
||
| echo "Fetching tags from destination." | ||
| git fetch --no-tags --quiet "${DEST_REMOTE}" "+${TAG_REF}/*:${DEST_REF}/*" | ||
| } | ||
|
|
||
| # Synchronize a single already-fetched tag onto the destination. | ||
| sync_tag() { | ||
| local tag="$1" | ||
|
|
||
| # Create tags that do not yet exist on the destination. | ||
| if ! git show-ref --verify --quiet "${DEST_REF}/${tag}"; then | ||
| echo "Creating: ${tag}" | ||
| push_ref "${tag}" | ||
| return | ||
| fi | ||
|
|
||
| local src_sha dest_sha | ||
| src_sha="$(git rev-parse "${SOURCE_REF}/${tag}")" | ||
| dest_sha="$(git rev-parse "${DEST_REF}/${tag}")" | ||
|
|
||
| # Tags that already match need no action. | ||
| if [ "${src_sha}" = "${dest_sha}" ]; then | ||
| echo "Up-to-date: ${tag}" | ||
| return | ||
| fi | ||
|
|
||
| # Overwrite diverged tags when force is enabled. An explicit lease value is required: tags are | ||
| # fetched into the custom ${DEST_REF} namespace rather than a standard remote-tracking ref, so | ||
| # a bare --force-with-lease has no remote-tracking ref to compare against and always rejects | ||
| # the push as stale, even when the destination has not changed since fetch_tags ran. | ||
| if [ "${FORCE}" = "true" ]; then | ||
| echo "Overwriting: ${tag}" | ||
| push_ref "${tag}" "--force-with-lease=${TAG_REF}/${tag}:${dest_sha}" | ||
| return | ||
| fi | ||
|
|
||
| die "Tag '${tag}' exists on both sides with different objects; use FORCE to overwrite it." | ||
| } | ||
|
|
||
| sync_tags() { | ||
| PUSH_OPTS=() | ||
| if [ "${DRY_RUN}" = "true" ]; then | ||
| echo "Dry run: no tags will be pushed." | ||
| PUSH_OPTS+=("--dry-run") | ||
| fi | ||
|
|
||
| local tag | ||
| while IFS= read -r tag; do | ||
| sync_tag "${tag}" | ||
| done < <(git for-each-ref --format='%(refname:lstrip=3)' "${SOURCE_REF}/") | ||
| } | ||
|
|
||
| # --- Main -------------------------------------------------------------------------------------- | ||
|
|
||
| main() { | ||
| validate_common_config | ||
| setup_working_repo | ||
| fetch_tags | ||
| sync_tags | ||
| } | ||
|
|
||
| main "$@" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.