-
Notifications
You must be signed in to change notification settings - Fork 10
feat: Add reusable workflow to sync branches and tags #158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
bthomee
wants to merge
10
commits into
main
Choose a base branch
from
bthomee/sync
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 2 commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
6937960
feat: Add reusable workflow to sync branches and tags
bthomee 06e22f0
Fix unbound variable error
bthomee ec579aa
Address review feedback
bthomee 89388bf
Address review feedback
bthomee 470b9bc
Get an fresh app token in each job so passing it around isn't needed …
bthomee e99b445
Merge branch 'main' into bthomee/sync
bthomee ee02f97
Address review feedback
bthomee 3d7dc73
Address review feedback
bthomee 272bf7c
Gate on XRPLF repo
bthomee 1f20bc1
Change logic to not run on fork PRs
bthomee File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,178 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # Synchronize important branches from a source repository into a destination repository. Missing | ||
| # branches are always created. By default branches are fast-forwarded only, while diverged branches | ||
| # are skipped and cause a non-zero exit. Force mode overwrites diverged branches with a force-push | ||
| # (using --force-with-lease). Dry-run mode reports what would change without pushing anything. | ||
|
mathbunnyru marked this conversation as resolved.
Outdated
|
||
| # | ||
| # Configuration is supplied via environment variables: | ||
| # | ||
| # Required: | ||
| # GH_TOKEN Token with read access to the source repository and write access to the | ||
| # destination. A public repository ignores the token as long as it is valid. When | ||
| # running this script locally, your personal access token must have at least the | ||
| # "repo", "read:org" (for branches in organization repos), and "workflow" scopes. | ||
|
bthomee marked this conversation as resolved.
|
||
| # SOURCE_REPO Source repository, as "owner/repo" (github.com is assumed). | ||
| # DEST_REPO Destination repository, as "owner/repo" (github.com is assumed). | ||
| # BRANCH A single branch name or glob to synchronize. A glob may match several branches, | ||
|
bthomee marked this conversation as resolved.
Outdated
|
||
| # all of which are synchronized. | ||
| # | ||
| # Optional: | ||
| # DRY_RUN "true" to report changes without pushing. | ||
| # (default: false) | ||
| # FORCE "true" to overwrite diverged destination branches. | ||
| # (default: false) | ||
| # WORKDIR Directory to initialize the working repo in. | ||
|
mathbunnyru marked this conversation as resolved.
Outdated
|
||
| # (default: a fresh mktemp dir) | ||
| # | ||
| # Exit status: | ||
| # 0 All branches synchronized successfully (or a clean dry run was performed). | ||
| # 1 At least one branch could not be synchronized (diverged without FORCE, or a push was | ||
| # rejected because the destination changed concurrently); other branches were still | ||
| # attempted. | ||
| # 2 Invalid configuration / usage error. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| # shellcheck source=sync-common-functions.sh | ||
| source "${BASH_SOURCE[0]%/*}/sync-common-functions.sh" | ||
|
|
||
| # --- Configuration ----------------------------------------------------------------------------- | ||
|
|
||
| set_common_defaults | ||
| BRANCH="${BRANCH:-}" | ||
|
|
||
| # Internal git ref/remote names used within the working repository. | ||
| BRANCH_REF="refs/heads" | ||
| SOURCE_REF="refs/remotes/source" | ||
| DEST_REF="refs/remotes/dest" | ||
| SOURCE_REMOTE="source" | ||
| DEST_REMOTE="dest" | ||
| REMOTE_REF_PREFIX="${BRANCH_REF}" | ||
|
|
||
| # --- Helpers ----------------------------------------------------------------------------------- | ||
|
|
||
| # Validate the configuration supplied via the environment, exiting with status 2 on any problem. | ||
| validate_config() { | ||
| validate_common_config | ||
| [ -n "${BRANCH}" ] || die "BRANCH must be a non-empty branch name or glob." 2 | ||
| } | ||
|
|
||
| # Fetch the branch (or all branches matching the glob) from both remotes into local tracking refs. | ||
| # The '+' prefix force-updates the tracking refs so they always reflect each remote. | ||
| fetch_branches() { | ||
| # Unlike the destination, a missing source branch is a real error as there is nothing to | ||
| # synchronize from. Both "no matching refs" and other failures are fatal, but they get distinct | ||
| # messages to aid diagnosis, and both exit 2 to match the documented usage/config status rather | ||
| # than git's bare exit 1. The 'git ls-remote --exit-code' command returns 2 only when no ref | ||
| # matches, so we capture the status with '|| ls_status=$?' to ensure a non-zero exit does not | ||
| # trip 'set -e' before it is inspected. Any other non-zero status than 2 is treated as a real | ||
| # failure. | ||
| echo "Fetching source branch: ${BRANCH}" | ||
| if ! git fetch --quiet "${SOURCE_REMOTE}" "+${BRANCH_REF}/${BRANCH}:${SOURCE_REF}/${BRANCH}"; then | ||
| local ls_status=0 | ||
| git ls-remote --exit-code --heads "${SOURCE_REMOTE}" "${BRANCH_REF}/${BRANCH}" >/dev/null 2>&1 || ls_status=$? | ||
| if [ "${ls_status}" -eq 2 ]; then | ||
| die "Source branch(es) '${BRANCH}' not found in '${SOURCE_REPO}'." 2 | ||
| fi | ||
| die "Failed to fetch source branch(es) '${BRANCH}' from '${SOURCE_REPO}'." 2 | ||
| fi | ||
|
mathbunnyru marked this conversation as resolved.
Outdated
|
||
|
|
||
| # A glob that matches no refs makes 'git fetch' succeed having fetched nothing (git does not | ||
| # treat an empty glob match as an error), so a missing or mistyped BRANCH would otherwise go | ||
| # undetected. | ||
| if [ -z "$(git for-each-ref "${SOURCE_REF}/")" ]; then | ||
| die "Source branch(es) '${BRANCH}' not found in '${SOURCE_REPO}'." 2 | ||
| fi | ||
|
|
||
| # A literal (non-glob) branch that does not yet exist on the destination makes 'git fetch' fail; | ||
| # that case is tolerated here and the branch is created below. Any other failure (e.g. a | ||
| # transient network or remote error) must not be masked, or the branch would be wrongly treated | ||
| # as missing. | ||
| echo "Fetching destination branch: ${BRANCH}" | ||
| if ! git fetch --quiet "${DEST_REMOTE}" "+${BRANCH_REF}/${BRANCH}:${DEST_REF}/${BRANCH}"; then | ||
| local ls_status=0 | ||
| git ls-remote --exit-code --heads "${DEST_REMOTE}" "${BRANCH_REF}/${BRANCH}" >/dev/null 2>&1 || ls_status=$? | ||
| if [ "${ls_status}" -ne 2 ]; then | ||
| die "Failed to fetch destination branch(es) '${BRANCH}' from '${DEST_REPO}'." 2 | ||
| fi | ||
| fi | ||
| } | ||
|
|
||
| # Synchronize a single already-fetched ref onto the destination. Returns 0 on success and 1 on a | ||
| # recoverable per-branch failure (so the caller can continue with other branches). | ||
| sync_ref() { | ||
| local ref="$1" | ||
|
|
||
| # Create branches that do not yet exist on the destination. The refspec is not forced, so if a | ||
| # racing push created the branch between our fetch and now, the destination is rejected rather | ||
| # than clobbered; this is recorded as a failure and the next branch is still attempted. | ||
| if ! git show-ref --verify --quiet "${DEST_REF}/${ref}"; then | ||
| echo "Creating: ${ref}" | ||
| if ! push_ref "${ref}"; then | ||
| echo "::error::Branch '${ref}' could not be created (it may have been created concurrently); skipping." | ||
| return 1 | ||
| fi | ||
| return 0 | ||
| fi | ||
|
|
||
| # Overwrite diverged branches when force is enabled. The --force-with-lease flag guards against | ||
| # clobbering destination commits that appeared after our fetch: if the destination advanced in | ||
| # the meantime the push is rejected; this is recorded as a failure and the next branch is still | ||
| # attempted. | ||
| if [ "${FORCE}" = "true" ]; then | ||
| echo "Overwriting: ${ref}" | ||
| if ! push_ref "${ref}" --force-with-lease; then | ||
| echo "::error::Branch '${ref}' could not be overwritten (the destination changed since fetch); skipping." | ||
| return 1 | ||
| fi | ||
| return 0 | ||
| fi | ||
|
|
||
| # Check out the destination branch and attempt a fast-forward to the source. | ||
| echo "Syncing: ${ref}" | ||
| git checkout --quiet -B "${ref}" "${DEST_REF}/${ref}" | ||
| if ! git merge --ff-only "${SOURCE_REF}/${ref}"; then | ||
| echo "::error::Branch '${ref}' has diverged and cannot be fast-forwarded; skipping." | ||
| return 1 | ||
| fi | ||
|
|
||
| if ! git push "${PUSH_OPTS[@]+"${PUSH_OPTS[@]}"}" "${DEST_REMOTE}" "HEAD:${BRANCH_REF}/${ref}"; then | ||
| echo "::error::Branch '${ref}' could not be pushed (the destination changed since fetch); skipping." | ||
| return 1 | ||
| fi | ||
| } | ||
|
|
||
| # Synchronize every ref that was fetched under SOURCE_REF (a glob may match several). Returns 1 if | ||
| # any branch failed to synchronize, 0 otherwise. | ||
| sync_branches() { | ||
| # A dry run adds --dry-run to every push so nothing is actually written. The refspecs are | ||
| # expanded as "${PUSH_OPTS[@]+"${PUSH_OPTS[@]}"}" to avoid the "unbound variable" error that an | ||
| # empty array triggers under 'set -u' on older versions of Bash. | ||
| PUSH_OPTS=() | ||
| if [ "${DRY_RUN}" = "true" ]; then | ||
| echo "Dry run: no branches will be pushed." | ||
| PUSH_OPTS+=("--dry-run") | ||
| fi | ||
|
|
||
| # Process substitution feeds the loop so it runs in the current shell and 'failed' persists. | ||
| local failed=0 | ||
| local ref | ||
| while IFS= read -r ref; do | ||
| [ -n "${ref}" ] || continue | ||
| sync_ref "${ref}" || failed=1 | ||
| done < <(git for-each-ref --format='%(refname:lstrip=3)' "${SOURCE_REF}/") | ||
|
|
||
| return "${failed}" | ||
| } | ||
|
|
||
| # --- Main -------------------------------------------------------------------------------------- | ||
|
|
||
| main() { | ||
| validate_config | ||
| setup_working_repo | ||
| fetch_branches | ||
| sync_branches | ||
| } | ||
|
|
||
| main "$@" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| #!/usr/bin/env bash | ||
|
mathbunnyru marked this conversation as resolved.
Outdated
|
||
| # | ||
| # Common helpers shared by sync-branches.sh and sync-tags.sh. Intended to be sourced, not executed | ||
| # directly. | ||
|
|
||
| # Emit an error and exit with the given status (default 2 for usage/config errors). | ||
| die() { | ||
| local status="${2:-2}" | ||
|
mathbunnyru marked this conversation as resolved.
Outdated
|
||
| echo "::error::$1" >&2 | ||
| exit "${status}" | ||
| } | ||
|
|
||
| # Apply the defaults shared by all sync scripts. These match what is set in the workflow file, so | ||
| # the scripts can also be run locally. Keep in sync with the workflow file. | ||
| set_common_defaults() { | ||
| DRY_RUN="${DRY_RUN:-false}" | ||
| FORCE="${FORCE:-false}" | ||
| } | ||
|
|
||
| # Push the given source ref to the destination. Reads the SOURCE_REF and REMOTE_REF_PREFIX globals | ||
| # (the caller sets REMOTE_REF_PREFIX to "refs/heads" or "refs/tags") and the PUSH_OPTS global (e.g. | ||
| # --dry-run). Any extra arguments (e.g. --force-with-lease) are inserted before the refspec. | ||
| # Args: <ref> [git-push-option...] | ||
| push_ref() { | ||
| local ref="$1" | ||
| shift | ||
| git push "$@" "${PUSH_OPTS[@]+"${PUSH_OPTS[@]}"}" "${DEST_REMOTE}" \ | ||
| "${SOURCE_REF}/${ref}:${REMOTE_REF_PREFIX}/${ref}" | ||
| } | ||
|
|
||
| # Add a remote pointing at a github.com repository and authenticate to it with GH_TOKEN. The token | ||
| # is passed via an HTTP Authorization header rather than embedded in the remote URL. Embedding would | ||
| # persist the credential in .git/config and, because git prints the remote URL in its error | ||
| # messages, risk leaking it into CI logs. Passing it as an 'extraheader' keeps git's output limited | ||
| # to 'https://github.com/<repo>'. This mirrors how actions/checkout authenticates. The 'tr -d' | ||
| # command strips any line breaks base64 may insert so the header stays on one line. A public | ||
| # repository ignores a valid token, so the same header is used for both. | ||
| # Args: <remote-name> <owner/repo> | ||
| add_authenticated_remote() { | ||
| local remote="$1" repo="$2" | ||
| local url="https://github.com/${repo}" | ||
| git remote add "${remote}" "${url}" | ||
| git config --local "http.${url}/.extraheader" \ | ||
| "AUTHORIZATION: basic $(printf 'x-access-token:%s' "${GH_TOKEN}" | base64 | tr -d '\n')" | ||
| } | ||
|
|
||
| # Validate configuration shared by all sync scripts: GH_TOKEN, SOURCE_REPO, DEST_REPO, DRY_RUN, | ||
| # FORCE, and git presence. | ||
| validate_common_config() { | ||
| [ -n "${GH_TOKEN:-}" ] || die "GH_TOKEN is required but not set." 2 | ||
| [ -n "${SOURCE_REPO:-}" ] || die "SOURCE_REPO is required but not set." 2 | ||
| [ -n "${DEST_REPO:-}" ] || die "DEST_REPO is required but not set." 2 | ||
|
|
||
| case "${DRY_RUN}" in | ||
| true | false) ;; | ||
| *) die "DRY_RUN must be 'true' or 'false', got '${DRY_RUN}'." 2 ;; | ||
| esac | ||
|
|
||
| case "${FORCE}" in | ||
| true | false) ;; | ||
| *) die "FORCE must be 'true' or 'false', got '${FORCE}'." 2 ;; | ||
| esac | ||
|
|
||
| command -v git >/dev/null 2>&1 || die "git is required but not found on PATH." 2 | ||
| } | ||
|
|
||
| # Initialize the working repository and wire up the authenticated remotes. Reachability is not | ||
| # checked here, because if the first real fetch against each remote (in fetch_branches/fetch_tags) | ||
| # fails it will provide a clear message if the repository is unreachable or the token is invalid, so | ||
| # checking it here would be redundant. | ||
| setup_working_repo() { | ||
| if [ -z "${WORKDIR:-}" ]; then | ||
| WORKDIR="$(mktemp -d)" | ||
| trap 'rm -rf "${WORKDIR}"' EXIT | ||
| fi | ||
| mkdir -p "${WORKDIR}" | ||
| cd "${WORKDIR}" || exit | ||
|
|
||
| git init --quiet | ||
| # Both remotes carry the token; a public source simply ignores it as long as the token is valid. | ||
| add_authenticated_remote "${SOURCE_REMOTE}" "${SOURCE_REPO}" | ||
| add_authenticated_remote "${DEST_REMOTE}" "${DEST_REPO}" | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.