Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
265 changes: 265 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,268 @@ jobs:

- name: Publish to PyPI (Trusted Publisher / OIDC)
uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0

# ---- Lambda Layer publishing ----------------------------------------
# See layer/MAINTAINER.md for one-time AWS setup. Three jobs:
# build-layer-zip: build the layer ZIP and attach to the GH Release.
# publish-layer: per-region matrix; assume OIDC role; publish layer
# version with public read; write SSM parameter.
# aggregate-arns: collect per-region ARNs into a JSON manifest +
# markdown table; upload manifest as a release asset
# and append the table to the release notes.
# All three are gated on `release_created == 'true'` from release-please.

build-layer-zip:
name: build Lambda Layer ZIP
needs: release-please
if: ${{ needs.release-please.outputs.release_created == 'true' }}
runs-on: ubuntu-24.04
permissions:
contents: write # for `gh release upload`
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false

- name: Install uv with Python 3.12
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
python-version: '3.12'
enable-cache: true

- name: Build wheel
run: uv build --wheel

- name: Repackage wheel as Lambda Layer ZIP
env:
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
# Strip leading 'v' from the tag (v1.2.0 -> 1.2.0).
version="${TAG#v}"
mkdir -p build/python
unzip -q dist/cfn_handler-*.whl -d build/python
# Lambda doesn't need pip's bookkeeping.
rm -rf build/python/*.dist-info
(cd build && zip -qr "../dist/cfn_handler-${version}-layer.zip" python/)
ls -la dist/

- name: Upload Layer ZIP to GitHub Release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh release upload "${{ needs.release-please.outputs.tag_name }}" dist/cfn_handler-*-layer.zip \
--repo "${{ github.repository }}" \
--clobber

set-layer-matrix:
name: build Layer publish matrix
needs: release-please
if: ${{ needs.release-please.outputs.release_created == 'true' }}
runs-on: ubuntu-24.04
outputs:
regions: ${{ steps.regions.outputs.regions }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false

- name: Read layer/regions.txt into a JSON array
id: regions
run: |
set -euo pipefail
# Strip blank lines and # comments; jq -R -s -c assembles the
# array. Output as `regions=[...]` for the matrix to consume.
regions=$(grep -vE '^\s*(#|$)' layer/regions.txt | jq -R -s -c 'split("\n") | map(select(length > 0))')
echo "regions=${regions}" >> "$GITHUB_OUTPUT"
echo "Matrix regions: ${regions}"

publish-layer:
name: publish Layer (${{ matrix.region }})
needs: [release-please, build-layer-zip, set-layer-matrix]
if: ${{ needs.release-please.outputs.release_created == 'true' }}
runs-on: ubuntu-24.04
environment: layer-publisher
permissions:
id-token: write # OIDC token to assume the publisher role
contents: read
strategy:
fail-fast: false
max-parallel: 10 # Be polite to AWS APIs; matrix has ~17 entries today.
matrix:
region: ${{ fromJSON(needs.set-layer-matrix.outputs.regions) }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false

- name: Download Layer ZIP from GitHub Release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
gh release download "$TAG" \
--repo "${{ github.repository }}" \
--pattern 'cfn_handler-*-layer.zip' \
--dir dist/
ls -la dist/

- name: Configure AWS credentials via OIDC
uses: aws-actions/configure-aws-credentials@00943011d9042930efac3dcd3a170e4273319bc8 # v5.1.0
with:
role-to-assume: ${{ secrets.LAYER_PUBLISHER_ROLE_ARN }}
aws-region: ${{ matrix.region }}
role-session-name: cfn-handler-layer-publisher-${{ matrix.region }}
role-duration-seconds: 1200

- name: Publish Layer + grant public read + write SSM parameters
env:
REGION: ${{ matrix.region }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
version="${TAG#v}"
# Resolve the layer zip via globbing (find handles non-alphanumerics
# better than `ls | head`).
zip_file=$(find dist -maxdepth 1 -name 'cfn_handler-*-layer.zip' -print -quit)
if [[ -z "$zip_file" ]]; then
echo "ERROR: no layer zip found in dist/" >&2
exit 1
fi

echo "Publishing layer in ${REGION} from ${zip_file}..."
arn=$(aws lambda publish-layer-version \
--region "$REGION" \
--layer-name cfn-handler \
--description "cfn-handler ${version}" \
--license-info "Apache-2.0" \
--zip-file "fileb://${zip_file}" \
--compatible-runtimes python3.10 python3.11 python3.12 python3.13 python3.14 \
--compatible-architectures x86_64 arm64 \
--query LayerVersionArn --output text)
echo "Published: $arn"
version_number=$(echo "$arn" | awk -F: '{print $NF}')

echo "Granting public read on version ${version_number}..."
# add-layer-version-permission fails if the statement-id already
# exists. Treat that as success (idempotent re-runs).
aws lambda add-layer-version-permission \
--region "$REGION" \
--layer-name cfn-handler \
--version-number "$version_number" \
--statement-id PublicRead \
--action lambda:GetLayerVersion \
--principal '*' \
>/dev/null 2>&1 || echo " (PublicRead already granted; idempotent)"
Comment thread
igorlg marked this conversation as resolved.
Outdated

echo "Writing SSM parameters in ${REGION}..."
aws ssm put-parameter \
--region "$REGION" \
--name "/cfn-handler/${REGION}/layer-arn/latest" \
--type String \
--value "$arn" \
--overwrite \
>/dev/null
# Version-specific param: don't overwrite (each version's ARN is
# immutable; if this re-runs with a different ARN, fail loudly).
aws ssm put-parameter \
--region "$REGION" \
--name "/cfn-handler/${REGION}/layer-arn/${TAG}" \
--type String \
--value "$arn" \
>/dev/null 2>&1 || echo " (version param already exists; idempotent)"
Comment thread
igorlg marked this conversation as resolved.
Outdated

# Emit per-region artifact for the aggregate-arns job.
mkdir -p artifacts
jq -nc --arg region "$REGION" --arg arn "$arn" \
'{region: $region, arn: $arn}' > "artifacts/arn-${REGION}.json"

- name: Upload per-region ARN artifact
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
with:
name: layer-arn-${{ matrix.region }}
path: artifacts/arn-${{ matrix.region }}.json
if-no-files-found: error
retention-days: 7

aggregate-arns:
name: aggregate Layer ARNs into release notes + manifest
needs: [release-please, publish-layer]
if: ${{ always() && needs.release-please.outputs.release_created == 'true' }}
runs-on: ubuntu-24.04
permissions:
contents: write # gh release edit + gh release upload
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false

- name: Download all per-region ARN artifacts
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
with:
path: artifacts/
pattern: layer-arn-*
merge-multiple: false

- name: Build layer-arns.json + ARN markdown table
env:
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
version="${TAG#v}"

# Each per-region artifact is a directory containing arn-<region>.json.
# Collect into an array, sort by region, build the manifest.
regions_json=$(find artifacts -type f -name 'arn-*.json' \
-exec cat {} + \
| jq -s 'sort_by(.region) | map({(.region): .arn}) | add // {}')
jq -n --arg version "$version" --argjson regions "$regions_json" '{
version: $version,
layer_name: "cfn-handler",
regions: $regions
}' > layer-arns.json
cat layer-arns.json

# Markdown table of region -> ARN, sorted alphabetically by region.
{
printf '| Region | ARN |\n'
printf '|---|---|\n'
jq -r '.regions | to_entries | sort_by(.key)[] | "| `\(.key)` | `\(.value)` |"' layer-arns.json
} > arns-table.md
cat arns-table.md

- name: Upload layer-arns.json to GitHub Release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
gh release upload "$TAG" layer-arns.json \
--repo "${{ github.repository }}" \
--clobber

- name: Append ARN table to GitHub Release notes
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
# Read existing release notes; append our section if not already
# present (so re-runs are idempotent).
gh release view "$TAG" --repo "${{ github.repository }}" --json body --jq .body > existing-notes.md
if ! grep -q '^## Lambda Layer ARNs' existing-notes.md; then
{
cat existing-notes.md
printf '\n\n## Lambda Layer ARNs\n\n'
cat arns-table.md
printf '\n\n_Layer ZIP also attached as a release asset; programmatic manifest at_ `layer-arns.json`. _See `layer/README.md` for usage._\n'
} > new-notes.md
gh release edit "$TAG" --repo "${{ github.repository }}" --notes-file new-notes.md
echo "Release notes updated with ARN table."
else
echo "Release notes already contain ARN table; skipping append."
fi
30 changes: 30 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
[![CI](https://github.com/igorlg/cfn-handler/actions/workflows/ci.yml/badge.svg)](https://github.com/igorlg/cfn-handler/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/cfn-handler.svg)](https://pypi.org/project/cfn-handler/)
[![Python Versions](https://img.shields.io/pypi/pyversions/cfn-handler.svg)](https://pypi.org/project/cfn-handler/)
[![Lambda Layer](https://img.shields.io/github/v/release/igorlg/cfn-handler?label=lambda%20layer&color=ff9900&logo=amazonaws)](https://github.com/igorlg/cfn-handler/releases/latest)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)

A modern, well-engineered Python library for writing AWS CloudFormation
Expand Down Expand Up @@ -71,6 +72,35 @@ uv add cfn-handler
Polling support uses `boto3` lazily; `boto3` ships preinstalled in the AWS
Lambda Python runtimes, so no extra install is needed there.

### Or use the AWS Lambda Layer

Every release publishes a public Lambda Layer in ~17 commercial regions. To
use it, reference the ARN in your function definition — no `pip install`
during deploy, no vendoring into your function package:

```yaml
# SAM
Resources:
MyFunction:
Type: AWS::Serverless::Function
Properties:
Runtime: python3.12
Layers:
- arn:aws:lambda:us-east-1:<account-id>:layer:cfn-handler:N
```

Find the right ARN for your region in the [latest release notes][latest-release]
(per-region table) or the JSON manifest:

```sh
curl -fsSL https://github.com/igorlg/cfn-handler/releases/latest/download/layer-arns.json
```

See [`layer/README.md`](layer/README.md) for SAM/CDK snippets, the alternative
deploy-it-yourself path, and the maintainer's account ID.

[latest-release]: https://github.com/igorlg/cfn-handler/releases/latest

## Comparison to crhelper

If you're coming from [`crhelper`][upstream], the model will feel familiar:
Expand Down
41 changes: 38 additions & 3 deletions docs/CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Six workflows under `.github/workflows/`:
- **`codeql.yml`** — Python security-and-quality scan.
- **`dependency-review.yml`** — license + vulnerability gate on PRs.
- **`examples-lint.yml`** — cfn-lint over example SAM templates (PR-only, path-filtered).
- **`release.yml`** — release-please + PyPI Trusted Publishing OIDC.
- **`release.yml`** — release-please + PyPI Trusted Publishing OIDC; also drives the Lambda Layer publish across ~17 commercial regions via OIDC into a maintainer-owned AWS account.
- **`secure-workflows.yml`** — enforces commit-SHA pinning of every action.

Plus a Dependabot config (`.github/dependabot.yml`) and an `act` runner
Expand All @@ -34,7 +34,7 @@ with no API tokens stored anywhere.
| `codeql.yml` | `pull_request: main`, `push: main`, weekly cron | `analyze (python)` | yes |
| `dependency-review.yml` | `pull_request: main` | `review dependencies` | yes |
| `examples-lint.yml` | `pull_request: main` (paths: `examples/**`) | `cfn-lint over examples` | no (informational) |
| `release.yml` | `push: main`, `workflow_dispatch` | `release-please bot`, `build + attach release artifacts`, `publish to PyPI` | n/a (post-merge) |
| `release.yml` | `push: main`, `workflow_dispatch` | `release-please bot`, `build + attach release artifacts`, `publish to PyPI`, `build Lambda Layer ZIP`, `build Layer publish matrix`, `publish Layer (matrix over regions)`, `aggregate Layer ARNs into release notes + manifest` | n/a (post-merge) |
| `secure-workflows.yml` | `pull_request: main` | `ensure SHA-pinned actions` | yes |

Branch protection on `main` requires the four "yes" checks above (the
Expand Down Expand Up @@ -198,7 +198,7 @@ feat: / fix: / feat!: commit
creates git tag + GH Release
┌───────────────────┴────────────────────┐
┌───────────────────┴────────────────────┐
▼ ▼
publish-artifacts publish-pypi
(uv build → gh release upload) (uv build → pypa/gh-action-pypi-publish)
Expand All @@ -211,6 +211,41 @@ feat: / fix: / feat!: commit
wheel + sdist on PyPI
```

In parallel with the PyPI publish, the Lambda Layer publishing surface
runs (see [`layer/MAINTAINER.md`](../layer/MAINTAINER.md)):

```text
release-please ──► build-layer-zip ──► publish-layer (matrix over ~17 regions)
(wheel → ZIP) (assume OIDC role; aws lambda
publish-layer-version; public read;
SSM param; per-region artifact)
aggregate-arns
(download per-region artifacts;
build layer-arns.json; upload as
release asset; append ARN table to
GitHub Release notes)
```

The layer pipeline is gated on the `layer-publisher` GitHub environment
which holds `LAYER_PUBLISHER_ROLE_ARN`. Per-region failures are isolated
(`fail-fast: false`); the rest of the release succeeds even if a few
regions fail. The `aggregate-arns` job runs `if: always()` so a partial
publish still produces an inventory of the regions that did succeed.

Three public ARN-discovery surfaces (no AWS credentials required):

1. **GitHub Release body** — the `aggregate-arns` job appends a per-region
ARN markdown table to the release notes.
2. **`layer-arns.json` release asset** — structured manifest at
`https://github.com/igorlg/cfn-handler/releases/latest/download/layer-arns.json`.
3. **README badge** — current Layer version inline with the existing
PyPI / Python / License badges (uses GitHub-native release endpoint).

See [`layer/README.md`](../layer/README.md) for consumer usage (SAM/CDK
snippets, ARN format, alternative deploy in your own account).

### Conventional Commits → version bump

| Commit prefix | Version bump |
Expand Down
Loading