Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
2b9f838
Add rendered workflow contracts for Python template
leynos May 31, 2026
6da5178
Add act switch to generated test target
leynos May 31, 2026
958ee77
Add parent template test Makefile
leynos May 31, 2026
1e8f361
Address workflow validation review comments
leynos May 31, 2026
527e70f
Document rendered test helpers
leynos Jun 1, 2026
13ff3b3
Address helper contract review findings
leynos Jun 1, 2026
12e52e5
Normalize developer guide spelling
leynos Jun 3, 2026
04e678f
Add direct helper error-path tests
leynos Jun 3, 2026
0afdfb8
Tighten helper unit test contracts
leynos Jun 3, 2026
c4e02b2
Cover parent Makefile and helper module roles
leynos Jun 3, 2026
e47324d
Anchor Makefile tests and strengthen snapshots
leynos Jun 3, 2026
38beb8b
Document snapshot quality guidance
leynos Jun 3, 2026
8e1cda7
Validate AGENTS make target command contracts
leynos Jun 3, 2026
ed0e2d3
Guard parent test target uvx lookup
leynos Jun 3, 2026
8577d0a
Document parent Makefile help target
leynos Jun 3, 2026
1cfa739
Prove Rust test target runs doctests
leynos Jun 3, 2026
d193267
Tighten doctest regression review fixes
leynos Jun 3, 2026
7be16d8
Add generated dependency audit gate
leynos Jun 3, 2026
dd96c24
Split act validation into separate workflows
leynos Jun 3, 2026
a22c033
Stop installing mdformat-all in parent CI
leynos Jun 4, 2026
c7e44d5
Always run Rust doctests in generated tests
leynos Jun 4, 2026
c22ba24
Harden parent workflow tool installs
leynos Jun 4, 2026
3e88bf1
Document parent Makefile and CI workflows
leynos Jun 4, 2026
ffbdbe2
Harden generated act validation workflow
leynos Jun 4, 2026
a280a9a
Document audit in README quality gate flow
leynos Jun 5, 2026
add4767
Clarify workflow contract assertion failures
leynos Jun 5, 2026
22de12b
Clear RUSTFLAGS during generated Rust tool installs
leynos Jun 5, 2026
e235e3a
Report act coverage failures after log checks
leynos Jun 5, 2026
67cf116
Document temporary lint suppression follow-ups
leynos Jun 5, 2026
60ebe6a
Update shared actions coverage pin
leynos Jun 5, 2026
e480d72
Require Node24-capable act for validation
leynos Jun 5, 2026
b44b9a3
Forward nested act GitHub token explicitly
leynos Jun 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.PHONY: help test

MAKEFLAGS += --no-print-directory

test: ## Run template tests
uvx --with pytest-copier --with pyyaml --with syrupy pytest tests/

help: ## Show available targets
@grep -E '^[a-zA-Z_-]+:.*?##' $(MAKEFILE_LIST) | \
awk 'BEGIN {FS=":.*?## "; printf "Available targets:\n"} {printf " %-15s %s\n", $$1, $$2}'
15 changes: 9 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,19 @@ The test suite relies on the `pytest-copier` plugin and renders generated
projects that run Ruff, Pylint via a PyPy-backed runner, `ty`, pytest, and, when
the Rust extension is enabled, Clippy, Whitaker, and nextest-aware Rust tests.

Install the `pytest-copier` test dependency before running this repository's
`pytest` suite. Generated projects install and run their own tooling, including
Ruff, Pylint via PyPy, `ty`, pytest, and, when Rust is enabled, Clippy, Whitaker,
and nextest.
Run the parent template tests through the repository `Makefile`. The `test`
target uses `uvx` to provide `pytest-copier`, `PyYAML`, and `syrupy` without a
manually managed virtual environment:

```bash
pip install pytest-copier
make test
```

You can also run `scripts/setup_test_deps.sh` to install them automatically.
Generated projects install and run their own tooling, including Ruff, Pylint via
PyPy, `ty`, pytest, and, when Rust is enabled, Clippy, Whitaker, and nextest.

You can also run `scripts/setup_test_deps.sh` to install parent test
dependencies into the current Python environment manually.

## Generated Quality Gate Flow

Expand Down
2 changes: 1 addition & 1 deletion scripts/setup_test_deps.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
# Generated projects install their own linting and type-checking dependencies.
set -euo pipefail

pip install pytest-copier syrupy
pip install pytest-copier PyYAML syrupy
1 change: 1 addition & 0 deletions template/.github/actions/build-wheels/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ runs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@v5
with:
Expand Down
1 change: 1 addition & 0 deletions template/.github/actions/pure-python-wheel/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ runs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@v5
with:
Expand Down
2 changes: 2 additions & 0 deletions template/.github/workflows/build-wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ jobs:
cibw_arch: arm64
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: ./.github/actions/build-wheels
with:
python-version: ${{ inputs['python-version'] }}
Expand Down
2 changes: 2 additions & 0 deletions template/.github/workflows/ci.yml.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ jobs:
steps:
- name: Check out repository
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
with:
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on:
- 'v*.*.*'

concurrency:
group: release-${{ github.ref }}
group: release-${{ "{{" }} github.ref {{ "}}" }}
cancel-in-progress: true

permissions:
Expand All @@ -15,28 +15,43 @@ permissions:

jobs:

{% if use_rust %}
build-wheels:
uses: ./.github/workflows/build-wheels.yml
with:
python-version: '3.13'
{% else %}
pure-wheel:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4
with:
fetch-depth: 0
persist-credentials: false
- uses: ./.github/actions/pure-python-wheel
with:
python-version: '3.13'
artifact-name: wheels-pure
{% endif %}

release:
{% if not use_rust %}
# This project has no C or Rust extensions, so cross-platform
# builds are unnecessary. Only the pure Python wheel is published.
{% endif %}
needs:
{% if use_rust %}
- build-wheels
{% else %}
- pure-wheel
{% endif %}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
persist-credentials: false
- uses: softprops/action-gh-release@v2
with:
generate_release_notes: true
Expand All @@ -47,6 +62,6 @@ jobs:
run: |
set -eu
find dist/wheels-* -type f -name "*.whl" -print0 | \
xargs -0 -r gh release upload "${{ github.ref_name }}"
xargs -0 -r gh release upload "${{ "{{" }} github.ref_name {{ "}}" }}" --clobber
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ "{{" }} secrets.GITHUB_TOKEN {{ "}}" }}
3 changes: 3 additions & 0 deletions template/AGENTS.md.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,9 @@
- Formatting is correct and validated.
- **For Python files:**
- **Testing:** Passes all relevant unit and behavioural tests (`make test`).
Use `make test WITH_ACT=1` when local GitHub Actions workflow validation
should run through `act`; this sets `RUN_ACT_VALIDATION=1` for the pytest
invocation.
- **Linting:** Passes lint checks (`make lint`).
- **Formatting:** Adheres to formatting standards (`make check-fmt`; use
`make fmt` to apply fixes).
Expand Down
4 changes: 3 additions & 1 deletion template/Makefile.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ USER_BIN_PATH := $(HOME)/.cargo/bin:$(HOME)/.local/bin:$(HOME)/.bun/bin
TOOLS = $(MDFORMAT_ALL) $(MDLINT)
VENV_TOOLS = pytest
UV_ENV = PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 UV_CACHE_DIR=.uv-cache UV_TOOL_DIR=.uv-tools
WITH_ACT ?= 0
ACT_TEST_ENV = $(if $(filter 1 true yes on,$(WITH_ACT)),RUN_ACT_VALIDATION=1,)
PYTEST_XDIST_WORKERS ?= auto
PYTHON_TARGETS ?= {{ package_name }} tests
PYLINT_PYTHON ?= pypy
Expand Down Expand Up @@ -156,7 +158,7 @@ nixie: ## Validate Mermaid diagrams
$(NIXIE) --no-sandbox

test: build $(VENV_TOOLS) ## Run tests
$(UV_ENV) $(UV) run pytest -v -n $(PYTEST_XDIST_WORKERS)
$(UV_ENV) $(ACT_TEST_ENV) $(UV) run pytest -v -n $(PYTEST_XDIST_WORKERS)
{% if use_rust %}
@test -n "$(CARGO_AVAILABLE)" || { \
printf "Error: cargo is required for Rust tests, but '%s' was not found on PATH\n" "$(CARGO)" >&2; \
Expand Down
25 changes: 25 additions & 0 deletions tests/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
"""Support utilities for the Copier template validation test suite.

The package exposes shared helpers and fixtures used by the repository tests,
including rendered-file parsers, generated-project command helpers, workflow
contract assertions, container runtime environment helpers, and pytest fixtures
loaded from :mod:`tests.conftest`.

Import the package directly when a test only needs package discovery, or import
specific utilities from their modules when using them in assertions. Importing
``tests`` has no side effects beyond normal package initialisation; runtime
probing for tools such as Docker, Podman, or ``act`` stays inside fixtures and
helper functions.

Examples
--------
Use helpers from the package in template tests::

import tests
from tests.helpers.generated_files import parse_yaml_mapping
from tests.utilities import docker_environment

assert tests.__doc__
workflow = parse_yaml_mapping("name: CI\n", "workflow")
env = docker_environment()
"""
1 change: 1 addition & 0 deletions tests/__snapshots__/test_template.ambr
Original file line number Diff line number Diff line change
Expand Up @@ -57,5 +57,6 @@
nixie Validate Mermaid diagrams
test Run tests
help Show available targets

'''
# ---
1 change: 1 addition & 0 deletions tests/helpers/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Shared helper modules for rendered Copier template tests."""
175 changes: 175 additions & 0 deletions tests/helpers/generated_files.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
"""Parse and validate generated project files in template tests."""

from __future__ import annotations

import tomllib
from pathlib import Path
from typing import Any

import pytest
import yaml


def read_generated_text(path: Path) -> str:
"""Read a generated file with assertion-focused error context.

Parameters
----------
path
Path to the generated file to read.

Returns
-------
str
UTF-8 decoded file contents.

Raises
------
pytest.fail.Exception
Raised when the file cannot be read.

Examples
--------
Read a rendered workflow before parsing it::

workflow = read_generated_text(project / ".github/workflows/ci.yml")
"""
try:
return path.read_text(encoding="utf-8")
except OSError as error:
pytest.fail(f"could not read generated file {path}: {error}")


def parse_toml_file(path: Path) -> dict[str, Any]:
"""Parse generated TOML with assertion-focused error context.

Parameters
----------
path
Path to the generated TOML file.

Returns
-------
dict[str, Any]
Parsed TOML mapping.

Raises
------
pytest.fail.Exception
Raised when the file cannot be read or the TOML is invalid.

Examples
--------
Parse generated project metadata::

pyproject = parse_toml_file(project / "pyproject.toml")
"""
text = read_generated_text(path)
try:
parsed = tomllib.loads(text)
except tomllib.TOMLDecodeError as error:
pytest.fail(f"could not parse generated TOML {path}: {error}")
return parsed


def parse_yaml_mapping(text: str, label: str) -> dict[str, Any]:
"""Parse generated YAML as a mapping with clear failure context.

Parameters
----------
text
YAML document text to parse.
label
Human-readable label used in assertion failure messages.

Returns
-------
dict[str, Any]
Parsed YAML mapping.

Raises
------
pytest.fail.Exception
Raised when the YAML is invalid or the parsed document is not a mapping.

Examples
--------
Parse a rendered CI workflow::

workflow = parse_yaml_mapping(ci_workflow, "CI workflow")
"""
try:
parsed = yaml.safe_load(text)
except yaml.YAMLError as error:
pytest.fail(f"could not parse generated {label}: {error}")
if not isinstance(parsed, dict):
pytest.fail(f"expected generated {label} to parse as a mapping")
return parsed


def require_mapping(mapping: dict[str, Any], key: str, label: str) -> dict[str, Any]:
"""Return a nested mapping or fail with the missing schema path.

Parameters
----------
mapping
Parent mapping to inspect.
key
Nested key expected to contain a mapping.
label
Human-readable schema path used in assertion failure messages.

Returns
-------
dict[str, Any]
Nested mapping value.

Raises
------
pytest.fail.Exception
Raised when the key is missing or the value is not a mapping.

Examples
--------
Extract workflow jobs after parsing YAML::

jobs = require_mapping(workflow, "jobs", "CI workflow")
"""
value = mapping.get(key)
if not isinstance(value, dict):
pytest.fail(f"expected {label} to include mapping key {key!r}")
return value


def require_sequence(mapping: dict[str, Any], key: str, label: str) -> list[Any]:
"""Return a nested sequence or fail with the missing schema path.

Parameters
----------
mapping
Parent mapping to inspect.
key
Nested key expected to contain a sequence.
label
Human-readable schema path used in assertion failure messages.

Returns
-------
list[Any]
Nested sequence value.

Raises
------
pytest.fail.Exception
Raised when the key is missing or the value is not a sequence.

Examples
--------
Extract workflow steps from a parsed job::

steps = require_sequence(job, "steps", "CI lint-test job")
"""
value = mapping.get(key)
if not isinstance(value, list):
pytest.fail(f"expected {label} to include sequence key {key!r}")
return value
Loading