Regenerate discovered nested lockfiles during lading bump (regenerate-lockfiles)

[leynos]

Summary

This branch makes lading bump regenerate every git-tracked Cargo.lock in the repository, not just the workspace root plus manifests listed in bump.lockfile_manifests. Previously, lading publish validated all tracked lockfiles under --locked, so any nested fixture lockfile that bump had not been configured to refresh went stale and publish aborted with a manual repair message — leaving the user wondering why bump had not done the repair itself. The users' guide already promised discovery-based refresh; this branch makes the code keep that promise.

Execplan: docs/execplans/regenerate-lockfiles.md (implemented in full; status COMPLETE, including the Stage A prototype evidence, red/green test transcripts, an end-to-end acceptance run, and the retrospective).

The branch also carries three supporting changes:

• The publish pre-flight stale-lockfile message no longer blames lading bump; it now names the remaining causes (manifest edits made without bump, or bump runs with --no-rebuild-lockfiles).
• The make typecheck gate is restored and future-proofed: six pre-existing diagnostics under the drifting unpinned ty toolchain are fixed, and ty is now pinned (TY_VERSION ?= 0.0.56) in the Makefile with CI running the pinned version via make typecheck rather than installing ty separately.
• A formatting-only mdformat reflow of the developers' guide is committed separately to keep future make fmt runs clean.

Review walkthrough

• Start with lading/commands/bump_lockfiles.py for the new merge_discovered_manifests, which unions configured bump.lockfile_manifests entries (order preserved) with manifests implied by git-tracked lockfiles (sorted, de-duplicated by resolved path). Discovery reuses discover_tracked_lockfiles, the same helper the publish pre-flight uses; a non-git workspace degrades to the configured list with a warning.
• Then review lading/commands/bump.py — _process_lockfiles now feeds the merged tuple to both the dry-run and live regeneration paths, so both report the same set.
• The behaviour is specified in tests/bdd/features/cli.feature (scenario "Bump refreshes a discovered nested Cargo.lock file") with its given step in tests/bdd/steps/test_bump_steps.py, and unit-tested in tests/unit/test_bump_lockfiles.py and tests/unit/test_bump_lockfile_rebuild.py. The autouse fixture in tests/unit/conftest.py also stubs the merge so manifest-focused suites never shell out to git.
• The reworded pre-flight hint lives in lading/commands/publish_preflight.py with its regenerated snapshot in tests/unit/publish/__snapshots__/test_preflight_lockfile_validation.ambr.
• The typecheck restoration is in lading/commands/publish_index_check.py (guard cascades restructured as if/elif/else because ty 0.0.8 did not narrow bindings after NoReturn calls) and lading/commands/bump_toml.py; the pinning is in the Makefile and .github/workflows/ci.yml.
• Documentation updates: docs/users-guide.md (bump discovery paragraph, quoted pre-flight message, re-scoped lockfile_manifests key) and docs/developers-guide.md (module descriptions, TY_VERSION variable).

Validation

• make test: 683 passed, 62 snapshots passed.
• make lint: Ruff clean; interrogate 100%; Pylint 10.00/10.
• make check-fmt: 132 files already formatted.
• make typecheck (ty 0.0.56, pinned): all checks passed.
• make markdownlint and make nixie: clean.
• uv tool run mbake validate Makefile: valid syntax.
• coderabbit review --agent: run after each milestone (feature, docs, ty pinning); zero findings each time.
• End-to-end acceptance with real git and cargo against a prototype workspace mirroring the reported failure: bump discovered two tracked lockfiles, listed - fixtures/minimal/Cargo.lock (lockfile) in its output, refreshed the nested lockfile to the new version, and cargo metadata --locked subsequently exited 0.
• Red/green evidence: the new unit tests and BDD scenario were observed failing for the expected reasons before implementation (transcripts recorded in the execplan).

Notes

• Tests follow red-green-refactor; the red tests were committed together with the implementation so the suite stays green at every commit boundary.
• One edge surfaced by the live acceptance run: a nested non-member package that pins a versioned path dependency on a bumped crate (alpha = { path = ..., version = "0.1.0" }) now fails at bump time with cargo's version-selection error, because bump does not rewrite non-member manifests. Such a repository already failed at publish time, so the error merely surfaces earlier and more actionably; rewriting those requirements is recorded in the execplan as an open follow-up.
• bump.lockfile_manifests remains supported for lockfiles that git does not track; --no-rebuild-lockfiles remains the global escape hatch.
• Regeneration is still non-atomic across manifests (pre-existing behaviour, documented; see issue Handle workspace inconsistency when lockfile refresh fails after manifest rewrites in bump #84).

References

• Lody session

[sourcery-ai]

Sorry @leynos, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 036a1e94-fe19-4eab-b566-844e9a392551

📥 Commits

Reviewing files that changed from the base of the PR and between fd3e38c and 926e04f.

📒 Files selected for processing (9)

• .github/workflows/ci.yml
• Makefile
• docs/developers-guide.md
• docs/execplans/regenerate-lockfiles.md
• lading/commands/publish_plan.py
• lading/workspace/models.py
• tests/unit/publish/conftest.py
• tests/unit/test_bump_lockfile_regeneration_metrics.py
• tests/unit/test_lockfile.py

🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

• leynos/cmd-mox (auto-detected)
• leynos/cuprum (auto-detected)
• leynos/shared-actions (auto-detected)

---

Summary

• Update lading bump to discover Git-tracked nested Cargo.lock files and merge their manifests with configured bump.lockfile_manifests.
• Use the merged manifest set for dry-run reporting and live regeneration.
• Fall back to configured manifests with a warning in non-Git workspaces.
• Add validated, ordered, de-duplicated manifest resolution and resilient regeneration with aggregated repair errors.
• Refactor lockfile handling into focused modules while preserving the public façade and repository boundary.
• Add bounded regeneration metrics, injectable timing, and suppressed dry-run discovery telemetry.
• Add unit, property-based, BDD, snapshot, and acceptance coverage.
• Clarify user and developer documentation, stale-lockfile diagnostics, and non-atomic regeneration.
• Add the completed Regenerate discovered nested lockfiles execplan.
• Pin ty to 0.0.56 and Ruff to 0.15.21 consistently in the Makefile and CI.
• Restore type-checking support through uv tool run --from ty==... ty.

Related references

• Extend the cargo-and-git-backed lockfile repository design in docs/lading-design.md (issue #82).
• Add coverage for aggregated regeneration failures (issue #84).
• Retain path-validation coverage related to issue #93.

Walkthrough

lading bump now discovers tracked nested lockfiles, merges their manifests, validates targets, regenerates each lockfile, and aggregates failures. Tests and documentation cover the workflow. The PR also pins Ruff and ty, and refines publish validation diagnostics.

Changes

Nested lockfile regeneration

Layer / File(s)	Summary
Manifest discovery and path contracts lading/commands/bump_lockfile_manifests.py, lading/commands/bump_lockfile_paths.py, tests/unit/test_bump_lockfile_manifest_merge.py, tests/unit/test_bump_lockfile_path_resolution.py	Merge configured and tracked manifests, validate workspace boundaries, and derive ordered lockfile paths.
Cargo regeneration and façade wiring lading/commands/bump_lockfile_regeneration.py, lading/commands/bump_lockfiles.py, tests/unit/test_bump_lockfiles.py, tests/unit/test_bump_lockfile_rebuild.py, tests/unit/conftest.py, tests/unit/test_bump_lockfile_regeneration_metrics.py	Run Cargo updates for each merged manifest, aggregate failures, record metrics, and preserve the façade.
Discovery observability and integration coverage lading/commands/lockfile.py, tests/unit/test_lockfile.py, tests/bdd/features/cli.feature, tests/bdd/steps/test_bump_steps.py	Control discovery metrics and logs. Verify nested lockfile regeneration.
Lockfile workflow documentation docs/execplans/regenerate-lockfiles.md, docs/users-guide.md, docs/developers-guide.md, docs/lading-design.md, docs/contents.md, docs/repository-layout.md	Document discovery, regeneration, stale-lock validation, metrics, and execution plans.

Pinned typechecking toolchain

Layer / File(s)	Summary
Pinned typecheck execution Makefile, .github/workflows/ci.yml, docs/developers-guide.md	Pin Ruff to 0.15.21, pin ty to 0.0.56, and align the ty invocation.

Publish validation control flow

Layer / File(s)	Summary
Dependency placement control flow lading/commands/publish_index_check.py	Use explicit early-return branches for missing dependency versions.
Stale lockfile diagnostics lading/commands/publish_preflight.py, docs/users-guide.md, tests/unit/publish/__snapshots__/test_preflight_lockfile_validation.ambr	Document stale locks caused by direct manifest edits or --no-rebuild-lockfiles.

Documentation wording cleanup

Layer / File(s)	Summary
Docstring wording lading/commands/publish_plan.py, lading/workspace/models.py, tests/unit/publish/conftest.py	Clarify descriptive docstrings without changing behaviour.

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant LadingBump
  participant Git
  participant Cargo
  User->>LadingBump: run lading bump
  LadingBump->>Git: discover tracked Cargo.lock files
  Git-->>LadingBump: return lockfile paths
  LadingBump->>Cargo: update workspace per merged manifest
  Cargo-->>LadingBump: return success or failure
  LadingBump-->>User: list refreshed lockfiles or repair commands

Loading

Possibly related PRs

• leynos/lading#85 — Extends the same lockfile regeneration workflow.
• leynos/lading#129 — Tests the shared resolve_lockfile_paths logic.
• leynos/lading#177 — Shares the lockfile orchestration refactoring.

Suggested reviewers: codescene-access

Poem

Nested locks emerge from Git,
Cargo updates each target bit;
ty is pinned, the paths align,
Failed repairs report in line.
Bump and publish now speak clear.

---

Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

• Ignore

❌ Failed checks (1 error, 1 warning, 3 inconclusive)

Check name	Status	Explanation	Resolution
Unit Architecture	❌ Error	resolve_lockfile_paths is presented as a path query, but its adapter calls merge_discovered_manifests, which runs git ls-files; the protocol documents only path errors, not discovery failures.	Separate Git discovery from pure path projection, or expose and document the fallible discovery dependency and handle LockfileDiscoveryError and runner failures at the command boundary.
User-Facing Documentation	⚠️ Warning	The user guide clearly documents lockfile discovery, dry-run behaviour, configuration, failures, and stale-lock repairs, but no n+1 migration document signposts this new behaviour for the 0.2.0 pre...	Add a 0.3.0 migration document or migration section that explains tracked-lockfile discovery, dry-run output, configuration fallback, and the regeneration escape hatch.
Testing (Unit And Behavioural)	❓ Inconclusive	Evidence gathering is still in progress.	Inspect the full unit and behavioural test boundaries before deciding.
Concurrency And State	❓ Inconclusive	Placeholder only; evidence collection is still in progress.	Inspect the regeneration tests and shared metrics state before deciding.
Architectural Complexity And Maintainability	❓ Inconclusive	Evidence gathering is incomplete; no verdict submitted as a final assessment.	Inspect the parent implementation, dependency graph, and tests before deciding whether the new module split reduces complexity.

✅ Passed checks (15 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title describes the main lockfile regeneration change and includes the required (regenerate-lockfiles) ExecPlan reference.
Description check	✅ Passed	The description directly explains the lockfile discovery change, supporting updates, implementation scope, and validation evidence.
Docstring Coverage	✅ Passed	Docstring coverage is 85.53% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Testing (Overall)	✅ Passed	Mark as PASS: merge/path property tests, regeneration failure/metric tests, and an actual CLI BDD scenario with exact root and discovered Cargo calls guard the changed behaviour.
Developer Documentation	✅ Passed	The developer guide documents the new lockfile modules, repository boundary, discovery, dry-run behaviour, failure handling, tooling pins, and metrics; the design doc and completed ExecPlan also re...
Module-Level Documentation	✅ Passed	All 18 changed Python modules have module-level docstrings; new lockfile modules describe their purpose and component boundaries, and lockfile.py documents publish/bump relationships.
Testing (Property / Proof)	✅ Passed	Mark this check PASS: Hypothesis properties exercise merge ordering/deduplication, path safety, and all-manifest failure aggregation; no new formal proof assumption needs exhaustive proof.
Testing (Compile-Time / Ui)	✅ Passed	Pass this check: the repository has no Rust or TypeScript sources, so trybuild is not applicable; focused Syrupy snapshots and semantic assertions cover bump, stale-lock, and regeneration output.
Domain Architecture	✅ Passed	Bump and publish workflows depend on repository protocols, while Cargo/git runners, path handling, discovery, and regeneration stay behind injected adapters; CLI wiring supplies the runner.
Observability	✅ Passed	The lockfile change adds bounded regeneration and discovery metrics, elapsed-time metrics, and logs at discovery, per-manifest, aggregate-failure, and completion boundaries.
Security And Privacy	✅ Passed	No secrets or new credential handling found. Git and Cargo commands use argument sequences with shell=False; manifest paths are resolved and workspace-bound, and metrics use bounded labels.
Performance And Resource Use	✅ Passed	Approve: the change adds one Git query, linear filtering, one O(n log n) deterministic sort, and one intentional Cargo call per required manifest; no quadratic walk, retry loop, or async blocking p...
Rust Compiler Lint Integrity	✅ Passed	The PR changes no Rust files; the repository has no .rs files, Rust lint suppressions, or Rust clone calls, and Cargo.toml is unchanged.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch regenerate-lockfiles

---

_{Comment @coderabbitai help to get the list of available commands.}

[leynos]

@coderabbitai resume

[coderabbitai]

✅ Action performed

Reviews resumed.

[leynos]

@coderabbitai Have the following failed checks now been resolved?

If further work is required, please provide an AI agent prompt for the remaining work to be done to address these failures.

Do not treat warnings as optional or aspirational. Where a change is out of scope for this PR, propose a GitHub issue unless one exists already. (Treat documentation and validation coverage as in scope).

❌ Failed checks (4 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 76.19% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Developer Documentation	⚠️ Warning	docs/developers-guide.md still has a malformed lading.commands.lockfile .discover_tracked_lockfiles ref, and the execplan still leaves ty pinning and dry-run wording stale.	Fix the module path, update the execplan to match implemented live/dry-run behaviour, and remove or mark the ty pinning follow-up as complete.
Testing (Property / Proof)	⚠️ Warning	merge_discovered_manifests adds order/dedup invariants, but only example tests were added; repo guidance calls for Hypothesis on such invariants.	Add a Hypothesis test for merge_discovered_manifests covering arbitrary configured/discovered orders, resolved-path de-duplication, and the non-git fallback.

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lading/commands/bump_lockfile_regeneration.py`:
- Around line 183-189: Update _regeneration_failure_cause to use structural
pattern matching on error.__cause__ instead of the current isinstance chain,
preserving the existing CommandSpawnError precedence, ValueError mapping, and
cargo_exit fallback.

In `@tests/unit/test_bump_lockfile_regeneration_metrics.py`:
- Around line 107-125: The parametrized test
test_regenerate_lockfiles_records_failure_cause currently omits the runner_value
failure branch. Add a runner helper that raises ValueError and include it with
expected_cause set to runner_value, ensuring _regeneration_failure_cause() is
verified for this operational failure.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 31d3af26-7bb7-425d-a779-539c97e818d0

📥 Commits

Reviewing files that changed from the base of the PR and between 19b3479 and 4b0b785.

📒 Files selected for processing (9)

• docs/developers-guide.md
• lading/commands/bump_lockfile_manifests.py
• lading/commands/bump_lockfile_regeneration.py
• lading/commands/bump_lockfiles.py
• lading/commands/lockfile.py
• tests/unit/test_bump_lockfile_manifest_merge.py
• tests/unit/test_bump_lockfile_rebuild.py
• tests/unit/test_bump_lockfile_regeneration_metrics.py
• tests/unit/test_lockfile.py

🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

• leynos/cmd-mox (auto-detected)
• leynos/cuprum (auto-detected)
• leynos/shared-actions (auto-detected)

[leynos]

@coderabbitai resume

[coderabbitai]

✅ Action performed

Reviews resumed.

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/execplans/regenerate-lockfiles.md`:
- Around line 346-351: Update the documentation sentence describing
discover_tracked_lockfiles to say “In a non-Git directory, it logs”, preserving
the existing behavior description and applying the required en-GB-oxendict
grammar.

In `@tests/unit/test_bump_lockfile_regeneration_metrics.py`:
- Around line 78-91: Update every bare assert at
tests/unit/test_bump_lockfile_regeneration_metrics.py:78-91, 108-115, and
179-194, plus tests/unit/test_lockfile.py:663-665, to use assert …, "focused
diagnostic message". Add messages for the returned lockfiles, success/duration
checks, expected failure-cause counter, partial-success and Cargo-exit counters,
discovered paths, suppressed counter, and suppressed informational log; preserve
all existing semantic checks.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 9474ed13-45aa-442c-92d4-eebb6371b3d0

📥 Commits

Reviewing files that changed from the base of the PR and between 19b3479 and fd3e38c.

📒 Files selected for processing (10)

• docs/developers-guide.md
• docs/execplans/regenerate-lockfiles.md
• lading/commands/bump_lockfile_manifests.py
• lading/commands/bump_lockfile_regeneration.py
• lading/commands/bump_lockfiles.py
• lading/commands/lockfile.py
• tests/unit/test_bump_lockfile_manifest_merge.py
• tests/unit/test_bump_lockfile_rebuild.py
• tests/unit/test_bump_lockfile_regeneration_metrics.py
• tests/unit/test_lockfile.py

🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

• leynos/cmd-mox (auto-detected)
• leynos/cuprum (auto-detected)
• leynos/shared-actions (auto-detected)

[codescene-access]

No quality gates enabled for this code.

[buzzybee-df12]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[leynos]

@coderabbitai please create a new GitHub issue for the following:

User-Facing Documentation ⚠️ Warning The user guide clearly documents lockfile discovery, dry-run behaviour, configuration, failures, and stale-lock repairs, but no n+1 migration document signposts this new behaviour for the 0.2.0 pre... Add a 0.3.0 migration document or migration section that explains tracked-lockfile discovery, dry-run output, configuration fallback, and the regeneration escape hatch.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '=== Matching open issues ==='
gh issue list \
  --repo leynos/lading \
  --state open \
  --limit 100 \
  --search 'migration 0.3.0 lockfile discovery dry-run' \
  --json number,title,url,body \
  --jq '.[] | {number,title,url,body}' || true

echo '=== PR metadata ==='
gh pr view 160 \
  --repo leynos/lading \
  --json url,title \
  --jq '{url,title}'

echo '=== Requesting comment permalink ==='
gh api 'repos/leynos/lading/issues/160/comments?per_page=100' \
  --jq '.[] | select(.user.login == "leynos") | select(.body | contains("please create a new GitHub issue for the following")) | {url: .html_url, body: .body}' || true

Length of output: 1626

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

body=$(cat <<'EOF'
## Summary

Add a 0.3.0 migration document or a 0.3.0 migration section for the lockfile regeneration changes delivered by PR `#160`.

## Rationale

The user guide documents the behaviour in detail. Users upgrading to the next release need a migration entry that signposts the changed lockfile behaviour and the available recovery path.

## Affected areas

- Add the migration document or section in the repository's release or migration documentation.
- Link to the existing user-guide material when that avoids duplicated operational detail.

## Required content

Document all of the following:

1. `lading bump` discovers Git-tracked `Cargo.lock` files and includes their adjacent `Cargo.toml` manifests.
2. Dry-run discovers and reports the same lockfile set without modifying files.
3. In a non-Git workspace, discovery falls back to `bump.lockfile_manifests` and emits a warning.
4. `bump.lockfile_manifests` extends coverage for untracked lockfiles and nested lockfiles in non-Git workspaces.
5. `lading bump --no-rebuild-lockfiles` is the regeneration escape hatch. Explain that users must regenerate stale lockfiles before publishing when they use it or modify manifests directly.

## Acceptance criteria

- Add a 0.3.0 migration document or migration section.
- Explain tracked-lockfile discovery, dry-run output, configuration fallback, and the regeneration escape hatch.
- Keep the content consistent with `docs/users-guide.md`.
- Add navigation or an index link when the documentation structure requires one.
- Validate documentation formatting and links with the repository's documented validation targets.

## Backlinks

- Source PR: https://github.com/leynos/lading/pull/160
- Source comment: https://github.com/leynos/lading/pull/160#issuecomment-5150893389
- Requested by: `@leynos`
EOF
)

gh issue create \
  --repo leynos/lading \
  --title 'Document 0.3.0 migration for lockfile discovery and regeneration' \
  --assignee leynos \
  --body "$body"

Length of output: 2223

---

@leynos Created and assigned #219 — Document 0.3.0 migration for lockfile discovery and regeneration.

The issue includes the required migration content, acceptance criteria, and backlinks to PR #160 and your request.

_{You are interacting with an AI system.}