Add non-blocking Loom tests to CI (#55) - #154
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdd a ChangesLoom CI integration
Possibly related issues
Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 19 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (19 passed)
📋 Issue PlannerBuilt with CodeRabbit's Coding Plans for faster development and fewer bugs. View plan used: ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci.yml:
- Around line 52-56: The Loom Concurrency Tests step is currently gated only on
matrix.privilege, so it gets skipped after earlier failures instead of remaining
a non-blocking advisory signal. Update the workflow condition on the Loom
Concurrency Tests (non-blocking) step to use always() together with the existing
unprivileged check, so the step still runs even when prior steps fail.
In `@docs/developers-guide.md`:
- Around line 48-52: Update the scheduler-budget note in the developers guide to
describe all three Loom bounds together, not just preemption_bound. In the
section referencing src/env/loom_tests.rs, revise the wording so max_threads,
max_branches, and preemption_bound are presented as jointly constraining the
search space, while still noting that changing any of them may require
justification and CI timeout adjustments.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: f3db5f07-5fe4-4f3f-bd52-0d9024476357
📒 Files selected for processing (4)
.github/workflows/ci.ymlMakefiledocs/developers-guide.mdsrc/env/loom_tests.rs
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
leynos/cuprum(auto-detected)
|
@coderabbitai resume |
✅ Action performedReviews resumed. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/test_workflow_integration.py`:
- Line 20: The docstring for run_act is out of sync with its return contract and
omits the artefact path. Update the run_act docstring so it explicitly states
that the function returns an exit code, an artefact path, and logs, keeping the
wording aligned with the implementation and the test_workflow_integration
helper.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: d2ce7a44-cdbc-4370-8287-b5ddd913a3a5
📒 Files selected for processing (1)
tests/test_workflow_integration.py
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
leynos/cuprum(auto-detected)leynos/rstest-bdd(auto-detected)
|
@coderabbitai resume |
✅ Action performedReviews resumed. |
Add `make test-loom` as the canonical entry point for the ignored Loom suite. Document the current scheduler budget and the reason the preemption bound should not be raised without justification. Keep the target out of `all` so Loom remains opt-in locally until CI coverage is ready to become blocking.
Add the existing Loom suite to the unprivileged CI path after coverage. Keep the step informational with `continue-on-error` and cap it at one minute so expanded model scenarios fail fast instead of consuming a CI run indefinitely.
Add Rustdoc comments to the Loom-specific helper symbols and model tests. This keeps the changed test module understandable and gives docstring coverage tools real documentation to count on the branch delta.
Keep the advisory Loom CI step running after earlier unprivileged failures by combining `always()` with the existing matrix guard. Clarify that `max_threads`, `max_branches`, and `preemption_bound` jointly constrain the Loom search space, and that changing any of these bounds may need timeout adjustments.
Add function docstrings to the Python workflow integration test module so external docstring coverage checks can measure the file correctly.
Clarify that `run_act` returns the exit code, artefact path, and logs so the function documentation matches the helper contract.
Add `interrogate --fail-under 100 .` to `make lint` so Python docstring coverage is enforced alongside rustdoc and Clippy. Install `uv` and then install `interrogate` as a uv tool in CI before the lint step so the Makefile target has the expected executable available.
Update the workflow integration helper docstring so it names the returned artefact directory alongside the exit code and captured logs.
49dbdad to
49b7ccb
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci.yml:
- Around line 35-39: The CI setup step for installing interrogate currently
pulls latest releases via uv, which makes the lint job non-deterministic. Update
the workflow step that runs in the ci.yml install block to pin both the uv tool
installation and the interrogate package version, using the existing install
command sequence around the uv tool install invocation so the lint lane stays
reproducible.
In `@docs/developers-guide.md`:
- Around line 39-42: The Loom test instructions in developers-guide are pointing
contributors to the wrong runner for the non-default-features lane. Update the
guidance around the Loom suite to match make test by replacing the follow-up
cargo test command with cargo nextest run, and keep the wording aligned with the
existing loom-tests / dev-worker flow so the documented local repro path matches
the actual Makefile behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: bbebe1c2-99b4-4571-981c-766a278363ef
📒 Files selected for processing (5)
.github/workflows/ci.ymlMakefiledocs/developers-guide.mdsrc/env/loom_tests.rstests/test_workflow_integration.py
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
leynos/cuprum(auto-detected)leynos/rstest-bdd(auto-detected)
Pin the CI-installed uv and interrogate versions so the lint lane is reproducible instead of resolving latest releases on each run. Update the Loom testing guide to name cargo nextest for the non-default-features follow-up lane, matching the Makefile test target.
Add the local Memtrace database directory to gitignore so runtime daemon state does not appear as an outstanding repository change.
Summary
This branch adds the existing Loom concurrency suite to CI as an advisory,
time-capped check for issue #55. It adds a canonical
make test-loomentrypoint, documents the scheduler budget that keeps the Loom model tractable, and
runs the suite in the unprivileged CI path without blocking pull requests while
coverage is still being expanded.
Closes #55.
Review walkthrough
test-loomtarget and confirm it remains out ofall.Validation
mbake validate Makefile: passed.make check-fmt: passed.make markdownlint: passed.make nixie: passed.make lint: passed.make test: passed.make test-loom: passed.make help: showedtest-loom.actionlint .github/workflows/ci.yml: passed.coderabbit review --agent: passed with 0 findings after the Makefile/docs milestone.coderabbit review --agent: passed with 0 findings after the CI milestone.Notes
The second CodeRabbit review initially hit the free CLI rate limit. The agent
waited with
vsleepfor the requested randomized interval and retried thereview successfully.