Cut the mutation testing timeout from 60s to 20s - #6
Merged
Conversation
Twenty mutants in the parser turn a byte scanner's `pos += 1` into `-=` or `*=`, which loops forever. That is expected and they are reported as timeouts, but at a minute each they cost twenty minutes of a run that should take twelve, and they do not spread evenly: the run on 2026-08-22 had five land consecutively in shard 2 and the job died at exit 143 with the report never uploaded. The unit tests finish in half a second, so twenty seconds is still more than thirty times the headroom they need, and it brings the whole set of timeouts under seven minutes.
The pull request that tuned the mutation timeouts touched only .cargo/mutants.toml and arrived with no label, because nothing in the config matched it. That is the failure this file already warns about a few lines down, in the comment explaining why the ci rule is .github/** rather than a list of the files in it. cargo-mutants' scope and timeouts are test configuration, so .cargo/** joins the tests label.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Came out of validating PR #5: the mutation run on main failed, and it was not the actions bump.
Twenty mutants in the parser turn a byte scanner's
pos += 1into-=or*=, which loops forever. That is expected and cargo-mutants reports them as timeouts rather than survivors, but at the configured minute each they cost twenty minutes of a run that should take twelve, and they do not spread evenly across the shards. Five landed consecutively in shard 2, which died at exit 143 with its report never uploaded.The unit tests finish in half a second, so 20s is still more than thirty times the headroom they need, and it brings the whole set of timeouts under seven minutes.
Checked locally on two shards: the same mutants are caught, and the ones that loop still time out rather than being reported as survivors.