test(coverage-registry): freshness テストの予算を実作業量に合わせ 30秒張り付きの偽赤を解消 (#2397) - #2658
Merged
Conversation
Each of these copies three shipped subtrees and spawns the generator twice, which costs tens of seconds -- 26.1s and 30.5s measured on CI against the suite's 30s default. A budget with no headroom is a coin flip, and it came up tails on two unrelated PRs: #2618 at 30514ms and #2646 at 30006ms, six milliseconds over, both green on rerun with no code change. The work is not pathological; the ceiling was set below it. Give the three heavy tests one sized to what they actually do. Refs #2397
📝 WalkthroughWalkthrough鮮度差分関連の高負荷な ChangesCoverage registry テスト
Estimated code review effort: 1 (Trivial) | ~5分 Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
症状
tests/unit/gen-coverage-registry.test.tsの freshness テストが、無関係な PR の CI で落ちる。amadeus-state.tsの--user-inputtrimamadeus-lib.tsの presence 列挙両方とも再実行でコード変更なしに green。同一 run の兄弟テストも同じ帯にいる:
wall-clock drift レポートも同ファイルを恒常的に large と測っている:
機序
--check freshness diffの各テストはcpSyncで temp へコピー(buildTempTree)genInto= ベースライン生成、checkAgainst= 検査)を行う。これは数十秒かかる実作業で、ハングでも無限ループでもない。CI ランナーはローカルより数倍遅く(本 repo のローカル実測ではファイル全体 34 tests で 18.3 秒、CI では当該2件だけで 26s / 30.5s)、スイート既定の 30 秒はこの作業量に対して余裕がゼロだった。
つまり制限が実作業を下回っていただけで、負荷のわずかな揺れが赤に直結する構造になっていた。
修正
重い3テストに、実作業量に見合う予算を明示的に与える。
3件それぞれの
test(...)第3引数として渡し、理由(実測値と2件の実害)をコメントに残した。スイート既定(30秒)は他のテストに対して変えていない。落ちる実証
予算が実際に効いていることを確認するため、
FRESHNESS_DIFF_TIMEOUT_MSを一時的に200へ絞って実行:復元後:
34 pass / 0 fail。注入 → 赤 → 復元 → 残渣ゼロ(git status --porcelainが対象1ファイルのみ)を1セットで実施。検証(各コマンド単独実行、exit code を個別に取得)
bun test tests/unit/gen-coverage-registry.test.ts --timeout 120000→ 0(34 pass / 0 fail、18.26s)bun run typecheck→ 0この修正が #2397 に対して何を閉じるか
#2397 が追う「coverage job 限定で毎回別のテストが落ちる」回転のうち、本セッションで2回再現した1本を機序特定のうえ塞ぐ。回転の他のメンバー(t224 / t427 / t99)については PR #2593 が fixture コピー経路のリトライ・件数検証・診断を入れており、そちらは別機序。
Refs #2397
Note
Low Risk
Test-only timeout and comment changes; no production or coverage tool behavior changes.
Overview
Stops flakey CI failures on the coverage registry freshness ratchet tests by giving the three heaviest cases an explicit 120s per-test timeout instead of the suite default 30s.
Those tests copy three shipped subtrees into a temp tree and spawn
gen-coverage-registrytwice each, which routinely runs 26–30+ seconds on CI with no headroom under 30s (e.g. unrelated PRs timing out by a few ms). A comment documents the measured CI times and links #2397. The lighter “missing committed registry” case is unchanged; other tests in the file still use the default timeout.Reviewed by Cursor Bugbot for commit 69e4338. Configure here.
Summary by CodeRabbit