fix(ci): budget the root-rollup VK test to its measured cost, drop a redundant circuit copy

[AztecBot]

The flake

barretenberg/cpp/scripts/run_test.sh dsl_tests HonkRecursionConstraintTestWithoutPredicate/2.GenerateVKFromConstraints fails ~12% of the time on next (history). Over the last 75 tracked runs:

	n	runtimes
PASSED	66	102s … 486s (median 206s)
FAILED	9	600, 602, 603, 605, 606, 606, 608, 611, 612 — all code: 124

Every failure is timeout firing at the default TIMEOUT=600s, and the failure runtimes cluster in a 12-second band right at that deadline.

It is not a hang, and not a cleanup issue

• 8 of the 9 failures never reach [ OK ]. They are SIGTERM'd mid-computation, 10–90s short of finishing — e.g. b36f4b69785190ee dies 11s after populating trace... in the final block.
• The one that did reach [ OK ] (01dc57c4852bab07) reports (598699 ms) and is killed 1.3s later. It consumed 598.7s of its 600s budget; there is no post-PASSED stall, just no clock left for teardown.
• A fast run (da88666655b773c2, 112s) exits cleanly with no timeout line at all.
• The memory profile is identical between fast and slow runs (~19.8GiB peak in both), so nothing leaks or accumulates. Only wall-clock varies.

The ~5x spread comes from the runner: ci3/parallelize runs -j$((nproc / 2)) = 32 concurrent commands, and ci3/exec_test runs each under taskset -c 0-63 — the whole CPU set. CPUS=8 only sets HARDWARE_CONCURRENCY inside bb; it does not confine the test to 8 cores. Separately, GNU parallel's --memsuspend SIGSTOPs jobs under memory pressure while timeout keeps counting, and at ~19.8GiB this is the largest-footprint test in the suite.

Where the time actually goes

Profiled on an idle 192-core box with HARDWARE_CONCURRENCY=8 (matching CI), by timing each phase of test_vk_independence. Baseline total 182.5s:

Phase	ms	share
generate_constraints (inner circuits + proofs)	748	0.4%
A: create_circuit (with witness)	18,434	10.1%
A: get_num_finalized_gates_inefficient	5,010	2.7%
A: ProverInstance	8,159	4.5%
A: VerificationKey (28 MSMs over 8.4M points)	21,657	11.9%
A: CircuitChecker::check	85,142	46.7%
A: destructors	3,748	2.1%
B: create_circuit (no witness)	10,698	5.9%
B: ProverInstance	5,754	3.2%
B: VerificationKey	19,847	10.9%
B: destructors	2,446	1.3%

CircuitChecker::check breaks down further into: prepare_circuit (a second full copy of the 6.35M-gate builder + finalize) 9.1s, lookup hash-table build 15.3s, and the serial per-row relation loop 62.1s over 6.35M rows across 9 blocks (~10µs/row).

What this PR changes

1. TIMEOUT=20m, MEM=16g → 20g on the HonkRecursionConstraintTestWithoutPredicate/2.* prefix, matching what ChonkPinnedIvcInputsTest.AllPinnedFlows already does. 20m is 2x headroom over the worst observed contended runtime; the default 600s left none over a 486s observed maximum. MEM is documentation for non-ISOLATE tests (only docker_isolate enforces it), but 16g understated the real ~19.8GiB peak and it should be true.

2. Drop get_num_finalized_gates_inefficient() from test_vk_independence. That helper deep-copies the entire builder, finalizes the copy, counts gates, and throws the copy away — its doc comment says as much. Constructing the ProverInstance on the next line already finalizes the real builder, so get_num_finalized_gates() returns the identical value for free. Measured: 192.5s → 182.6s on the root rollup, and the transient copy (~2.7GiB) is gone. Every acir_format constraint test that calls test_vk_independence benefits.

The returned num_gates is discarded by all 14 call sites, so no behavior depends on it; finalize_circuit() takes no arguments, so both paths finalize identically.

Things measured and rejected

• batch_commit instead of the per-polynomial commit loop in NativeVerificationKey_ (would target the 41.5s of VK commitments, 23%): implemented and measured — slower, 221s vs 192s baseline. Batching 28 MSMs over 8.4M points costs more in memory traffic than it saves at 8 threads. Not pursued.
• Raising CPUS 8 → 16: 182.6s → 159.2s (-13%); 8 → 32 regresses to 188.0s. Left alone deliberately — CPUS does not confine the test to a cpuset, so extra threads are taken from the 31 other jobs sharing the box. Worth revisiting only alongside real cpuset isolation.
• Skipping the duplicate CircuitChecker::check: tempting, since test_tampering normally covers it via InvalidWitness::Target::None. But for IsRootRollup the target list is deliberately trimmed to { VKHash } ("Only one for Root because it is very heavy"), so Target::None never runs for this circuit. GenerateVKFromConstraints is the only place the valid root-rollup circuit is checked for satisfiability. Removing it would silently drop that coverage, so it stays.

The remaining lever

CircuitChecker::check is 47% of this test and is serial by construction — check_block threads TagCheckData/MemoryCheckData accumulators through a row-by-row loop. Parallelizing it (or giving it a fast path that skips prepare_circuit's copy when the builder is already finalized, worth 9s here) would speed up every bb circuit test, not just this one. That is real work on the reference checker with correctness stakes, so it is not bundled here.

Validation

• bash -n barretenberg/cpp/bootstrap.sh passes; the new prefix parses correctly through ci3/source_test_params (CPUS=8 MEM=20g TIMEOUT=20m), and test_cmd is unchanged, so the existing history list stays continuous.
• dsl_tests --gtest_filter='*GenerateVKFromConstraints*' (excluding the two heaviest recursion params): 178/178 pass.
• HonkRecursionConstraintTestWithoutPredicate/2.GenerateVKFromConstraints passes locally; 192.5s baseline → 182.6s after, peak RSS 19.83GiB.
• The flake itself is not locally reproducible on demand — it needs a loaded 64-core CI runner — so there is no red/green test to show. The evidence is the runtime distribution and the per-log kill points above.

---

Created by claudebox · group: slackbot · requested by ludamad (@ludamad) · Slack thread