solver: return the best iterate seen when terminating unsuccessfully

[batterseapower]

Problem

On ill-conditioned problems the primal residual reaches its attainable floor and then diverges as μ is driven further down, while the duality gap is still converging. The solver oscillates for more iterations and gives up — reporting a hard failure where a reduced-accuracy exit was available, and returning an iterate materially worse than ones it had already computed.

Motivating case (portfolio-rebalance SOCP, n=18,421, m=43,266, one SOC of dim 6,270, default tolerances):

iter 24   gap_rel 3.1e-06   pres 1.6e-12    <- pres floor
iter 25   gap_rel 6.6e-07   pres 1.4e-10    <- best balanced iterate
iter 28   gap_rel 8.7e-09   pres 8.7e-06    <- gap meets tol, pres already diverged
iter 29-42  ...oscillation, pres 1e-5..1e-3, steps collapse...
iter 43   InsufficientProgress, returned pres ~1.2e-04  -> hard fail

The divergence is the double-precision conditioning floor at small μ, not regularization: it is unchanged with static_regularization_constant ∈ {1e-8, 1e-10, 1e-12, 0}. The solver had its best answer in hand at iter 25 and walked away from it. Downstream, a caller running a tolerance-descent ladder re-solves the whole problem at looser tolerances (~287s in the production trace that motivated this) when the first solve already contained an AlmostSolved-quality iterate.

Change

Checkpoint the best iterate seen so far and restore it on unsuccessful exits (InsufficientProgress, NumericalError, MaxIterations, MaxTime), before the reduced-tolerance checks run — so both the reported status and the returned solution describe the best point encountered.

Three rules, each of which exists because of a specific failure mode:

1. Merit = distance from passing is_solved at the reduced tolerances, i.e. each quantity normalized by its tolerance and combined exactly as in that test: max(res_p/rtol_feas, res_d/rtol_feas, min(gap_abs/rtol_gap_abs, gap_rel/rtol_gap_rel)). Those are the tolerances check_convergence_almost applies to the restored iterate, so ranking by them gives a guarantee: if any candidate would have passed that check, the selected one passes it too. Normalizing by the full tolerances instead can rank an iterate that fails the reduced check above one that passes it.
2. Only κ/τ ≤ 1 iterates are candidates, and the restore is declined when the final iterate has κ/τ > 1 — those lie on the infeasibility branch, where cost and gap are not meaningful, so a restore would mask an emerging infeasibility certificate.
3. Never-worse guard: restore only if the checkpointed iterate actually beats the iterate currently held. The iteration loop can exit before it checkpoints — an undersized step or a failed KKT solve breaks out of the loop body ahead of the checkpoint call — so the iterate held at exit has often never been offered as a candidate, and is frequently better than anything stored. Without this guard the mechanism discards the best answer instead of recovering it. This was found by out-of-sample testing (below), not by reasoning.

This replaces the previous-iterate restore (save_prev_iterate/reset_to_prev_iterate → checkpoint_iterate/reset_to_best_iterate; Solver.prev_vars → Solver.best_vars): with rule 3 the best iterate dominates the previous one, and restoring scalars together with variables also fixes a latent inconsistency where the almost-convergence check previously evaluated restored variables against the final iterate's κ/τ. The previous-iteration scalars used by the poor-progress tests in check_termination are retained unchanged.

No numeric constants introduced — the merit is built entirely from existing settings.

Established practice: ECOS returns its best iterate on reduced-accuracy exits (Domahidi, Chu & Boyd, ECOS: An SOCP solver for embedded systems, ECC 2013 — the ECOS_OPTIMAL + ECOS_INACC_OFFSET path); MOSEK's stall handling likewise returns the best near-optimal point.

Validation suites used throughout this series

All timing is on one Apple M-series core, single-threaded, --release with
debug symbols, and every comparison is interleaved: the two binaries are
run alternately problem-by-problem within each round, so thermal drift and
machine noise affect both sides equally. Reported times are the minimum over
rounds. (Session-to-session noise on this machine is large — up to 20% between
identical binaries at different times — so non-interleaved comparisons are not
trustworthy and none are quoted.)

1. Unit / integration tests. cargo test — 20 test binaries, all passing,
plus the specific new tests listed per PR below. cargo clippy clean on the
touched files.

2. Portfolio-rebalance set (9 problems, the motivating workload).
Real conic problems from a production portfolio-rebalance backtest:
n = 9,448–18,424, m = 22,329–43,273, nnz(A) = 271k–538k, diagonal P, two
large nonnegative cones and one second-order cone of dimension 3,279–6,271
(a factor-model risk constraint: ~156 dense-ish factor rows plus a diagonal
idiosyncratic block). Used for the headline timings.

3. In-sample public corpus (224 problems). Every tiny/small/medium
non-PSD problem in a locally converted corpus: 101 Maros–Mészáros QPs,
85 Netlib LPs, 24 structured conic problems (SOCP/EXP/POW built from UCI
data), 12 Netlib-Kennington LPs, 2 Mittelmann LPs. Each has an
independently verified reference objective. Used as the status/objective
regression gate for every change.

4. Out-of-sample suites (92 problems, fetched fresh from the internet).
Because suite 3 became a tuning gate, two further suites were added that
were never used to guide any decision:

• MIPLIB 2017 benchmark set as LP relaxations — 45 instances. Downloaded
  from miplib.zib.de; the MPS reader ignores INTORG/INTEND markers, so
  reading a MIPLIB file yields exactly the continuous relaxation. These are
  substantially harder for an interior-point method than the portfolio
  problems. 90s time limit per solve.

• CBLIB (Conic Benchmark Library) — 47 instances, sampled across families
  from cblib.zib.de, including the DIMACS classics nb, nb_L1, nb_L2,
  nql30/60/180, qssp30/60, sched_*. Converted with a new CBF reader;
  integer instances are taken as continuous relaxations, and rotated
  quadratic cones are mapped to second-order cones by the orthogonal
  rotation u=(x1+x2)/√2, v=(x1−x2)/√2.

  Conversion validated independently: nb solves to −5.0703094644e-2,
  matching its published DIMACS optimum (−0.05070309), and HiGHS (installed
  for the purpose) agrees on the LP-only conversions, e.g. gen_ip054
  6765.209042728 vs Clarabel 6765.2090428.

Tests specific to this PR

• test_best_iterate_checkpoint_and_restore — checkpoint selection (better iterate replaces, worse does not), the κ/τ > 1 exclusion on both checkpoint and restore, and that a restore brings back variables and scalars.
• test_best_iterate_never_restores_something_worse — the rule-3 regression test: a better-but-uncheckpointed current iterate is kept, a genuinely worse one is replaced.
• test_best_iterate_no_checkpoint_no_restore — no candidate ⇒ no restore.

Performance: before → after

This PR is not a performance change; checkpointing is an O(n+m) copy on improving iterations only, and no measurable timing difference was observed on any suite. What changes is what the solver returns when it fails:

suite	statuses better	statuses worse	notes
Portfolio set (9)	1 — 20260105_full InsufficientProgress → AlmostSolved	0	returned objective moves closer to the independent reference (2.8e-7 vs 3.9e-7 relative)
In-sample corpus (224)	0	0	mm_yao (already AlmostSolved in both) returns its better iterate, objective shifts 5.8e-8
MIPLIB LP relaxations (45)	1	0	see below
CBLIB SOCP (47)	0	0

Successful solves are unaffected by construction: checkpointing is passive and no restore happens unless an unsuccessful status was set.

Out-of-sample testing found a bug in the first version of this PR

The first version had only rules 1 (with full tolerances) and 2. It passed the 224-problem in-sample corpus with zero status changes. On the MIPLIB LP relaxations two problems then regressed:

problem	main	this PR (first version)
brazil3	AlmostSolved, obj 1.9999982324	NumericalError, obj 1.9998233752
comp07_2idx	AlmostSolved, obj 1.5433363423e-9	NumericalError, obj 1.4936432170e-5

Bisecting across the whole branch series showed every performance PR was clean and this PR was the culprit — the cause being the missing rule 3 described above. After adding it (and switching rule 1 to the reduced tolerances):

problem	main	this PR (current)
brazil3	AlmostSolved, obj 1.9999982324	AlmostSolved, obj 1.9999982324 — identical
comp07_2idx	AlmostSolved, obj 1.5433363423e-9	AlmostSolved, obj 1.5433363423e-9 — identical

with the motivating improvement preserved. Worth stating plainly for reviewers: this class of bug is invisible to a test that asks "did the hard cases improve", and only surfaces when you check that the easy cases did not get worse.

API note

Info trait: save_prev_iterate/reset_to_prev_iterate are replaced by checkpoint_iterate (now takes settings, for the tolerances in the merit) and reset_to_best_iterate (takes settings, returns whether the restore happened); Solver.prev_vars is renamed best_vars. External implementors of Info — rare — need the mechanical rename. This diverges from Clarabel.jl until synced.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CqGdm2vZZ6HA5HT8DsRePa

[batterseapower]

Out-of-sample testing found a bug here — now fixed (commit above)

I validated this series against two public suites that had not been used as a gate: the MIPLIB 2017 benchmark set as LP relaxations (45 instances) and CBLIB (47 SOCP instances). The 224-problem corpus in the original description had become a tuning gate, so it no longer counted as out-of-sample.

On MIPLIB, two problems — brazil3 and comp07_2idx — went AlmostSolved → NumericalError and returned worse objectives. Bisecting across the branch stack showed every performance branch was clean and this PR was the culprit, despite the in-sample corpus passing with zero status changes.

Cause. The iteration loop can exit before it checkpoints: an undersized step or a failed KKT solve breaks out of the loop body ahead of the checkpoint call. So the iterate held at exit has usually never been offered as a candidate — and on these problems it was better than everything stored. The unconditional restore therefore discarded the best answer rather than recovering it, the exact opposite of the intent. Previously this was masked because main simply keeps whatever iterate it exits with.

Fix, two parts:

1. Never-worse guard. The restore is skipped unless the checkpointed iterate actually beats the iterate currently held, by the same merit. This makes the mechanism never-worse by construction rather than by argument.
2. Merit normalized by the reduced tolerances rather than the full ones. Those are the tolerances check_convergence_almost applies to the restored iterate on an unsuccessful exit, so ranking candidates by them yields a guarantee: the selected iterate passes that check whenever any candidate would have. Normalizing by the full tolerances can rank an iterate that fails the reduced check above one that passes it.

After the fix:

• brazil3 and comp07_2idx return objectives identical to main.
• The motivating case still improves: 20260105_full InsufficientProgress → AlmostSolved.
• MIPLIB LP suite: 1 status better, 0 worse, 0 objective disagreements > 1e-6.
• In-sample 224-problem corpus: unchanged (still 0 regressions from this PR).
• New regression test test_best_iterate_never_restores_something_worse covers the guard.

Worth noting for reviewers: this class of bug is invisible to a "does it improve the hard cases" test and only shows up when you check that the easy cases did not get worse.

[batterseapower]

Closing: this is the same feature as upstream oxfordcontrol#229, which is already in flight and gets the design right in a way this branch did not — it checkpoints at the top of the iteration loop, so the iterate held at exit is always a candidate, whereas this branch checkpointed near the end of the loop body and needed a never-worse guard to compensate.

The two refinements worth keeping from here (tolerance-normalised merit, and covering MaxIterations/MaxTime) have been suggested on oxfordcontrol#229 directly: oxfordcontrol#229 (comment)