Fall back to the best iterate when terminating with a numerical error

[batterseapower]

The bug

When the interior-point loop terminates with NumericalError or InsufficientProgress, the reduced-tolerance ("almost solved") acceptance check is evaluated against the final iterate. But on a numerical failure the final iterate is typically the one that just went bad -- that is why the solver stopped. The good iterate from one step earlier is discarded, and the solve is reported as a failure on the strength of a point the solver had already superseded.

This shows up on problems whose attainable gap floor sits close to the requested tolerance: the primal residual jumps from ~1e-10 to ~3e-5 in a single step, the loop bails out, and the almost-solved test is then applied to the ruined point and fails. The solver had a perfectly acceptable answer in hand and threw it away.

The fix

Track the best iterate seen so far and restore it before the almost-solved check runs.

• Merit is the iterate's distance from passing is_solved at the reduced tolerances: each quantity divided by the tolerance that will judge it, combined exactly as in that test, so the duality gap enters through whichever of its absolute or relative forms is closer to passing. Those are the tolerances check_convergence_almost applies to the restored iterate, so ranking by them makes the choice safe rather than merely reasonable: the selected iterate has the smallest reduced merit of all candidates, so if any candidate would have passed that check, the selected one passes it too. Comparing the raw quantities instead can prefer an iterate that fails the check over one that passes it, because the tolerances differ from each other -- by default reduced_tol_feas is 1e-4 while reduced_tol_gap_rel is 5e-5.
• Iterates with ktratio > 1 are never candidates: those lie on the infeasibility-certificate path and must not be presented as approximate solutions.
• A non-finite merit is never an improvement. Every comparison against NaN is false, so a naive merit >= best test would let a NaN iterate be stored as the best one and, because the restore requires a finite best merit, silently disarm the fallback for the rest of the solve -- on exactly the solves it exists to rescue. Testing strict improvement rejects it, and also rejects the infinite merit a reduced tolerance of zero would produce.
• The restore is a no-op when the final iterate is already the best one, so the path is inert on every solve that terminates Solved.
• It fires on MaxIterations and MaxTime as well as the error statuses, because post_process runs the same acceptance check for those exits and their final iterate is no more likely to be the best one seen.

Cost is one extra DefaultVariables and a copy on strict improvement.

Evidence

It only ever helps. Verified against stock on 224 public problems (101 Maros-Meszaros QPs, 85 Netlib LPs, 24 structured conic, 12 Netlib-Kennington, 2 Mittelmann): no status changes and no iteration-count changes, and a single objective difference, on mm_yao, which terminates AlmostSolved in both and returns its better iterate. That is the expected result -- problems that terminate normally never reach the restore -- and it bounds the blast radius: the patch cannot regress a solve that was already succeeding.

Two further suites, used only as gates: 45 MIPLIB 2017 benchmark instances as LP relaxations and 84 CBLIB instances. No status changes and no objective disagreements above 1e-6 on either, and timing unchanged throughout (-0.1% and -0.0%, i.e. noise), as expected for a path that is inert unless a solve fails.

Two of those MIPLIB relaxations, brazil3 and comp07_2idx, are the cases most sensitive to how the merit is defined; both return objectives identical to stock (1.9999982324 and 1.5433363423e-9).

Tests

Five unit tests cover the semantics: a degraded final iterate is discarded in favour of the best seen; a final iterate that is itself the best is left untouched; iterates on the infeasibility path are never eligible to be saved or restored; the merit ranks by distance from the acceptance check rather than by raw magnitude, encoding a case where the raw comparison would prefer an iterate that fails the check over one that passes; and a NaN iterate cannot displace the best one or disarm the fallback. The last two were each checked to fail against the code they replaced, so they test what they claim to.

Full suite green: cargo test, 20 test binaries, 0 failures.

[CLAassistant]

All committers have signed the CLA.

[batterseapower]

Two suggested refinements, from testing this mechanism against a wider set of problems.

1. Normalise the merit by the reduced tolerances

The merit is currently max(gap_rel, res_primal, res_dual), comparing raw quantities. But the check that decides the outcome on an unsuccessful exit is check_convergence_almost, which tests each quantity against a different tolerance — with the defaults, reduced_tol_feas = 1e-4 but reduced_tol_gap_rel = 5e-5, so feasibility is allowed to be twice as loose. Ranking candidates by the raw maximum therefore misweights them, and can prefer an iterate that fails the check over one that passes it.

Concretely, with those defaults:

candidate	gap_rel	res_p = res_d	raw merit	passes almost-check?
A	8e-5	1e-5	8e-5 (ranked better)	no — gap exceeds 5e-5
B	2e-5	9e-5	9e-5 (ranked worse)	yes

A is selected, and the solve is reported as a failure even though B was available.

Normalising each quantity by the tolerance that will judge it, and combining them exactly as is_solved does — so the duality gap enters through whichever of its absolute/relative forms is closer to passing — removes that failure mode and gives a guarantee: the selected iterate is the one with the smallest reduced merit, so if any candidate would have passed the check, the selected one passes it too.

let gap = T::min(
    self.gap_abs / settings.reduced_tol_gap_abs,
    self.gap_rel / settings.reduced_tol_gap_rel,
);
let feas = T::max(
    self.res_primal / settings.reduced_tol_feas,
    self.res_dual / settings.reduced_tol_feas,
);
T::max(gap, feas)

This also picks up gap_abs, which the current merit ignores even though is_solved accepts on it.

2. Restore on MaxIterations and MaxTime as well

The restore currently fires on is_errored(). But post_process runs the reduced-tolerance check for MaxIterations and MaxTime too, and those exits have the same property that motivates this PR — the final iterate is not necessarily the best one seen. Extending the condition to cover them makes the two consistent.

On the evidence

The 133 Maros–Mészáros result is a good blast-radius bound. If it helps, the same property holds on a wider set I ran while testing this area: 224 public problems (Maros–Mészáros, Netlib, Netlib-Kennington, Mittelmann, structured conic) plus 45 MIPLIB 2017 LP relaxations and 47 CBLIB second-order-cone instances. Two MIPLIB relaxations, brazil3 and comp07_2idx, are worth adding as regression cases: they sit exactly in the regime this PR targets, and they are sensitive to the merit definition in point 1.

One observation in this PR's favour, since I arrived at the same feature independently and got this part wrong: checkpointing at the top of the iteration loop, as done here, is the right choice. Checkpointing near the end of the loop body instead leaves the iterate held at exit uncheckpointed — an undersized step or failed KKT solve breaks out before the call — so a restore can then replace a good final iterate with an older, worse one. The placement here avoids that by construction, and the merit <= best_merit guard makes it belt-and-braces.

[batterseapower]

Pushed a commit applying both of the refinements suggested above.

Merit now ranks by distance from the acceptance check. Each quantity is divided by the tolerance that will judge it, and they are combined exactly as is_solved does, so the duality gap enters through whichever of its absolute/relative forms is closer to passing:

fn termination_merit(&self, settings: &DefaultSettings<T>) -> T {
    let gap = T::min(
        self.gap_abs / settings.reduced_tol_gap_abs,
        self.gap_rel / settings.reduced_tol_gap_rel,
    );
    let feas = T::max(
        self.res_primal / settings.reduced_tol_feas,
        self.res_dual / settings.reduced_tol_feas,
    );
    T::max(gap, feas)
}

This makes the selection safe rather than merely reasonable: the chosen iterate has the smallest reduced merit of all candidates, so if any candidate would have passed check_convergence_almost, the chosen one passes it too. It also brings in gap_abs, which the previous merit ignored even though is_solved accepts on it. Both trait methods now take settings to reach the tolerances.

The restore covers MaxIterations and MaxTime as well as is_errored(), since post_process runs the same acceptance check for those exits.

Tests

A fourth test, merit_ranks_by_distance_from_the_acceptance_check, encodes the counterexample directly: the first candidate has the smaller raw maximum (8e-5 against 9e-5) but its gap is outside the reduced tolerance, the second is inside on every count, and the test asserts both facts via is_solved and that the second is the one saved and restored. The three existing tests are unchanged apart from threading settings through. Full suite green (cargo test, 20 binaries); cargo clippy clean.

Verification against stock

suite	status changes	iteration changes	objective differences
224 public problems (Maros–Mészáros, Netlib, Netlib-Kennington, Mittelmann, structured conic)	0	0	1 — mm_yao, AlmostSolved in both, returning its better iterate (5.8e-8 relative)
45 MIPLIB 2017 LP relaxations	0	—	none above 1e-6
84 CBLIB instances	0	—	none above 1e-6

Timing is unchanged on all three (−0.1% and −0.0%, i.e. noise), as expected for a path that is inert unless a solve fails.

Two MIPLIB relaxations, brazil3 and comp07_2idx, are the cases most sensitive to the merit definition; both return objectives identical to stock (1.9999982324 and 1.5433363423e-9).

[batterseapower]

One more commit, from re-reading the save path: a NaN iterate could displace the best one and silently disarm the fallback.

The improvement test was merit >= self.best_merit, and every comparison against NaN is false. So a NaN merit counted as an improvement: the NaN iterate was stored as the best, and because reset_to_best_iterate requires a finite best_merit, the fallback was then disabled for the remainder of the solve. A solve whose iterates have gone non-finite is precisely the case this PR exists to rescue, so that is the worst possible place to lose it.

Testing strict improvement instead rejects it, and is identical on finite inputs:

if !(merit < self.best_merit) {
    return;
}

That also rejects an infinite merit, which is what a reduced tolerance of zero would produce — the tolerances are not validated positive, so this makes a misconfiguration degrade to "feature inert" rather than to something stranger.

a_nan_iterate_cannot_displace_the_best_one covers it: a good iterate, then one whose residuals, gap and κ/τ are all NaN, then a degraded final iterate; the good one must be the one restored, and best_merit must still be finite. I checked that the test fails against the previous comparison and passes against this one, so it is testing the thing it claims to.

Nothing changes on finite input, as expected for a defensive fix — 224 public problems, 45 MIPLIB 2017 LP relaxations and 84 CBLIB instances report exactly the same statuses, iteration counts and objectives as the previous commit, with timing within noise (+0.1% / +0.2%).