[pelesh]

Description

This PR adds HiOp interface to HyKKT solver from Re::Solve library.

HyKKT operates directly on the sparse KKT matrix blocks instead of using the existing Re::Solve sparse linear solver interface. The interface takes KKT matrix and rhs vector blocks from HiOp, solves KKT system and sends the solution back to HiOp.

The integration supports CPU, hybrid, and GPU execution using HiOp's existing memory and compute modes.

Proposed changes

• Add hiopKKTLinSysCompressedSparseXDYcYdHyKKT as the HiOp interface to ReSolve HyKKT.
• Build and maintain the Re::Solve Hessian, slack diagonal, equality and inequality constraint Jacobians matrix blocks.
• Reuse the matrix structure and update numerical values between solves on the host or device.
• Support CPU, hybrid, and GPU execution, with CUDA and HIP device backends, based on HiOp's mem_space and compute_mode settings.
• Copy the RHS and solution between HiOp and ReSolve using the appropriate host or device memory space.
• Add linear_solver_sparse=hykkt, hykkt_gamma, and hykkt_residual_tol options.
• Require the xdycyd KKT formulation for HyKKT, selecting it when KKTLinsys=auto and falling back to linear_solver_sparse=auto when an incompatible KKT formulation is explicitly requested.
• Use the inertia-free factorization path when HyKKT is selected.
• Check the residual returned by Re::Solve and report unsuccessful HyKKT solves back to HiOp.
• Add -hykkt support to the sparse and RAJA example drivers.
• Disable HiOp outer iterative refinement in the HyKKT example paths since HyKKT uses conjugate gradient iterations as part of solve().
• Add HyKKT regression tests for the sparse driver and the CUDA and HIP RAJA paths.
• Propagate build_kkt_matrix() failures through hiopKKTLinSysCurvCheck::factorize() and factorize_inertia_free() so HyKKT setup failures are returned to the optimization algorithm.

Checklist

• All tests pass. Code tested on:
  • CPU backend
  • CUDA backend
  • HIP backend
• I have manually run the relevant HyKKT tests and their registered self checks pass. Code tested on:
  • CPU backend: NlpSparse2_HyKKT
  • CUDA backend: NlpSparse2_HyKKT (hybrid) and NlpSparseRaja2_HyKKT_CUDA
  • HIP backend: NlpSparse2_HyKKT and NlpSparseRaja2_HyKKT_HIP
• Code compiles cleanly with flags -Wall -Wpedantic -Wconversion -Wextra.
  • Warning builds from this commit were compared with the target branch baseline, and no new warnings were introduced.
• Regression tests were run.
• The new code is documented.
• The feature branch is rebased with respect to the target branch.

Further comments

There are several current limitations to the HyKKT integration:

• HyKKT currently does not support the dual-regularization blocks used by HiOp. KKT matrix construction reports failure if nonzero dual regularization is requested.
• Re::Solve performs the HyKKT factorization as part of HyKKTSolver::solve(). As a result, factorization failure cannot currently participate in HiOp's regularization retry path in the same way as the existing direct sparse solvers.
• Cases with no equality constraints or no inequality constraints have not yet been generalized and tested through this interface.
• The example-driver HyKKT paths set ir_outer_maxit=0 because HyKKT performs a complete factorization and solve for each call. This is currently set by the example drivers rather than enforced for all uses of linear_solver_sparse=hykkt.

This also changes the shared hiopKKTLinSysCurvCheck factorization path so that failures returned by build_kkt_matrix() are propagated instead of being discarded. Among the existing implementations, this can affect hiopKKTLinSysCondensedSparse and hiopKKTLinSysSparseNormalEqn, which have matrix-build failure paths, in addition to HyKKT.

[pelesh]

Validation results

@tamar-dewilde validated the HiOp interface to ReSolve HyKKT against HiOp b8f8cd and Re::Solve develop at 4d13cf1. This branch has been rebased to the current rebase-develop; another round of testing is desirable.

CPU

• Ubuntu 24.04.4, GCC 13.3.0
• HiOp full test suite: 27/27 passed
• NlpSparse2_HyKKT passed its registered self check

CUDA

• Ubuntu 24.04.4, GCC 13.3.0, CUDA 12.8
• NVIDIA GeForce RTX 5090 Laptop GPU, compute capability 9.0
• HiOp full test suite: 36/36 passed
• NlpSparse2_HyKKT hybrid path passed its registered self check
• NlpSparseRaja2_HyKKT_CUDA passed its registered self check

HIP

• Frontier, PrgEnv-cray 8.6.0, ROCm 6.4.2
• AMD MI250X, gfx90a
• HiOp full test suite: 42/42 passed
• NlpSparse2_HyKKT passed its registered self check
• NlpSparseRaja2_HyKKT_HIP passed its registered self check

The targeted HyKKT tests completed successfully on CPU, hybrid CUDA, CUDA/RAJA, and HIP/RAJA paths.

Full validation logs:

• cpu-hykkt-validation.log
• cuda-hykkt-validation.log
• hip-hykkt-validation.log

[pelesh]

Here is CPU validation log on my machine (only sparse solvers test outputs). HiOp was built with GCC 13.4 with default options plus

-DHIOP_SPARSE=Off \
-DHIOP_USE_COINHSL=On \
-DHIOP_USE_RESOLVE=On \
-DHIOP_USE_MPI=Off

cpu.log

[shakedregev]

I'm not sure that this is correct mathematically. Even if it is, we should explain it better.

[shakedregev]

I think we need to condition requiring inertia free on the specific solver. HyKKT can simple return (n,m,0) if it succeeds, and some arbitrary wrong inertia if not.

[shakedregev]

Suggested change

	if((use_resolve_cuda_glu || use_resolve_cuda_rf || use_resolve_hip || use_hykkt) && !(inertia_free)) {
	if((use_resolve_cuda_glu || use_resolve_cuda_rf || use_resolve_hip || use_hykkt) && !(inertia_free))
	"HyKKT does not require an inertia free method, but the current implementation does. TODO: Allow HyKKT to use inertia requiring methods."
	{

[pelesh]

CUDA tests for Re::Solve interface all pass on V100 architecture. Log attached.
cuda.log

[pelesh]

All tests still pass, but now the code matches HiOp's standards. HyKKT solver interface implementation is now in src/LinAlg directory, separate from hiopKKTLinSysCompressedSparseXDYcYdHyKKT class.

[pelesh]

tracked upstream