Describe the bug
A long-lived Python process using cuOpt 26.8.0 aborted on its 24th sequential MIP solve with free(): invalid size. glibc raised SIGABRT while an OpenMP initial-clique task destroyed its task-local simplex::user_problem_t<int, double> copy. The failure occurred just after the concurrent LP root relaxation started.
The same workload is stable when each solve runs in a fresh process. Disabling both mip_clique_cuts and mip_zero_half_cuts is the targeted A/B control because that prevents the asynchronous initial-clique task from launching. There was no OOM kill (about 4.9 GB peak on a 61 GB host) and no NVIDIA Xid or GPU error.
Source-level race on current main
branch_and_bound_t::solve launches an OpenMP task that eventually assigns a newly allocated table to the member clique_table_. Meanwhile the root-solve thread can copy that same std::shared_ptr member into cut_generation_t. Concurrent assignment and copy of one shared_ptr object is an unsynchronized read/write and therefore undefined behavior. It can also leave cut generation holding nullptr even after the task publishes the table.
The table should instead be allocated and assigned to the owning member before task creation. The task can then mutate only the pointee, with the existing signal/task dependency joining mutation before cut generation reads it.
Native stack
free(): invalid size
malloc_printerr
free
cuopt::mathematical_optimization::simplex::user_problem_t<int,double>::~user_problem_t()
libcuopt.so
OpenMP task worker
The destructor is where heap corruption was detected, not necessarily where it originated. The model and evolving separator-cut pool can be provided privately if useful; together they are roughly 36 MB.
Environment
- cuopt-cu12 26.8.0, git hash 400863c
- Python 3.14.2
- Arch Linux x86-64, glibc 2.44
- NVIDIA driver 610.57.04, CUDA 12.9 wheels
- GeForce RTX 4070 Ti SUPER
- AMD Ryzen 7 7800X3D
Possibly related to #1768, but this occurrence uses sequential solves in one long-lived process and has a specific initial-clique-task stack and source-level publication race.
Describe the bug
A long-lived Python process using cuOpt 26.8.0 aborted on its 24th sequential MIP solve with
free(): invalid size. glibc raised SIGABRT while an OpenMP initial-clique task destroyed its task-localsimplex::user_problem_t<int, double>copy. The failure occurred just after the concurrent LP root relaxation started.The same workload is stable when each solve runs in a fresh process. Disabling both
mip_clique_cutsandmip_zero_half_cutsis the targeted A/B control because that prevents the asynchronous initial-clique task from launching. There was no OOM kill (about 4.9 GB peak on a 61 GB host) and no NVIDIA Xid or GPU error.Source-level race on current main
branch_and_bound_t::solvelaunches an OpenMP task that eventually assigns a newly allocated table to the memberclique_table_. Meanwhile the root-solve thread can copy that samestd::shared_ptrmember intocut_generation_t. Concurrent assignment and copy of oneshared_ptrobject is an unsynchronized read/write and therefore undefined behavior. It can also leave cut generation holdingnullptreven after the task publishes the table.The table should instead be allocated and assigned to the owning member before task creation. The task can then mutate only the pointee, with the existing signal/task dependency joining mutation before cut generation reads it.
Native stack
The destructor is where heap corruption was detected, not necessarily where it originated. The model and evolving separator-cut pool can be provided privately if useful; together they are roughly 36 MB.
Environment
Possibly related to #1768, but this occurrence uses sequential solves in one long-lived process and has a specific initial-clique-task stack and source-level publication race.