Fix double-close of the CUDA-imported external-memory fd - #67
Merged
duburcqa merged 1 commit intoJul 23, 2026
Merged
Conversation
cudaImportExternalMemory() with a cudaExternalMemoryHandleTypeOpaqueFd handle takes ownership of the file descriptor - the CUDA docs state any operation on it afterwards is undefined behavior. CudaImportedBuffer's destructor nonetheless close()d it on top of cudaDestroyExternalMemory(), a double-close: the driver closes that fd during teardown, and once its number is recycled by an unrelated open() the process hits spurious 'Bad file descriptor' errors. This surfaced as batch-render tests crashing genesis's JIT source hashing when multiple GPU tests share a process or run under pytest --forked. Let CUDA own the fd; drop the close().
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
CudaImportedBuffer's destructorclose()s the Vulkan-exported memory fd (ext_fd_). But that fd is imported into CUDA viacudaImportExternalMemory()with acudaExternalMemoryHandleTypeOpaqueFdhandle, which transfers ownership of the fd to the CUDA driver — the CUDA docs state that performing any operation on the fd after import is undefined behavior.So the destructor does a double-close:
cudaDestroyExternalMemory()lets the driver close the fd, and then weclose()it again. Once that fd number has been recycled by an unrelatedopen(), the driver's teardown close (a raw syscall) lands on the wrong file, and the process starts seeing spuriousOSError: [Errno 9] Bad file descriptorin completely unrelated code.Symptom
In Genesis this manifested as batch-render tests crashing the quadrants JIT while it hashed a kernel's Python source — but only when multiple GPU tests share a process or run under
pytest --forked(where fd churn makes the recycle likely). A single test in a fresh process was unaffected, which is why it looked like a fork/isolation problem rather than a plain double-close.Fix
Drop the erroneous
close(ext_fd_); CUDA owns the fd after import and releases it with the external memory.Validation (cluster, RTX)
The two scenarios that failed deterministically with EBADF before now both pass:
pytest --forked