[Interop] Prototype Vulkan opaque-FD import for CUDA and AMDGPU

[rglebovamd]

Context

The Nyx AMD/ROCm integration needs an API-neutral way to import Vulkan external-memory allocations into the active Quadrants CUDA or AMDGPU backend. Nyx exports an opaque FD plus the full allocation size, logical payload size, and byte offset. The Genesis plugin should not select or call CUDA/HIP directly.

This is the quick prototype requested for API/design review. It is intentionally a draft, Linux-only, and not presented as the final production implementation.

Prototype approach

• Adds quadrants.interop.VkImport.
• Imports VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD through CUDA Driver or HIP based on the active Quadrants backend.
• Maps only the requested logical range while importing the complete Vulkan allocation.
• Implements DLPack producer methods for PyTorch views.
• Provides release_to_vulkan() and acquire_from_vulkan() ownership-boundary methods.
• Uses transactional FD ownership: imports a duplicate and consumes the caller FD only after import and mapping both succeed.
• Rejects invalid ranges, shapes, dtypes, unsupported backends, and mismatched GPUs with actionable errors.
• Frees mapped memory and the imported external-memory object on close().

Validation performed

Using the Nyx integration on AMD and NVIDIA:

• CUDA and HIP opaque-FD import/read/write passed.
• 16-byte allocation / 12-byte logical payload passed.
• Nonzero byte offset mapping passed.
• DLPack tensor shape, dtype, and device passed.
• Failed-construction FD ownership passed.
• AMD-Vulkan-to-CUDA and NVIDIA-Vulkan-to-HIP mismatch rejection passed.
• Full Genesis -> Quadrants VkImport -> Nyx rendering passed on both backends.
• Unit metadata/ownership validation in this PR: 8 tests passed.

Important prototype limitations / review questions

1. Synchronization: the prototype uses device-wide synchronization. What should the production stream/queue API be? Do we want imported/exported binary or timeline semaphores for Vulkan visibility?
2. Implementation layer: should the final external-memory import live in Quadrants C++ bindings rather than Python ctypes?
3. DLPack lifetime: the prototype requires the VkImport object to outlive every tensor view and uses a no-op DLPack deleter. What ownership model should the final API expose?
4. Device selection: CUDA currently uses device zero in the active process. The final version should use the exact initialized Quadrants device and ideally verify PCI/device identity against Vulkan.
5. Platform scope: this implements Linux opaque FDs only. Win32 external-memory handles need a separate ownership contract.
6. API shape: please review the constructor fields and the release_to_vulkan() / acquire_from_vulkan() names before the Nyx plugin pins them.

Test

pytest tests/python/test_vk_import.py -q

The Nyx plugin PRs remain drafts until this API is finalized, released/pinned, and the real CUDA/AMD conformance matrix passes.

Semaphore capability note

Both tested Vulkan devices report timeline-semaphore support and CUDA exposes the required external-semaphore functions. However, the ROCm 7.2 HIP headers explicitly document hipImportExternalSemaphore, hipSignalExternalSemaphoresAsync, and hipWaitExternalSemaphoresAsync as unsupported on Linux. The public API should therefore keep semaphores optional and retain a device/queue synchronization fallback for the initial AMD Linux path unless runtime probing shows the HIP documentation is stale.

[hughperkins]

please remove the word 'prototype' from the doc

[hughperkins]

.md docs should not be line-wrapped

[hughperkins]

remove 'current'

[hughperkins]

replace 'A production implementation' with 'FIXME:'

[hughperkins]

remove 'currently'

[hughperkins]

comments should be 120c wrapped, not 80c