Skip to content

Fix DeviceHostFile mapping views - #1658

Open
fallintoplace wants to merge 2 commits into
rapidsai:mainfrom
fallintoplace:fix-device-host-file-mapping-views
Open

Fix DeviceHostFile mapping views#1658
fallintoplace wants to merge 2 commits into
rapidsai:mainfrom
fallintoplace:fix-device-host-file-mapping-views

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Jun 20, 2026

Copy link
Copy Markdown

Summary

  • include host_buffer keys in DeviceHostFile mapping views
  • deduplicate keys that are visible through more than one internal store
  • clear old internal storage before overwriting a key, so stale others, device, or host values cannot win later lookups
  • add CPU-only regressions for host-only mapping views and overwrite/delete transitions

Root cause

Host-only values are stored in host_buffer, but __len__() and __iter__() only looked at device_buffer and others. That drops host-only keys when device_memory_limit=None, where device_buffer is a plain dict instead of a zict buffer that can see the host layer.

There was a related overwrite issue: __setitem__() only removed an existing key when it was visible through device_buffer, while __getitem__() checks others first. Reusing a key across others, host, and device storage could leave stale values behind.

Validation

  • uv run --no-project --with ruff ruff check dask_cuda/device_host_file.py dask_cuda/tests/test_device_host_file_mapping.py
  • uv run --no-project --with ruff ruff format --check dask_cuda/device_host_file.py dask_cuda/tests/test_device_host_file_mapping.py
  • git diff --check
  • direct NumPy/pandas behavior check for device_memory_limit=None and device_memory_limit=1
  • direct overwrite/delete behavior checks for host, device-like, and spillable-like values

@fallintoplace
fallintoplace requested a review from a team as a code owner June 20, 2026 15:30
@copy-pr-bot

copy-pr-bot Bot commented Jun 20, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant