Skip to content

Enable sparse file support on macOS - #468

Open
AdityaPainuli wants to merge 1 commit into
composefs:mainfrom
AdityaPainuli:macos-sparse-support
Open

Enable sparse file support on macOS#468
AdityaPainuli wants to merge 1 commit into
composefs:mainfrom
AdityaPainuli:macos-sparse-support

Conversation

@AdityaPainuli

Copy link
Copy Markdown

Sparse file detection in Builder is cfg-gated to Linux, Android, and FreeBSD, so macOS silently falls back to dense copies. The existing implementation already works on macOS unchanged: the non-Linux path probes fpathconf(_PC_MIN_HOLE_SIZE) before using SEEK_HOLE/SEEK_DATA, and APFS supports all three (libc exposes the constants for apple targets).

This PR adds target_os = "macos" to the three cfg gates in src/builder.rs and enables the writing_sparse size assertion on macOS.

I found this while debugging snapshot write amplification in qdrant (qdrant/qdrant#9858). Archiving a 32 MiB hole-backed file with 4 KiB of data via append_path_with_name:

  • Linux (ext4): 9,728 byte archive, GNU sparse entry
  • macOS (APFS), before: 33,555,968 bytes, dense regular entry
  • macOS (APFS), after this change: 34,304 bytes, GNU sparse entry, extracted contents byte-identical

Same code, same file, the gate was the only difference.

Verified on macOS 15 (arm64, APFS): full test suite passes including all 5 sparse tests, and writing_sparse measures 37,888 bytes, the same 4k-block bound as ext4, so the assertion reuses the 37 KiB limit.

@xzfc

xzfc commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

It's not enabled deliberately due to bugs in macOS found by other projects, see #375 (+F macos), https://bugs.launchpad.net/qemu/+bug/1776920. AFAIK, GNU tar still doesn't write sparse entries on macOS (because it uses gnulib (see first link)). I'd hesitant to enable it until investigating what the exact issue, how to reproduce it, and which APFS/macOS versions are affected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants