fix: detect site-packages clobbering of conda packages by PyPI wheels#6344
Merged
baszalmstra merged 2 commits intoJun 11, 2026
Merged
Conversation
31b622a to
b188175
Compare
b188175 to
99b94df
Compare
The conda-pypi clobber check compared wheel RECORD paths against conda's paths.json entries by joining regular wheel files onto the wheel's destination site-packages directory and then stripping the prefix root. That directory comes from the interpreter's *virtualenv scheme*, which is relative to the prefix (e.g. lib/python3.12/site-packages), so the strip_prefix of an absolute prefix root always failed and every regular wheel file was silently skipped: site-packages clobbering (the common case) was never detected. Only PEP 427 .data/<scheme> files, which are joined onto the interpreter's absolute scheme directories, could match — which is why script clobbering (e.g. bin/prek) worked. The relative form is exactly the prefix-relative shape conda's paths.json uses, so accept it directly (rejecting paths that escape the prefix) and keep the prefix strip for the absolute .data paths. Also rename the misleading root_scheme field to site_packages with a doc comment on the relative convention, debug-log the previously silent skip and error paths, and align the unit-test fixture with the production (relative) convention; the old fixture used an absolute path, which is why the tests did not catch this. Reproduced with a conda + pypi boltons overlap: the warning now lists the overwritten site-packages files.
99b94df to
c26fcb6
Compare
Instead of joining regular wheel files onto the interpreter's virtualenv scheme *template* and .data files onto individually-listed absolute scheme dirs, derive all destinations from interpreter().layout().scheme — the exact layout uv's installer writes with — relative-ized against the interpreter's own sys_prefix. Prediction and writes now share one source and cannot drift; a custom site-packages location (python_site_packages_dir) flows through because the prediction is the interpreter's real layout, not the venv template. The check no longer needs the pixi-supplied prefix root at all: everything is compared in conda's prefix-relative form, and entries escaping the prefix (leading .. after normalization, or absolute RECORD entries) are skipped. get_wheel_info now returns the wheel kind instead of a scheme directory, and the kind selects between the purelib/platlib destinations. User-tested: boltons site-packages clobber (python 3.12 and free-threaded 3.13t, which relocates site-packages) and the prek bin/prek scripts clobber all warn correctly.
d2ea2ca to
76ef184
Compare
baszalmstra
approved these changes
Jun 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
While user-testing #6333 I expected an explicit "this conda package is not a PyPI package" mapping to trigger the clobber warning from #6292 (conda
boltons+ PyPIboltonsinstalled over each other), but nothing showed up. Digging in, it turns out the clobber detection never worked for regular site-packages files — only for PEP 427.data/<scheme>/files.Before this PR, this is what happened:
paths.jsongives us prefix-relative paths, the wheel RECORD gives us site-packages-relative paths:interpreter().virtualenv()and is relative:.strip_prefix("/proj/.pixi/envs/default")on it. An absolute prefix never matches a relative path, so this always failed and the entry was silentlycontinued — even though the path from step 2 already equals thepaths.jsonkey..data/<scheme>/entries joined with an absolute directory instead, so the strip worked there:bin/prekexample from feat: Improve PyPI > Conda clobber detection #6292 warned, but site-packages files never did.The fix (after review discussion, see below): derive all destinations from
interpreter().layout().scheme— the exact layout uv's installer writes with — relative-ized against the interpreter's ownsys_prefix. Prediction and writes then share one source and cannot drift, and the whole comparison happens in conda's prefix-relative form: nostrip_prefixagainst an externally-supplied root anymore. Entries that escape the prefix (a leading..after normalization, or an absolute RECORD entry) are skipped; entries that merely escape site-packages (like prek'sbin/prek) normalize into the prefix and compare fine.With the fix this now warns again for the common case:
Some smaller things in here as well:
get_wheel_infonow returns the wheel kind (purelib/platlib) instead of a scheme directory; the destination struct holds prefix-relative paths only, so the absolute/relative mix that hid the bug is gone entirely.pkg/__init__.pycase.Errat the call site, two silentcontinues per wheel); these now log at debug level so a dead check can't hide again.CondaPrefixPathnewtype: the registry is keyed by it and the only constructors are "from a condapaths.jsonentry" and "from a wheel RECORD entry", so the two sides of the comparison can no longer disagree about the path convention without failing to compile.Reproducer
pixi installwarns with this PR and is silent on main. Once #6333 lands,boltons = falsein an inline mapping is the one-liner version of this.How Has This Been Tested?
All unit tests in
pixi_install_pypipass, including the new regression test; the two existing prefix-escape tests keep their exact expectations under the corrected fixture. I also ran the reproducer above against a debug build and verified the warning appears with the overlapping site-packages files listed.@baszalmstra asked whether this respects the python package's
python_site_packages_dir. After some back and forth this led to a better design: the check now derives its destinations frominterpreter().layout().scheme, which is literally the layout uv installs with (pixi constructs the environment viaInterpreter::query+PythonEnvironment::from_interpreter, so the probed sysconfig scheme is what both the installer and the check see). That makes the prediction-matches-writes property an identity instead of a convention, and a relocated site-packages flows through end to end. The record key itself is never consulted —paths.jsonis already the ground truth of where conda's files are, and the layout is the ground truth of where uv writes; the check is the intersection of the two.Verified with
python-freethreading(the case the key exists for), where the warning correctly reportslib/python3.13t/site-packages/..., alongside the regular boltons and prek reproducers.AI Disclosure
Tools: Claude Code with Fable 5
Checklist: