feat: make conda-pypi-map additive and add pypi-conda-map build overrides

[tdejager]

Description

We have a way to map conda packages to PyPI packages (used during the solve to figure out which conda packages already satisfy pypi-dependencies) and the other way around in pixi-build-python. The problem was that the user could only replace the whole mapping: as soon as you set conda-pypi-map for a channel, you lost the entire default prefix.dev mapping for it and had to replicate thousands of entries just to fix one name. The reverse direction had no override at all, only an on/off toggle.

The tricky part is that the mapping shapes are different (the default forward mapping is even keyed on repodata sha256 hashes), so this deliberately does not merge any data. Instead the user mapping is just consulted first, and on a miss we fall through to the default chain (hash → compressed → conda-forge verbatim). Additive becomes resolver layering instead of data merging.

Before, fixing one name meant hosting a full mapping file:

[workspace]
conda-pypi-map = { conda-forge = "9000-line-mapping.json" }

After:

[workspace.conda-pypi-map]
conda-forge = { mapping = { pytorch = "torch", not-on-pypi = false } }
my-company = { location = "https://internal.example.com/map.json", cache-ttl = "24h" }
internal = false   # no lookups for this channel

[package.build.config]            # pixi-build-python
ignore-pypi-mapping = false
pypi-conda-map = { torch = "pytorch", my-internal-pkg = false }

Concretely this PR:

1. Makes conda-pypi-map entries per-channel configurable: a bare location string, false, or a table with location, inline mapping entries (inline wins over the file), mode = "extend" | "replace" and cache-ttl.
2. Adds conda-pypi-map = false as the canonical global disable. conda-pypi-map = {} still works but is soft-deprecated with a warning.
3. Adds cache-ttl for URL locations: the fetched map is cached on disk and only re-fetched once it is older than the TTL. If the re-fetch fails we fall back to the stale copy with a warning, so solves keep working offline. This also makes it practical to pin the full parselmouth mapping from the raw GitHub URL (documented).
4. Adds pypi-conda-map to pixi-build-python: overrides consulted before the mapping service in both passes (project.dependencies → run and build-system.requires → host), false drops a dependency silently, per-key merge for target-specific config.
5. Gives network failures an actionable error that points at mode = "replace", <channel> = false or conda-pypi-map = false, so firewalled setups know how to opt out.

Breaking behavior

This is breaking on purpose, in two ways:

Existing manifest	Before	After	To get the old behavior
conda-forge = "mapping.json" (bare string)	Exclusive: only packages in your file got purls, everything else from that channel got none.	Additive: your entries win, everything else falls back to the default mapping.	conda-forge = { location = "mapping.json", mode = "replace" }
A mapping for channel A, while also using channel B	Configuring any mapping suppressed the conda-forge verbatim fallback for all channels, including B.	Channel B behaves exactly as if no mapping were configured.	B = false if you really want B's lookups off
conda-pypi-map = {}	Disabled all mapping.	Same, but emits a deprecation warning.	conda-pypi-map = false

The second one I'm fairly convinced was accidental: the suppression was keyed on the global mode instead of the record's channel, so mapping one channel silently degraded purl coverage everywhere else.

How Has This Been Tested?

Automated tests for the whole behavior matrix: extend/replace/disabled per channel, inline-overrides-file, the cache-ttl fresh/expired/stale/no-cache paths (the offline ones are proven network-free with a blocking middleware), parse errors with snapshots, and the reverse-direction override/skip/marker/merge cases. The extend-miss fallthrough and the unmapped-channel-keeps-verbatim cases are online tests that I verified against the live mapping. Schema is regenerated and pixi run test-schema passes.

I still want to user-test this end-to-end, therefore it's in draft.

User-test checklist

• A Inline override + explicit false: pixi lock with manifest A, check numpy has pkg:pypi/my-renamed-numpy, boltons has no purl, and python still has its normal purl (proves extend falls through).
• B The breaking flip: manifest B with the bare string → boltons still gets a purl (old behavior: none). Switch the entry to mode = "replace" → boltons loses it.
• C Global disable: manifest C → pytables gets the verbatim pkg:pypi/pytables instead of the real pkg:pypi/tables; remove the false line → pkg:pypi/tables comes back. Also try conda-pypi-map = {} and check the deprecation warning.
• D cache-ttl + parselmouth pin: manifest D → first pixi lock fetches (file appears under <cache>/conda-pypi-mapping/project-defined/), delete pixi.lock and lock again with networking off → still works; set cache-ttl = "0s" with networking off → stale-copy warning, still works.
• Firewall story: with no warm cache, block conda-mapping.prefix.dev and lock a plain manifest → the error suggests the escape hatches.
• Same channel by name and URL in the map → clear "configured more than once" error.
• http:// location → warning about tampering shows.
• E pixi-build-python: manifest E with PIXI_BUILD_BACKEND_OVERRIDE="pixi-build-python=/path/to/locally/built/pixi-build-python" → pixi build, check the rendered recipe has pytorch in run deps and my-internal-helper is gone; add the linux-64 target table and check the per-key merge.
• pypi-conda-map without ignore-pypi-mapping = false → inert warning shows.

Test manifests

A — inline override + explicit false (extend)

[workspace]
name = "map-test-inline"
channels = ["conda-forge"]
platforms = ["osx-arm64"]

[workspace.conda-pypi-map]
conda-forge = { mapping = { numpy = "my-renamed-numpy", boltons = false } }

[dependencies]
python = "3.12.*"
numpy = "*"
boltons = "*"

# any pypi dep, just so the purl amending runs
[pypi-dependencies]
rich = "*"

pixi lock && grep -B 3 "my-renamed-numpy" pixi.lock; grep -A 5 "name: boltons" pixi.lock

B — the breaking bare-string flip vs replace

echo '{ "numpy": "my-renamed-numpy" }' > mapping.json

[workspace]
name = "map-test-flip"
channels = ["conda-forge"]
platforms = ["osx-arm64"]

[workspace.conda-pypi-map]
# additive now; boltons still gets a purl from the default mapping.
conda-forge = "mapping.json"
# then switch to the old exclusive behavior; boltons loses its purl:
# conda-forge = { location = "mapping.json", mode = "replace" }

[dependencies]
python = "3.12.*"
numpy = "*"
boltons = "*"

[pypi-dependencies]
rich = "*"

C — global disable keeps the verbatim heuristic

pytables is a nice probe because its real PyPI name is tables: with lookups disabled the verbatim fallback yields pkg:pypi/pytables, with the default mapping you get pkg:pypi/tables.

[workspace]
name = "map-test-disable"
channels = ["conda-forge"]
platforms = ["osx-arm64"]
conda-pypi-map = false   # also try {} to see the deprecation warning

[dependencies]
python = "3.12.*"
pytables = "*"

[pypi-dependencies]
rich = "*"

D — pin the parselmouth mapping with cache-ttl

[workspace]
name = "map-test-ttl"
channels = ["conda-forge"]
platforms = ["osx-arm64"]

[workspace.conda-pypi-map]
conda-forge = { location = "https://raw.githubusercontent.com/prefix-dev/parselmouth/main/files/compressed_mapping.json", mode = "replace", cache-ttl = "24h" }

[dependencies]
python = "3.12.*"
pytables = "*"

[pypi-dependencies]
rich = "*"

E — pixi-build-python overrides

# pixi.toml
[workspace]
name = "map-test-build"
channels = ["conda-forge"]
platforms = ["osx-arm64"]
preview = ["pixi-build"]

[dependencies]
map-test-build = { path = "." }

[package]
name = "map-test-build"
version = "0.1.0"

[package.build]
backend = { name = "pixi-build-python", version = "*" }

[package.build.config]
ignore-pypi-mapping = false
pypi-conda-map = { torch = "pytorch", my-internal-helper = false }

# for the per-key merge check:
# [package.build.target.linux-64.config]
# pypi-conda-map = { nvidia-cublas-cu12 = false }

[package.host-dependencies]
python = "*"

# pyproject.toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "map-test-build"
version = "0.1.0"
dependencies = ["torch>=2.0", "requests", "my-internal-helper"]

PIXI_BUILD_BACKEND_OVERRIDE="pixi-build-python=$PWD/target/debug/pixi-build-python" pixi build

AI Disclosure

• This PR contains AI-generated content.
  • I have tested any AI-generated content in my PR.
  • I take responsibility for any AI-generated content in my PR.

Tools: Claude Code with Fable 5

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added sufficient tests to cover my changes.
• I have verified that changes that would impact the JSON schema have been made in schema/model.py.