Skip to content

feat: add PyPI dependencies to pixi global environments#6334

Open
wolfv wants to merge 8 commits into
mainfrom
claude/global-pypi-dependencies
Open

feat: add PyPI dependencies to pixi global environments#6334
wolfv wants to merge 8 commits into
mainfrom
claude/global-pypi-dependencies

Conversation

@wolfv

@wolfv wolfv commented Jun 10, 2026

Copy link
Copy Markdown
Member

Description

Adds support for PyPI dependencies in pixi global environments, built on top of the command-dispatcher refactor.

Note: This PR is stacked on #6324 and includes its commits. Once #6324 merges, this PR reduces to the pixi global feature commits. If only the refactor should land for now, merge #6324 and leave this one open.

Manifest support:

[envs.jupyter]
channels = ["conda-forge"]
dependencies = { python = "3.13.*" }
pypi-dependencies = { jupyterlab = "*" }

PyPI packages are resolved against the environment's python interpreter (via CommandDispatcher::solve_pypi_environment) and installed into its site-packages (via install_pypi_environment) after the conda packages.

CLI support:

  • pixi global install python --pypi httpx --pypi "flask>=2"
  • pixi global add --environment env --pypi httpx
  • pixi global remove --environment env httpx (falls back to PyPI deps when the name is not a conda dependency)

Behavior details:

  • PyPI console scripts are exposed like conda executables (enumerated from each pixi-installed distribution's dist-info/RECORD); declared deps auto-expose, transitive ones don't.
  • Sync detection: an environment is out of sync when a declared PyPI package is missing from site-packages, or when pixi-installed leftovers remain after the last pypi-dependency was removed.
  • pixi global remove of a PyPI dep forces a reinstall so the now-extraneous distribution is uninstalled, and unexposes its scripts.
  • Known limitation (documented in code): a changed version requirement for an already-installed package does not mark the environment out of sync; pixi global update always re-resolves environments with PyPI deps.

How Has This Been Tested?

  • Unit tests for manifest parsing and the add/remove PyPI dependency round-trip.
  • Verified end-to-end against conda-forge + pypi.org with a debug build: install with --pypi (script exposed and runnable), add, partial remove (only the removed package is uninstalled), last-dep remove (site-packages cleanup + unexpose), and no-op re-sync.
  • A perf regression found via CI (duplicate conda-meta parsing in the expose sync, which pushed the macOS pytest job past its 10-minute timeout) is fixed in the last commit.

https://claude.ai/code/session_01S4nY8g4frki9JvtuUnZm85


Generated by Claude Code

claude added 8 commits June 10, 2026 06:00
Add an install-pypi operation to the CommandDispatcher so that PyPI
packages can be installed into a conda prefix through the same handle
that already drives conda solves and installs. This makes the PyPI
install pipeline reusable outside the workspace install path, e.g. for
pixi global.

- Extract UvReporter/UvReporterOptions into a new pixi_uv_reporter
  crate. pixi_reporters depends on pixi_command_dispatcher, so the
  reporter had to move below the dispatcher to let it depend on
  pixi_install_pypi without a cycle. pixi_reporters re-exports the
  types so existing users are unaffected.
- Add InstallPypiEnvironmentSpec and
  CommandDispatcher::install_pypi_environment, which wraps
  PyPIEnvironmentUpdater and derives the wheel link mode from the
  dispatcher's configured link options.
- Switch the workspace install path in pixi_core to build the spec and
  go through the dispatcher instead of wiring up the updater configs
  inline.
Complete the PyPI refactor started with install: resolution now also
runs through the CommandDispatcher, so both halves of the PyPI story
(solve + install) are reachable outside the workspace code, e.g. for
pixi global.

- Move the resolve pipeline (resolve_pypi, LazyBuildDispatch,
  CondaResolverProvider) from pixi_core::lock_file::resolve into
  pixi_install_pypi::resolve.
- Decouple it from the workspace with a new CondaPrefixProvider trait:
  when uv must build an sdist to get metadata, the provider supplies a
  conda prefix (python interpreter + activation env vars) on demand.
  pixi_core implements it as WorkspaceCondaPrefixProvider wrapping the
  memoized CondaPrefixUpdater and environment activation; both the
  lock-file update path and the satisfiability metadata checks use it.
- Move PypiPackageIdentifier into pixi_install_pypi; the
  satisfiability-specific satisfies() check stays in pixi_core as an
  extension trait.
- Add SolvePypiEnvironmentSpec and
  CommandDispatcher::solve_pypi_environment, deriving the link mode
  from the dispatcher's configured link options.
- Drop the dead conda_task plumbing: LazyBuildDispatch::conda_task was
  never assigned, so resolve_pypi now returns just the locked records
  and the PypiGroupSolved task keeps forwarding None.
- Remove the uv-* dependencies pixi_core no longer needs.
rattler_digest and uv-preview were only used by the PyPI resolve code
that moved to pixi_install_pypi.
Global environments can now declare PyPI packages in the manifest:

    [envs.jupyter]
    channels = ["conda-forge"]
    dependencies = { python = "3.13.*" }
    pypi-dependencies = { jupyterlab = "*" }

After the conda packages are installed, the declared PyPI packages are
resolved with CommandDispatcher::solve_pypi_environment and installed
into the prefix's site-packages with install_pypi_environment — the
first consumer of the dispatcher PyPI pipeline outside the workspace.
The CondaPrefixProvider implementation simply hands out the freshly
installed prefix when a source distribution must be built during
resolution.

Sync detection treats an environment as out of sync when a declared
PyPI package has no dist-info in site-packages (name presence only;
spec changes for an installed package are reconciled on the next
reinstall), and update operations always re-resolve environments with
PyPI dependencies.
PyPI dependencies of global environments can now be managed from the
command line instead of only by editing the manifest:

- pixi global install python --pypi httpx --pypi "flask>=2"
- pixi global add --environment env --pypi httpx
- pixi global remove --environment env httpx (falls back to the PyPI
  dependency when the name is not a conda dependency)

The new Manifest::add_pypi_dependency/remove_pypi_dependency methods
edit the [envs.*.pypi-dependencies] table. Removing the last PyPI
dependency now also cleans pixi-installed packages out of the prefix's
site-packages: leftover detection (dist-info INSTALLER == uv-pixi)
marks the environment out of sync and the installer removes the
packages on the next sync.
PyPI-installed executables are now treated like conda executables when
exposing binaries from a global environment:

- Executables are enumerated from each pixi-installed distribution's
  dist-info RECORD (entries that land in the prefix's binary folders).
  Declared pypi-dependencies are auto-exposed like direct conda
  dependencies; executables of transitive distributions are not
  auto-exposed but are protected from stale-mapping pruning.
- pixi global remove unexposes the removed package's scripts (using a
  snapshot taken before removal) and forces a reinstall: a removed PyPI
  dependency cannot be detected by the presence-based sync check since
  transitive distributions are legitimately undeclared, so the
  installer is invoked explicitly to drop the now-extraneous
  distribution.
The macOS pytest job of the previous run was cancelled by runner
infrastructure.
The PyPI executable enumeration added an extra find_installed_packages
call (a full parse of every conda-meta record) to
executables_of_all_dependencies and ran another one per expose sync in
executables_of_pypi_dependencies, roughly doubling the runtime of the
global integration test suite and pushing the macOS pytest job past
its 10 minute timeout.

Reuse the prefix records that executables_of_all_dependencies already
loads, and skip the prefix entirely in
executables_of_pypi_dependencies when the environment declares no PyPI
dependencies.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants