Implement auditwheel repair with patchelf by messense · Pull Request #742 · PyO3/maturin

messense · 2021-12-12T13:02:52Z

Linux dylib dependency resolution built with lddtree-rs, patch dylib's soname and rpath is done with patchelf which means user needs to install it beforehand. Would be great if there is a pure Rust version but unfortunately I don't have the expertise to do that.

The code isn't well organized yet, but hey it works™.

Closes #649

netlify · 2021-12-12T13:02:57Z

✔️ Deploy Preview for maturin-guide canceled.

🔨 Explore the source changes: 1af539a

🔍 Inspect the deploy log: https://app.netlify.com/sites/maturin-guide/deploys/61bf28a28ef35e00075972af

konstin

This is fancy!

The code isn't well organized yet, but hey it works™.

I couldn't even implement this, I'm happy that you're doing that :D

One other thing I've noticed is https://github.com/messense/lddtree-rs/blob/aa3ba6af1c7e00b1d084596fecc82dc3886a1a34/src/lib.rs#L138. Would this mean that a bogus lib could silently lead to a broken wheel?

BTW have you seen https://twitter.com/andy_kelley/status/1470338795266457600? It's not ready yet, but if it's completed we could drop the docker container and just compile for manylinux2010 directly. Also zig cc is on pypi so we could just list it as a dependency.

konstin · 2021-12-14T17:33:10Z

+                    "Error checking for manylinux/musllinux compliance".to_string()
+                }
+            })?;
+        let external_libs = if should_repair && !self.editable {


Is this even executed for editable installs?

I shouldn't, I was being defensive there.

messense · 2021-12-15T01:57:13Z

One other thing I've noticed is https://github.com/messense/lddtree-rs/blob/aa3ba6af1c7e00b1d084596fecc82dc3886a1a34/src/lib.rs#L138. Would this mean that a bogus lib could silently lead to a broken wheel?

Thanks for reporting. I forgot to remove that in messense/lddtree-rs@28d4c27#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759 now that it uses a placeholder Library for reporting library not found errors later.

Edit: fixed in messense/lddtree-rs@fdf698d

messense · 2021-12-15T06:35:44Z

BTW have you seen https://twitter.com/andy_kelley/status/1470338795266457600? It's not ready yet, but if it's completed we could drop the docker container and just compile for manylinux2010 directly. Also zig cc is on pypi so we could just list it as a dependency.

I have seen that. Hopefully when it's ready it can be used when C++ is not involved!

messense · 2021-12-15T13:59:47Z

Hmm, looks like Python 3.10.1 is broken.

https://github.com/NixOS/patchelf

Co-authored-by: konstin <konstin@mailbox.org>

https://build.opensuse.org/request/show/942187 by user mia + dimstar_suse - Update to 0.12.5 * Fix docs for new and init commands in maturin --help gh#PyO3/maturin#734 * Fix undefined auditwheel policy panic gh#PyO3/maturin#740 * Fix upload::canonicalize_name() regex subst gh#PyO3/maturin#741 * Bump serde from 1.0.130 to 1.0.131 gh#PyO3/maturin#745 * Bump sha2 from 0.9.8 to 0.10.0 gh#PyO3/maturin#746 * Add Cargo.lock to sdist when --locked or --frozen specified gh#PyO3/maturin#749 * Implement auditwheel repair with patchelf gh#PyO3/maturin#742 * Support pyo3 abi3-py310 feature gh#PyO3/maturin#750 - Changes in 0.12.4: * Bump anyhow from 1.0.50 to 1.0.51 gh#PyO3/maturin#717 * init: new command similar to cargo init gh#PyO3/maturin#719 * Don't package non-path-dep crates in sdist for workspaces gh#PyO

* Add support for repairing cross compiled linux wheels in [#754](PyO3/maturin#754) * Add support for `manylinux_2_28` and `manylinux_2_31` in [#755](PyO3/maturin#755) * Remove existing so file first in `maturin develop` command to avoid triggering SIGSEV in running process in [#760](PyO3/maturin#760) * Fix docs for `new` and `init` commands in `maturin --help` in [#734](PyO3/maturin#734) * Add support for x86_64 Haiku in [#735](PyO3/maturin#735) * Fix undefined auditwheel policy panic in [#740](PyO3/maturin#740) * Fix sdist upload for packages where the pkgname contains multiple underscores in [#741](PyO3/maturin#741) * Implement auditwheel repair with patchelf in [#742](PyO3/maturin#742) * Add `Cargo.lock` to sdist when `--locked` or `--frozen` specified in [#749](PyO3/maturin#749) * Infer readme file if not specified in [#751](PyO3/maturin#751) * Add a `maturin init` command as a companion to `maturin new` in [#719](PyO3/maturin#719) * Don't package non-path-dep crates in sdist for workspaces in [#720](PyO3/maturin#720) * Build release packages with `password-storage` feature in [#725](PyO3/maturin#725) * Add support for x86_64 DargonFly BSD in [#727](PyO3/maturin#727) * Add a Python import hook in [#729](PyO3/maturin#729) * Allow pip warnings in `maturin develop` command in [#732](PyO3/maturin#732)

polarathene · 2026-01-28T03:06:49Z

patch dylib's soname and rpath is done with patchelf which means user needs to install it beforehand.

Would be great if there is a pure Rust version but unfortunately I don't have the expertise to do that.

Just to let you know that appears to be supported by the elb crate now: igankevich/elb#6 (comment)

EDIT: Nevermind. I raised an issue to track this, but realized there's a PR opened recently which contributes a change to use the arwen crate instead.

konstin · 2026-01-28T08:12:25Z

The real difficulty is to scale the testing to ensure our patchelf replacement is similarly reliable, I've been finding panics and other bugs.

messense requested a review from konstin December 12, 2021 13:03

messense force-pushed the patchelf branch from fdb7ee3 to c7f7c8c Compare December 12, 2021 13:18

konstin approved these changes Dec 14, 2021

View reviewed changes

messense force-pushed the patchelf branch from 9f79754 to 741b548 Compare December 15, 2021 02:13

messense commented Dec 16, 2021

View reviewed changes

Comment thread src/build_context.rs Outdated

messense force-pushed the patchelf branch from 8e564ab to 277bdfd Compare December 19, 2021 03:56

messense added 4 commits December 19, 2021 11:58

Add a patchelf mod which calls out to the patchelf utility

68314ed

https://github.com/NixOS/patchelf

Split PlatformTagValidationError into more specified ones

27ee336

Implement auditwheel repair

4e5b2e4

Use lddtree

28262ec

messense force-pushed the patchelf branch from 277bdfd to f128586 Compare December 19, 2021 04:01

messense and others added 7 commits December 19, 2021 17:56

Fix a clippy warning

3913e3a

No repair support for cross compiled wheel yet

436ee3b

Apply suggestions from code review

e84fe76

Co-authored-by: konstin <konstin@mailbox.org>

Use Python 3.10.0 for test

51ce673

Update lddtree to 0.1.4

a534874

Update changelog for auditwheel repair

aa25d34

Preserving existing rpath entries

1af539a

messense force-pushed the patchelf branch from f128586 to 1af539a Compare December 19, 2021 12:42

messense merged commit fb112e3 into PyO3:main Dec 19, 2021

messense deleted the patchelf branch December 19, 2021 13:05

Peddle mentioned this pull request Jan 7, 2022

Need to either update docs or update installer to ensure users have patchelf #766

Closed

messense mentioned this pull request Oct 30, 2022

Configure bors #1214

Merged

polarathene mentioned this pull request Oct 31, 2025

Any plans for parity with patchelf? igankevich/elb#6

Closed

polarathene mentioned this pull request Jan 28, 2026

Consider using native Rust instead of patchelf dep #2952

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Implement auditwheel repair with patchelf#742

Implement auditwheel repair with patchelf#742
messense merged 11 commits into
PyO3:mainfrom
messense:patchelf

messense commented Dec 12, 2021

Uh oh!

netlify Bot commented Dec 12, 2021 •

edited

Loading

Uh oh!

konstin left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

konstin Dec 14, 2021

Uh oh!

messense Dec 15, 2021 •

edited

Loading

Uh oh!

messense commented Dec 15, 2021 •

edited

Loading

Uh oh!

messense commented Dec 15, 2021

Uh oh!

messense commented Dec 15, 2021

Uh oh!

Uh oh!

polarathene commented Jan 28, 2026 •

edited

Loading

Uh oh!

konstin commented Jan 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

messense commented Dec 12, 2021

Uh oh!

netlify Bot commented Dec 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

konstin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

konstin Dec 14, 2021

Choose a reason for hiding this comment

Uh oh!

messense Dec 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

messense commented Dec 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

messense commented Dec 15, 2021

Uh oh!

messense commented Dec 15, 2021

Uh oh!

Uh oh!

polarathene commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

konstin commented Jan 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

netlify Bot commented Dec 12, 2021 •

edited

Loading

messense Dec 15, 2021 •

edited

Loading

messense commented Dec 15, 2021 •

edited

Loading

polarathene commented Jan 28, 2026 •

edited

Loading