Skip to content
119 changes: 118 additions & 1 deletion peps/pep-0803.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
PEP: 803
Title: "abi3t": Stable ABI for Free-Threaded Builds
Author: Petr Viktorin <encukou@gmail.com>
Author: Petr Viktorin <encukou@gmail.com>, Nathan Goldbaum <nathan.goldbaum@gmail.com>
Discussions-To: https://discuss.python.org/t/106181
Status: Draft
Type: Standards Track
Expand Down Expand Up @@ -81,6 +81,123 @@ This PEP proposes additional API limitations, as required to be compatible
with both GIL-enabled and free-threaded builds of CPython.


Ecosystem maintainers want decreased maintenance burden
-------------------------------------------------------

A major advantage of the limited API and stable ABI wheels is that new
Python versions are supported on the day of release. Without stable ABI
wheels, maintainers are left with a choice between closely following CPython
releases and producing wheels during CPython beta periods or dealing with
inevitable user requests for support for new CPython versions.


Cryptography
^^^^^^^^^^^^

The cryptography project shipped 48 wheel files with their `most recent release
<https://pypi.org/project/cryptography/46.0.5/#files>`_. Cryptography is
somewhat unusual in that they ship 14 wheels each for both the ``cp38`` and
``cp311`` stable ABIs to enable optimizations available in newer limited API
versions. They also ship 14 additional ``cp314t`` wheels and 6 wheels for
pypy. If there is no free-threaded stable ABI, then with Python 3.15,
cryptography will be using roughtly the same amount of space on PyPI to support
Comment thread
ngoldbaum marked this conversation as resolved.
Outdated
two versions of the free-threaded build as *all* non-EOL versions of the
GIL-enabled build.

Cryptography maintainer Alex Gaynor `expressed a desire
<https://discuss.python.org/t/making-pyobject-opaque-in-the-limited-api/77206/2>`_
on discourse for a free-threaded stable ABI:
Comment thread
ngoldbaum marked this conversation as resolved.
Outdated

Just to state this explicitly from the PyCA maintainers perspective, as
long as we have O(1) builds, that’s ok. What we can’t/won’t do is O(n)
where we need new builds for every Python release.

When one of the PEP authors asked him in the ``#pyca`` Libera IRC channel, he
said:

One other thing I'll note that's *really* valuable about ``abi3`` is that it
means our old wheels keep working for new Python versions. If we have
per-Python release wheels, we have to do a bunch of work at various points
in the python release cycle (including potentially backport releases to add
new wheels, if we're not otherwise planning a release at that time).

As maintainers, we *really* like to structure our work to avoid being "on
the clock" like that.


moocore
^^^^^^^

The `moocore project ships <https://pypi.org/project/moocore/0.2.0/#files>`_
seven ``abi3`` wheels. It has a much smaller userbase than cryptography and has
a smaller appetite for build complexity. When the topic of adding support for
the free-threaded build `came up on the moocore issue tracker
<https://github.com/multi-objective/moocore/issues/37>`_, maintainer Manuel
López-Ibáñez let the person reporting the issue know that:

The free-threading versions are still experimental so each new
free-threading python version will need its own wheels.

Moreover, since free-threading is still experimental and not the default,
there may be bugs that only the Python developers can fix. I would expect
most users of moocore and pymoo to use the default non-free-threaded version
of Python 3.14.

[...]

**I don't want to build for 3.14 free-threading unless you really need it.**

Later, after discovering the tracking issue for supporting the limited API on the
free-threaded build, he commented:

By the way, python/cpython#111506 is about extending the stable ABI to
support free-threaded Python. If they do that, then the builds of moocore
will work in both classical and free-threaded Python versions, without
needing to build new wheels for each Python version.

[...]

I will revisit this again once Python 3.15 is released. Hopefully the ABI
will be stable (or even better, free-threading will be the default).

These two libraries -- cryptography and moocore -- are on opposite sides of the
maintainer bandwidth spectrum, but both would still really like to have a stable
ABI on the free-threaded build they can use to ship wheels and reduce their
maintenance burden.


Bindings Generators
-------------------
Comment thread
ngoldbaum marked this conversation as resolved.
Outdated

Both moocore and cryptography use bindings generators to interface with the C
API. Cryptography uses PyO3 and CFFI while moocore uses only CFFI. Both CFFI and
PyO3 already handle all the details of abstracting over the C API to enable
different build configurations and there is no need to laboriously port
extension types to use APIs that are only available on one build or another.

Using bindings generators will enable these projects to quickly adopt the new
stable ABI. Initial testing using the experimental ``_Py_OPAQUE_PYOBJECT`` flag
defined in CPython's ``main`` branch, indicates that PyO3, CFFI, and Cython will
all work with PEP 803 using packaging tools that have been patched to account
for an ``abi3.abi3t`` tag.

Cython maintainer David Woods `says
<https://github.com/cython/cython/issues/7399#issuecomment-3947605797>`_:

Cython is doesn't have huge problems with the number of wheels we distribute
Comment thread
ngoldbaum marked this conversation as resolved.
Outdated
because ultimately it works fine as pure-Python. We do distribute wheels for
a few of the smaller platforms as Stable ABI wheels but that's more
"dogfooding" than because we actually need to. So I'm adding this in
anticipation that other people will find it useful rather than because I
will.

I do remain slightly concerned that the performance trade-offs for this will
turn out to be too much for many Cython users (it's possible that the
trade-off may be different for other binding tools). That's not a huge
disaster since we're not getting rid of the regular compilation mode so
people are free to pick their own personal trade-offs.


Rationale
=========

Expand Down
Loading