Docutils 1.0 development changes break native doctests and table widths

Describe the bug

Hej,
this is the last of the the CI failures on #14677. I noticed that two node representation changes in the Docutils 1.0 development version break Sphinx's handling of native reStructuredText doctests and table column widths.

Docutils now parses a native doctest as a literal_block with the classes code, pycon, and doctest, instead of as a doctest_block.
Sphinx only recognizes the old node type when it assigns the highlighting language, collects automatic doctests, and moves an indented native doctest out of a block quote.
The new node can therefore lack a language attribute, be skipped by the doctest builder, and remain inside a <blockquote> in HTML.

Docutils also represents proportional colwidth attributes as strings.
Sphinx's text, LaTeX, and Texinfo writers perform integer arithmetic on these values, while autosummary still creates integer attributes that the Docutils development XML writer treats as strings. This produces errors in both directions at the Sphinx/Docutils boundary.

How to Reproduce

Run:

uv sync
uv run --no-sync --with \
  'docutils @ git+https://repo.or.cz/docutils.git@335c924f838ac9180231cf2164f301cc6f6b4ca1#subdirectory=docutils' \
  pytest -q --tb=short \
  tests/test_builders/test_build_text.py::test_lineblock \
  tests/test_extensions/test_ext_doctest.py::test_highlight_language_default \
  tests/test_ext_autosummary/test_ext_autosummary.py::test_escaping

The run produces three failed tests and exit code 1:

the text writer failed while combining integer and string column width
the doctest highlighting test failed with KeyError: 'language'
the XML writer failed while handling autosummary's integer colwidth

The same three tests pass with the locked Docutils 0.22.3.

Environment Information

Platform:              linux; (Linux-7.0.0-31-generic-x86_64-with-glibc2.43)
Python version:        3.13.13 (main, Apr  7 2026, 20:49:46) [Clang 22.1.1 ])
Python implementation: CPython
Sphinx version:        9.1.1+/e44a40eb2
Docutils version:      1.0b1.dev
Jinja2 version:        3.1.6
Pygments version:      2.19.2

Sphinx extensions

['sphinx.ext.autosummary', 'sphinx.ext.doctest']

Additional context

The AI under #14673 already proposes changes to the same compatibility surfaces, but it misses the transform that moves indented native doctests out of block quotes.

@jdillard

Hej, @jdillard I was checking the upstream Docutils 1.0 plans, while making #14689, with #14681, we currently cover the current failures, but there are some additions planned that could still affect sphinx, I made this small table of what I could find in the Docutils changes that affect Sphinx (with the help of AI to aggregate all the open already issues and PRs and summarize upstream changes):

Docutils change	Upstream status	Effect on Sphinx	Sphinx issue	Sphinx PRs	Review and merge status
Native doctests changed from `doctest_block` to classified `literal_block` nodes; table `colwidth` values changed from integers to strings	Landed	Breaks doctest highlighting and collection, movement of indented doctests, autosummary XML output, and table rendering in the text, LaTeX, and Texinfo writers	#14680, open	#14681 (AI: #14673 and #14685)	None approved or merged
RFC base changed to `https://www.rfc-editor.org/info/`, and generated RFC paths no longer end in `.html`	Landed	Sphinx defines its own base URL but uses Docutils' RFC path template. This produces changed URLs and four failing assertions	#14659, open	#14689, #14662	None approved or merged
`DanglingReferences` split into `MatchReferences`, `ReportDanglingReferences`, and `ReportUnreferencedTargets`	Landed	Sphinx removes only the old aggregate transform and adds its own subclass of it. The new transforms therefore remain active alongside Sphinx's old transform. Docutils documents `ReportUnreferencedTargets` as unsuitable for Sphinx's multi-document model	None	None	Unaddressed , Cant be addressed really as of now, as the depreciation warning appears in 1.0 but the replacements aren't available until 1.0
Figures without a caption or legend now produce a warning	Landed	Broke warning-as-error testing because two Sphinx test fixtures contained invalid figures	#14508 closed	#14540	#14540 was approved and merged.
Move HTML5 attribution markup after its blockquote	Announced for 1.0, not landed	Sphinx overrides blockquote rendering to add an inner `<div>` but inherits Docutils' attribution handling. The upstream change may alter Sphinx's HTML structure or conflict with its wrapper markup and themes	None	None	Awaiting upstream implementation and Sphinx regression coverage
Change the manpage `text_references` default to `False`	Announced for 1.0, not landed	Docutils installs an instance-level reference visitor based on this setting, shadowing Sphinx's `visit_reference`. The new default will select different Docutils rendering unless Sphinx controls dispatch itself	#14427, open	#14436	None approved or merged. It likely covers the future default change but must be retested after that change lands
Use `SOURCE_DATE_EPOCH` for the Docutils `date` directive and `datestamp` setting	Announced for 1.0, not landed	Sphinx uses the Docutils directive and already honors `SOURCE_DATE_EPOCH` elsewhere. The change is likely desirable but needs a Sphinx regression test	None	None	Awaiting upstream implementation and Sphinx regression coverage
Other document-tree and API changes relevant to third-party extensions	Some landed	Sphinx core has no reproduced failure. Extensions may be affected if they expect eager IDs, inspect old inline-target or doctest nodes, assume integer column widths, subclass `DanglingReferences`, or import removed Docutils APIs	None	None	Not a Sphinx-core blocker without a reproducer. Relevant migration notes may be needed before enabling Docutils 1.0

The last one is more indirect, as it doesnt affect Sphinx with its current test suite, but may other extensions.

Should maybe a tracking issue be opened to track Docutils 1.0 compatibility, as it seem they are currently landing one by one in dev so one can keep track of what is already landed, if it is addressed etc.

@jayaddison

I added the PR by @jayaddison to the table (#14696 provides a partial fix to #14680 and doesnt fix #14659, adjusting tests to pass for now.)

Docutils change	Upstream status	Effect on Sphinx	Sphinx issue	Sphinx PRs	Review and merge status
Native doctests changed from `doctest_block` to classified `literal_block` nodes; table `colwidth` values changed from integers to strings	Landed	Breaks doctest highlighting and collection, movement of indented doctests, autosummary XML output, and table rendering in the text, LaTeX, and Texinfo writers	#14680, open	#14681, #14696 (contains AI #14673) partial, (AI: #14685)	None approved or merged
RFC base changed to `https://www.rfc-editor.org/info/`, and generated RFC paths no longer end in `.html`	Landed	Sphinx defines its own base URL but uses Docutils' RFC path template. This produces changed URLs and four failing assertions	#14659, open	#14689, #14662, #14696 (test accommodation only)	None approved or merged
`DanglingReferences` split into `MatchReferences`, `ReportDanglingReferences`, and `ReportUnreferencedTargets`	Landed	Sphinx removes only the old aggregate transform and adds its own subclass of it. The new transforms therefore remain active alongside Sphinx's old transform. Docutils documents `ReportUnreferencedTargets` as unsuitable for Sphinx's multi-document model	None	None	Unaddressed , Cant be addressed really as of now, as the depreciation warning appears in 1.0 but the replacements aren't available until 1.0. Would need feature dependent handling and lifecycle testing.
Figures without a caption or legend now produce a warning	Landed	Broke warning-as-error testing because two Sphinx test fixtures contained invalid figures	#14508 closed	#14540	#14540 was approved and merged.
Move HTML5 attribution markup after its blockquote	Announced for 1.0, not landed	Sphinx overrides blockquote rendering to add an inner `<div>` but inherits Docutils' attribution handling. The upstream change may alter Sphinx's HTML structure or conflict with its wrapper markup and themes	None	None	Awaiting upstream implementation and Sphinx regression coverage
Change the manpage `text_references` default to `False`	Announced for 1.0, not landed	Docutils installs an instance-level reference visitor based on this setting, shadowing Sphinx's `visit_reference`. The new default will select different Docutils rendering unless Sphinx controls dispatch itself	#14427, open	#14436	None approved or merged. It likely covers the future default change but must be retested after that change lands
Use `SOURCE_DATE_EPOCH` for the Docutils `date` directive and `datestamp` setting	Announced for 1.0, not landed	Sphinx uses the Docutils directive and already honors `SOURCE_DATE_EPOCH` elsewhere. The change is likely desirable but needs a Sphinx regression test	None	None	Awaiting upstream implementation and Sphinx regression coverage
Other document-tree and API changes relevant to third-party extensions	Some landed	Sphinx core has no reproduced failure. Extensions may be affected if they expect eager IDs, inspect old inline-target or doctest nodes, assume integer column widths, subclass `DanglingReferences`, or import removed Docutils APIs	None	None	Not a Sphinx-core blocker without a reproducer. Relevant migration notes may be needed before enabling Docutils 1.0

Support of the environment variable SOURCE_DATE_EPOCH is now done (Docutils commit r10402).

@gmilde

Thanks @gmilde , I will get at it.

Docutils change	Upstream status	Effect on Sphinx	Sphinx issue	Sphinx PRs	Review and merge status
Native doctests changed from `doctest_block` to classified `literal_block` nodes; table `colwidth` values changed from integers to strings	Landed	Breaks doctest highlighting and collection, movement of indented doctests, autosummary XML output, and table rendering in the text, LaTeX, and Texinfo writers	#14680, open	#14681, #14696 (contains AI #14673) partial, (AI: #14685)	None approved or merged
RFC base changed to `https://www.rfc-editor.org/info/`, and generated RFC paths no longer end in `.html`	Landed	Sphinx defines its own base URL but uses Docutils' RFC path template. This produces changed URLs and four failing assertions	#14659, open	#14689, #14662, #14696 (test accommodation only)	None approved or merged
`DanglingReferences` split into `MatchReferences`, `ReportDanglingReferences`, and `ReportUnreferencedTargets`	Landed	Sphinx removes only the old aggregate transform and adds its own subclass of it. The new transforms therefore remain active alongside Sphinx's old transform. Docutils documents `ReportUnreferencedTargets` as unsuitable for Sphinx's multi-document model	None	None	Unaddressed , Cant be addressed really as of now, as the depreciation warning appears in 1.0 but the replacements aren't available until 1.0. Would need feature dependent handling and lifecycle testing.
Figures without a caption or legend now produce a warning	Landed	Broke warning-as-error testing because two Sphinx test fixtures contained invalid figures	#14508 closed	#14540	#14540 was approved and merged.
Move HTML5 attribution markup after its blockquote	Announced for 1.0, not landed	Sphinx overrides blockquote rendering to add an inner `<div>` but inherits Docutils' attribution handling. The upstream change may alter Sphinx's HTML structure or conflict with its wrapper markup and themes	None	None	Awaiting upstream implementation and Sphinx regression coverage
Change the manpage `text_references` default to `False`	Announced for 1.0, not landed	Docutils installs an instance-level reference visitor based on this setting, shadowing Sphinx's `visit_reference`. The new default will select different Docutils rendering unless Sphinx controls dispatch itself	#14427, open	#14436	None approved or merged. It likely covers the future default change but must be retested after that change lands
Use `SOURCE_DATE_EPOCH` for the Docutils `date` directive and `datestamp` setting	Landed	Sphinx uses the Docutils directive and already honors `SOURCE_DATE_EPOCH` elsewhere. The change is likely desirable but needs a Sphinx regression test	None	None	None approved or merged
Other document-tree and API changes relevant to third-party extensions	Some landed	Sphinx core has no reproduced failure. Extensions may be affected if they expect eager IDs, inspect old inline-target or doctest nodes, assume integer column widths, subclass `DanglingReferences`, or import removed Docutils APIs	None	None	Not a Sphinx-core blocker without a reproducer. Relevant migration notes may be needed before enabling Docutils 1.0

"Move HTML5 attribution markup after its blockquote" is now complete.

@gmilde

Thanks @gmilde , sorry for the delay, it should be up to date now again.

Docutils change	Upstream status	Effect on Sphinx	Sphinx issue	Sphinx PRs	Review and merge status
Native doctests changed from `doctest_block` to classified `literal_block` nodes; table `colwidth` values changed from integers to strings	Landed	Breaks doctest highlighting and collection, movement of indented doctests, autosummary XML output, and table rendering in the text, LaTeX, and Texinfo writers	#14680, open	#14681, #14696 (contains AI #14673) partial, (AI: #14685)	Core compatibility handling committed in `d19cbb5`. It provides preliminary support, a clean cross version behavior is outstanding.
RFC base changed to `https://www.rfc-editor.org/info/`, and generated RFC paths no longer end in `.html`	Landed	Sphinx defines its own base URL but uses Docutils' RFC path template. This produces changed URLs and four failing assertions	#14659, open	#14689, #14662, #14696 (test accommodation only)	Test expectations adjusted in `d19cbb5`. The underlying URL construction issue remains open, with currently differences between installed Docutils versions.
`DanglingReferences` split into `MatchReferences`, `ReportDanglingReferences`, and `ReportUnreferencedTargets`	Landed	Sphinx removes only the old aggregate transform and adds its own subclass of it. The new transforms therefore remain active alongside Sphinx's old transform. Docutils documents `ReportUnreferencedTargets` as unsuitable for Sphinx's multi-document model	None	None	Addressed in `d19cbb5`. Version-dependent transform handling is present.
Figures without a caption or legend now produce a warning	Landed	Broke warning-as-error testing because two Sphinx test fixtures contained invalid figures	#14508 closed	#14540	#14540 was approved and merged.
Move HTML5 attribution markup after its blockquote	Landed	Sphinx overrides blockquote rendering to add an inner `<div>` but inherits Docutils' attribution handling. The upstream change may alter Sphinx's HTML structure or conflict with its wrapper markup and themes	None	None	Preliminary compatibility handling and regression test committed in `d19cbb5`. Sphinx preserves its existing attribution placement inside the wrapper.
Change the manpage `text_references` default to `False`	Announced for 1.0, not landed	Docutils installs an instance-level reference visitor based on this setting, shadowing Sphinx's `visit_reference`. The new default will select different Docutils rendering unless Sphinx controls dispatch itself	#14427, open	#14436	None approved or merged. It likely covers the future default change but must be retested after that change lands
Use `SOURCE_DATE_EPOCH` for the Docutils `date` directive and `datestamp` setting	Landed	Sphinx uses the Docutils directive and already honors `SOURCE_DATE_EPOCH` elsewhere. The change is likely desirable but needs a Sphinx regression test	#14769, open	#14770	PR open, not merged.
Other document-tree and API changes relevant to third-party extensions	Some landed	Sphinx core has no reproduced failure. Extensions may be affected if they expect eager IDs, inspect old inline-target or doctest nodes, assume integer column widths, subclass `DanglingReferences`, or import removed Docutils APIs	None	None	Not a Sphinx-core blocker without a reproducer. Relevant migration notes may be needed before enabling Docutils 1.0

@AA-Turner

@AA-Turner, how would you like to proceed with the remaining Docutils 1.0 compatibility work?
d19cbb5 addresses several of the problems tracked here, so some open PRs now overlap with those changes, personally:

Fix docutils doctest table width compatibility #14681 overlaps with the doctest and table-width fixes in d19cbb5, which address Docutils 1.0 development changes break native doctests and table widths #14680. It also proposes additional regression coverage, version-independent node recognition, and stricter downstream compatible width validation.
Use canonical RFC Editor URLs independently of Docutils #14689 addresses Use canonical URL for RFCs #14659 by making the canonical RFC URLs independent of the installed Docutils version.
Use source date epoch #14770 addresses Make Docutils date substitutions honor SOURCE_DATE_EPOCH across supported versions #14769 by providing consistent SOURCE_DATE_EPOCH handling across supported Docutils versions, including compatibility handling for older releases.
Should these be updated as focused follow-ups to the committed changes, as they do normalize behavior, but also change it slightly? It would help to know which remaining differences between Docutils versions you would like Sphinx to normalize, and which should remain version-dependent as we look forward to Docutils 1.0?

Navigation Menu

Repository navigation

Docutils 1.0 development changes break native doctests and table widths #14680

Describe the bug

How to Reproduce

Environment Information

Sphinx extensions

Additional context

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Docutils 1.0 development changes break native doctests and table widths #14680

Description

Describe the bug

How to Reproduce

Environment Information

Sphinx extensions

Additional context

Activity

Darlokt commented on Sep 18, 2026

Darlokt commented on Sep 19, 2026

gmilde commented on Sep 19, 2026

Darlokt commented on Sep 19, 2026

gmilde commented on Oct 9, 2026

Darlokt commented on Oct 21, 2026

Darlokt commented on Oct 21, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions