Skip to content

Run doctests with doctest, not cabal-doctest#11793

Open
philderbeast wants to merge 1 commit intohaskell:masterfrom
cabalism:fix/doctest-ghc-9.12
Open

Run doctests with doctest, not cabal-doctest#11793
philderbeast wants to merge 1 commit intohaskell:masterfrom
cabalism:fix/doctest-ghc-9.12

Conversation

@philderbeast
Copy link
Copy Markdown
Collaborator

@philderbeast philderbeast commented May 6, 2026

Fixes #11493.

  • Uses the recommended way of running doctests with the doctest tool, not the way using the experimental cabal-doctest external command.
  • Runs cabal repl in quiet mode so that we only see the doctests summary output.
  • Adds a -Wno-inconsistent-flags warning suppression that only seems to be triggered by Cabal-syntax.

Note


  • Patches conform to the coding conventions.
  • Is this a PR that fixes CI? If so, it will need to be backported to older cabal release branches (ask maintainers for directions).

@ulysses4ever
Copy link
Copy Markdown
Collaborator

ulysses4ever commented May 6, 2026

there's a lot in here...

One thing I worry about is making the Makefile more complicated with generating targets ($(addprefix...). I prefer the previous setup, and I'd not want to maintain this sort of metaprogramming in Makefiles...

As for switching back to cabal repl, what's your motivation for that? I quite like the new way because the command line is much cleaner. Also, @sol would be welcome to opine on cabal doctest vs cabal repl -w doctest these days.

@philderbeast
Copy link
Copy Markdown
Collaborator Author

As for switching back to cabal repl, what's your motivation for that? I quite like the new way because the command line is much cleaner.

The motivation is fixing #11493. I agree, the cabal-doctest external command is cleaner but experimental and doesn't work for us with ghc-9.12 or ghc-9.14.

@philderbeast
Copy link
Copy Markdown
Collaborator Author

One thing I worry about is making the Makefile more complicated with generating targets ($(addprefix...). I prefer the previous setup, and I'd not want to maintain this sort of metaprogramming in Makefiles...

We already have variables with lists in the makefile:

cabal/Makefile

Lines 28 to 50 in e6ff0cc

# NOTE: Keep this in sync with `.github/workflows/format.yml`.
FORMAT_DIRS := \
Cabal \
Cabal-syntax \
cabal-install \
cabal-testsuite/src \
cabal-validate \
Cabal-tests/exes
FORMAT_DIRS_TODO := \
Cabal-QuickCheck \
Cabal-described \
Cabal-hooks \
Cabal-tests \
Cabal-tree-diff \
bootstrap \
buildinfo-reference-generator \
cabal-benchmarks \
cabal-dev-scripts \
cabal-install-solver \
cabal-testsuite/main \
cabal-testsuite/static \
solver-benchmarks

Instead, it'd be good to have a generic target e.g.

doctest-%:
    cd $* && $(DOCTEST)

-- doctest-% suggestion

I included your above suggestion but without the cd as there isn't a need to change directory.

I also added convenience doctest-PACKAGENAME targets to help speed things up when someone wants to use the makefile to run doctests on one package. I reuse DOCTEST_TARGETS in the recipe of doctest-PACKAGENAME and for the prerequisites of the doctest phony target.

DOCTEST_PACKAGES := \
  Cabal \
  Cabal-described \
  Cabal-syntax \
  cabal-install \
  cabal-install-solver \
  cabal-testsuite

DOCTEST_TARGETS := $(addprefix doctest-, $(DOCTEST_PACKAGES))

doctest-%: ## Run doctests for a specific package.
	$(DOCTEST) $*

doctest-PACKAGENAME: ## Run doctests for a single package (replace PACKAGENAME).
	@echo 'Please use one of the following targets:'
	@printf "%s\n" $(DOCTEST_TARGETS)

.PHONY: doctest
doctest: ## Run doctests in all packages.
doctest: $(DOCTEST_TARGETS)

@philderbeast
Copy link
Copy Markdown
Collaborator Author

I see doctest-0.25.0 was just released 2026-05-03. Maybe that fixes cabal-doctest?

@ulysses4ever
Copy link
Copy Markdown
Collaborator

Ah, got you. It was hard to discern given the slew of changes... We really should report it upstream: do you think you could do that?

@ulysses4ever
Copy link
Copy Markdown
Collaborator

I included your above suggestion but without the cd as there isn't a need to change directory.

I think one cd is much better than adding one $(addprefix...)

@sol
Copy link
Copy Markdown
Member

sol commented May 6, 2026

@philderbeast what is the reason you pinned doctest to 0.24.3?

In general, I would prefer to stay with cabal doctest. If there are bugs, then I think the right approach is to fix these.

@ulysses4ever thanks for looping me in.

@philderbeast
Copy link
Copy Markdown
Collaborator Author

what is the reason you pinned doctest to 0.24.3

The doctest-0.25.0 release breaks the doctests as written in this PR with ghc-9.14.1. Until I work out why, I'm pinning it on this PR.

@philderbeast
Copy link
Copy Markdown
Collaborator Author

there's a lot in here...

@ulysses4ever I've split this pull request. Please see the note in the description of this pull request for details.

@philderbeast philderbeast force-pushed the fix/doctest-ghc-9.12 branch 2 times, most recently from 9ba2e2f to eeb092e Compare May 9, 2026 11:32
@philderbeast philderbeast force-pushed the fix/doctest-ghc-9.12 branch from eeb092e to 7ef40f3 Compare May 10, 2026 00:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

doctests with ghc-9.12.2 can't parse

3 participants