Skip to content

ci(tests): cut redundant test work and keep the font worker off the database layer - #9893

Open
Pfannkuchensack wants to merge 4 commits into
invoke-ai:mainfrom
Pfannkuchensack:ci/more-test-speed
Open

Pfannkuchensack wants to merge 4 commits into
invoke-ai:mainfrom
Pfannkuchensack:ci/more-test-speed

Conversation

@Pfannkuchensack

@Pfannkuchensack Pfannkuchensack commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Summary

This follows #9859. On current main, the files with the most windows CI time were video splice tests, the font service tests and a few duplicate or vacuous checks. Font and video tests now spend most of their time starting worker processes rather than doing work. This PR removes the parts that cost time without adding coverage, plus one import cost that also affects the running app.

  • The font worker no longer loads the database layer. fonts_worker is spawned once per font parse and once per variable-font instance. Its docstring says it stays light, but it imported its helpers from fonts_default, which pulled in SQLAlchemy and config.
    • The byte-level helpers now live in fonts_parsing, and the error hierarchy and supported extensions in fonts_common.
    • The worker's import time drops from 1.1–1.3 s to ~0.25 s per spawn. This also speeds up font uploads in the app.
    • fonts_default no longer imports fontTools.
    • A fresh-interpreter guard test asserts the worker loads neither the database layer nor config.
  • The SIGINT subprocess test is skipped on windows. There os.kill(pid, SIGINT) is TerminateProcess, so the test passed even with run_node swallowing BaseException. The in-process test_run_node_propagates_keyboard_interrupt still catches that on windows.
  • The duplicate UMAP smoke test is removed. test_projection.py already runs real seeded UMAP and DBSCAN fits on every platform. test_numba_jit_is_active stays.
  • Three shape-only VAE tiling sweeps are marked platform_independent, like the sweeps already marked in test_vae_tiling_scope.py.
  • The event-loop heartbeat test is deterministic. It used to assert a 50 ms wall-clock threshold against a 100 ms fake delay. It now holds the fake threadpool on an asyncio.Event and asserts that the request is still pending.
  • Video concat tests do less repeated work.
    • The short-remainder test takes its first-clip lengths from the actual keyframe position, as 65–67 instead of 65–69. It asserts that precondition, and it asserts the splice point in the log.
    • The two repeated-extension tests ran the same three-round chain and are merged into one. It keeps every assertion of both.

Measured locally (CPU only, 8 xdist workers), worker time on the files present in both runs:

before after
whole suite 1987 s 1751 s (−12 %)
windows selection (not slow and not platform_independent) 1460 s 1281 s
test_fonts_service.py 73 s 20 s
test_video_concat_splice.py (alone) 129 s 107 s
test_umap_import.py 27 s ~0 s
SIGINT subprocess test (windows) 24 s skipped

Related Issues / Discussions

Follows #9859.

QA Instructions

  • The full suite passes with uv run --no-sync pytest -n 8: 13755 passed.
  • The font service and router tests, the video upload limit tests and routers/test_utilities.py passed twice in a row (109 passed).
  • Import measurements:
    • A fresh-interpreter -X importtime of fonts_worker drops from 1.1–1.3 s to ~0.25 s.
    • invokeai.app.services.shared.database and fontTools are no longer imported by the worker and by fonts_default, respectively.
  • Mutation checks:
    • SIGINT test on windows: with run_node swallowing BaseException, the subprocess test passed and only the in-process test failed.
    • Heartbeat test: it fails when authentication runs on the loop, and when the fake threadpool returns without waiting.
    • Short-remainder test: it fails under the combined mutation that also fails the old version (MIN_ENCODED_FRAMES=1 with the post-join timeline check disabled). With MIN_ENCODED_FRAMES=1 alone it now fails too, which the old version missed.
    • Merged repeated-extension test: it fails with _plan_splice returning None. Of the old pair, the soundtrack test passed in that case.
  • ruff check and ruff format --check are clean on all changed files.
  • Not run here: the macOS and linux CI runners. The win32 skip condition was checked by reading only.

Review

  • An earlier draft left SUPPORTED_FONT_EXTENSIONS in fonts_parsing, so the API process imported fontTools just for that constant. It now lives in fonts_common.
  • Accepted coverage change: the merged repeated-extension test runs with audio, so a silent concat-into-concat chain is no longer exercised. A single silent join and a concat-produced first clip with audio both still are.
  • No material findings remain. The video node's remaining cost is its own per-call decode and probe worker spawns, up to five per concat. Reducing that would be a production change and is outside this PR.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Meaningful regression coverage added / updated where needed; obsolete tests/code removed
  • Persisted-state and API changes include required migrations / compatibility validation
  • Relevant performance/efficiency opportunities considered; material claims have evidence
  • Material review findings resolved and relevant checks rerun
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

…atabase layer

The font worker imports its byte helpers from fonts_parsing instead of fonts_default, dropping SQLAlchemy and config from every spawn.
Skips the windows-vacuous SIGINT subprocess test, removes the duplicate UMAP smoke, marks shape-only VAE sweeps platform_independent.
Makes the upload heartbeat test deterministic and derives the concat short-remainder lengths from the keyframe position.
@github-actions github-actions Bot added api python PRs that change python files services PRs that change app services python-tests PRs that change python tests labels Oct 16, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api python PRs that change python files python-tests PRs that change python tests services PRs that change app services

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants