[docs-infra] Move type formatting into LS worker, parallelize validate#1317
Draft
[docs-infra] Move type formatting into LS worker, parallelize validate#1317
Conversation
Three performance changes that reduce validate --types runtime by ~5.4×:
1. In-worker formatting: formatComponentData/formatHookData/etc now run
inside the LS worker thread (processTypes.ts) instead of the main
validate-worker thread. This avoids serializing raw allTypes AST nodes
(45–50 MB per chart component) across the Unix socket — only the compact
formatted TypesMeta (~500 KB) crosses the wire.
2. Parallel LS workers: WorkerThreadTypesProcessor now runs processTypes
directly in-thread instead of routing through a single shared socket
server. Each validate worker gets its own TS language service via the
globalThis singleton in createOptimizedProgram. N workers = N parallel
LS instances. Worker count capped at 4 with staggered spawns to avoid
Node's BuiltinLoader rwlock contention on macOS.
3. Bug fixes:
- waitForSocketFile: replaced fs.watch with polling on Unix — macOS
FSEvents unreliably delivers filename in watch callbacks, causing
30s timeouts.
- removePrefixFromHighlightedNodes: recurse into nested elements
instead of breaking, fixing "type _ = " prefix leaking into output.
Benchmarked on 107 chart files (mui-x):
Before: 1313s wall, 3479s CPU, 299% avg
After: 245s wall, 705s CPU, 306% avg
Deploy previewhttps://deploy-preview-1317--mui-internal.netlify.app/ Bundle size
PerformanceTotal duration: 16.65 ms 🔺+2.47 ms(+17.4%) | Renders: 4 (+0) | Paint: 72.87 ms +3.75 ms(+5.4%)
Check out the code infra dashboard for more information about this PR. |
JCQuintas
added
type: enhancement
github-actions
Bot
added
the
PR: out-of-date
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Performance changes that reduce
validate --typesruntime by 2.3–5.4× depending on available cores.1. In-worker formatting (2.3× alone)
formatComponentData/formatHookData/formatFunctionData/formatClassData/formatRawDatanow run inside the LS worker thread (processTypes.ts) instead of the main validate-worker thread.Why: The previous architecture shipped raw
allTypesAST nodes across a Unix socket. Each chart component produced 45–50 MB of serialized data per file. With v8.serialize at ~2 MB/s effective throughput, IPC transport alone consumed 12–19 seconds per file — 98% of total runtime.Now only the compact formatted
TypesMeta[](~500 KB) crosses the wire. Payload reduction: ~100×.2. Parallel LS workers (additional 2.3×)
WorkerThreadTypesProcessornow runsprocessTypesdirectly in-thread instead of routing through a single shared socket server. Each validate worker gets its own TypeScript language service via theglobalThissingleton increateOptimizedProgram. N workers = N parallel LS instances.Worker count uses the existing
availableParallelism() - 1(unchanged from before).3. Bug fixes
waitForSocketFile: Replacedfs.watchwith polling on Unix. macOS FSEvents unreliably deliversfilenamein watch callbacks, causing 30s timeouts on every file.removePrefixFromHighlightedNodes: Recurse into nested highlight<span>elements instead of breaking. Fixestype _ =prefix leaking into rendered output when the highlighter produces deeply nested token spans.Benchmark (107 chart files, mui-x)
RAM: ~1 GB per LS worker instance.