Skip to content

Latest commit

 

History

History
550 lines (454 loc) · 43.7 KB

DEBUGGING.md

File metadata and controls

550 lines (454 loc) · 43.7 KB

Fix-wave workflow

All function bodies are ported; from here the conformance suite and the Go reference drive the work. Several agents fix different failure clusters at the same time. This is how they avoid stepping on each other.

Every reproduction and fix follows the fidelity rule. Verify the reference checkout and build the Go compiler before comparing outputs. Any shared $TSRS_WORK/bin/tsgo-ref used below must come from that verified checkout and be rebuilt when the TypeScript source pin changes.

Your worktree

$TSRS_WORK below is the directory that holds the main checkout ($TSRS_WORK/tsrs), the agent worktrees ($TSRS_WORK/wt/) and the reference binaries ($TSRS_WORK/bin/). $PRIVATE_PROJECT refers to a large private monorepo used for end-to-end checks, outside this repository.

  • You get a private git worktree $TSRS_WORK/wt/<agent-name> on branch fix/<agent-name>, created from origin/main. Work only there (never in the main checkout $TSRS_WORK/tsrs or another agent's worktree). ts-ref inside it may be a symlink to a shared reference checkout; verify it against the worktree's own Cargo.toml pin.
  • Build with your own target dir: CARGO_TARGET_DIR=$PWD/target cargo build --profile dev -p tsrs_testrunner -p tsrs_cli (dev: Cargo's default development settings with debug information disabled. Build --release only to measure performance.)

Landing changes (trunk-based, through GitHub)

Land small, often — every coherent fix, not one big drop at the end:

git add <changed-paths> && git commit -m "<area>: <what and why>"
git fetch -q origin main && git rebase origin/main        # resolve conflicts if any, keep both sides' intent
CARGO_TARGET_DIR=$PWD/target cargo check                  # must be 0 errors, 0 warnings after the rebase
cargo clippy --all-targets --config 'build.warnings="deny"'
# re-run the tests you touched, and before every push a full-suite run to check for regressions (see below)
git push -q origin HEAD:main                              # if rejected: fetch, rebase, check, push again

Never force-push, never rewrite origin/main, never commit build output. If a rebase conflict is in code you do not understand, read both versions and the Go source; do not blindly take one side.

Running tests

  • Full gate: cargo run --profile dev -p tsrs_determinism -- conformance runs every baseline in both checker modes, then the scheduling sweep (options and replay).
  • Full suite in Go history: ./target/debug/tsrs-test run --suite all -> summary table; per-class name lists in target/test-results/{pass,codes,fail,crash,timeout}.txt; summary.json has the first difference per test.
  • One test / a cluster: tsrs-test run --filter <substring|regex>, tsrs-test show <suite/name> (expected vs actual), tsrs-test crashes --top 20 (panics grouped by location).
  • Classes: pass = baseline byte-identical; codes = same (file, line, col, code) set but message text differs; fail = different diagnostics; crash; timeout.
  • No regressions: before each push, compare pass.txt against the list from before your change (comm -23 <(sort before/pass.txt) <(sort target/test-results/pass.txt) must be empty, or every lost test must be explained by an upstream change you just rebased onto). Record the new totals in your commit message.
  • Reference behavior for anything outside the suite: target/reference/tsgo, or a shared tsgo-ref built from the same verified pin. Oracle tools for scanner/parser/binder/module resolution/printer live in tools/oracle/ with binaries in $TSRS_WORK/bin/.

Fast mode and reference defaults

tsrs ports microsoft/TypeScript#64475 + #64526 (lazy member tables, tsrs_core::lazymembers, notes/lazy-members.md) and enables them with --fast, TSRS_FAST=1 or --lazyMembers. Compilation defaults to eager members, as in the pinned reference. The linter enables the fast preset by default; TSRS_FAST=0 disables it. --noLazyMembers / TSRS_LAZY_MEMBERS=0 overrides the fast preset. An explicit --fast false disables the environment preset but leaves per-feature overrides in effect. See performance options for the full preset and its diagnostic, memory, counter and cache-reuse effects.

The lazy-member master switch also enables TSRS_LAZY_TUPLES, TSRS_LAZY_COND_MAPPER, TSRS_LAZY_PROP_CACHE, TSRS_LAZY_UNMATCHED, TSRS_LAZY_EMPTY and TSRS_LAZY_INFERENCE_MAPPERS by default. Each accepts 0 to disable it; all require the master switch. TSRS_LAZY_HAS_PROP remains off unless explicitly set to 1 with lazy members enabled (implementation). The experimental shared type graph also remains off: it requires both a build with --features tsrs_core/shared-graph and TSRS_SHARED_GRAPH=1 (implementation).

The conformance gate runs Go history with eager members (TSRS_FAST=0) and canonical history with lazy members (TSRS_FAST=1); neither may lose passes. The raw harness retains single-threaded programs unless TS_TEST_PROGRAM_SINGLE_THREADED=false or tsrs-test --checkers N enables parallel checking. It still accepts TSRS_HISTORY=canonical as a legacy way to select the optimized mode. Explicit Go history/assignment overrides the fast preset's history without disabling its lazy members.

For counter comparisons against tsgo-ref, leave experiment variables unset, disable the fast preset, and match checker counts. Lazy members create fewer symbols, types and instantiations. Matching checker settings does not promise identical runtime or allocation statistics; DEFAULTS.md explains the differences. Older experiment notes may call the optimized mode "default"; read their measurements against the recorded commit.

Checker assignment (multi-checker counters)

The default uses Go's FENNEL assignment and checker history, preserving TypeScript's own diagnostic sensitivity to checker count and file order. Compare with the reference using the same count and input order. Fast mode assigns files by directory locality (--checkerAssignment locality selects the static variant, notes/mem-assignment.md). Canonical history removes selected assignment dependencies, but known cases still change diagnostics or exit status across schedules (notes/open-history-dependence.md). Multi-checker symbol/type/instantiation counters and peak memory depend on the assignment too. --checkerAssignment go (or TSRS_CHECKER_ASSIGNMENT=go) selects Go's assignment and history even in fast mode, but does not disable lazy members; use --fast false with other overrides unset for the reference defaults. The conformance, fourslash and tsc harnesses use Go history by default. TS_TEST_PROGRAM_SINGLE_THREADED=false TSRS_FAST=1 runs tsrs-test with the fast scheduler. TSRS_CHECKER_ASSIGNMENT=random:<seed> puts each file on a random checker with a random visit order in each checker. The scheduling gate requires identical diagnostics for its selected fixtures across seeds and checker counts; the upstream history-dependent cases listed above remain outside that guarantee. Single-threaded runs have no assignment. In fast mode without a named assignment the type-check pass also steals work: a checker that has run out of files takes unstarted files from the back of the busiest checker (notes/perf-checker-stealing.md). Counters can vary from run to run; naming an assignment (--checkerAssignment locality) turns stealing off and fixes the file placement and visit order. The scheduling gate checks output stability; this does not remove the known history-dependent cases. Work stealing can change assignment and visit order between identical fast-mode invocations, exposing those cases; the sequelize investigation records output variants under stealing. This is separate from varying checker counts or file order in default mode. tsrs-test --checkers N sets each test program's checker count and enables parallel programs. With work stealing enabled, each checker begins the type-check pass with the files of its queue whose static weight exceeds 1/200 of an average checker's share, heaviest first, then the rest in program order (notes/perf-checker-64.md, notes/perf-heavy-first-threshold.md); TSRS_HEAVY_SHARE_DIVISOR=<n> changes the divisor for experiments. --checkerCostCache <file> (opt-in, locality only) balances the checkers on the previous run's per-file CPU times (notes/perf-balance.md). From 16 checkers on (or 8 for programs with at least 24,000 checked files), the locality placement also counts the modules (files and their imports, library declaration files included) a group shares with each checker, then moves groups while that lowers the modules held per checker (notes/perf-clustered-assignment.md); TSRS_MODULE_AFFINITY=off restores the placement by import edges alone, and TSRS_MODULE_AFFINITY=<mu>, TSRS_MODULE_AFFINITY_GAMMA=<g> and TSRS_MODULE_AFFINITY_PASSES=<n> are for experiments. At the same checker/file thresholds, a thief also keeps stealing from the checker it stole from last while that checker has at least half the busiest one's work left; TSRS_STEAL_STICKY=0 picks the busiest checker for every file. From 8 to 15 checkers at most a quarter of the checkers take files from any one checker at a time (notes/perf-icount-assign.md); TSRS_STEAL_MAX_THIEVES=<m> sets that limit at every checker count, 0 for none.

Why a TS2590 comes and goes: TSRS_TRACE_UNION_REDUCTION

With fast mode's canonical history, a result computed while a TS2590 was reported is not cached, so every file that evaluates the type reports it, once per too-complex type at the first site that evaluated it (Checker::too_complex_since, flush_too_complex_reports, notes/open-history-dependence.md section 4). The default Go history keeps Go's caches and first-evaluation-only reports. The trace below helps explain those reports and identify which union a TS2590 is about. Other known history-dependent cases remain in fast mode.

TS2590 ("Expression produces a union type that is too complex to represent") comes from two places. removeSubtypes gives up when, after 100,000 comparisons, the comparisons per constituent so far times the union's length exceed 1,000,000, so only a union of more than 1,000 types can get it. checkCrossProductUnion gives up when a cross product would have 100,000 members or more: the union an intersection of unions expands to (X & (A | B) & (C | D) is X & A & C | X & A & D | ...), a spread of a union, a template literal over unions, a tuple with a union in a variadic element. Neither caches its failure, but what is built from the error type it returns is cached higher up, so in Go mode a checker reports the TS2590 only where it first builds the type (notes/open-history-dependence.md section 4). TSRS_TRACE_UNION_REDUCTION=1 prints three kinds of lines on stderr (crates/tsrs_checker/src/uniontrace.rs; the first two below are from the reporter's reproduction (PR #228, now testdata/regressions/union-too-complex-cross-product), the third from testdata/regressions/union-too-complex-canonical-order):

tsrs check file: checker 1 begins a.ts
tsrs cross product: checker 1 at a.ts(2,1) while checking a.ts | size 160000, too complex (TS2590) | union 400 set af528f69d0d47f2c order 984925cdeb94e613 first [...] | union 400 set f7cbe6dc7c301bb4 order c846db1e17ad27de first [...]
tsrs union reduction: checker 1 at b.ts(4,20) while checking b.ts | 1101 -> 1101 types, too complex (TS2590) | 100000 comparisons, at 100000: 91 sources begun, estimate 1208898 | set 2d3aac47bef09b0c order bffdf1259fb1b3bf removed d1fba762150c532c | first [...] last [...]
  • check file: the checker begins to check that file (statements a..b of a file: a piece of a split declaration file). One checker's lines, in order, are its history: what it had checked, and so may have built and cached, when it reached a TS2590 site. checker N is the checker's id, the same on all three kinds of lines (1 to the checker count in a CLI run). The checkers are created concurrently, so the same assignment can number them differently from run to run: compare runs by which files share a checker, not by the numbers.
  • cross product: every cross product of at least 1,000 members, too complex (TS2590) from 100,000 on, then each operand: a union as its length, its fingerprints and its first two members, another type described.
  • union reduction: every subtype reduction of a union of more than 1,000 types. -> n types is what was kept (or left when it gave up), from the cache an answer reused from an earlier reduction of the same list on that checker.

On both TS2590 lines, at is the node the error is reported at and while checking the file the checker was checking. When they differ, the error is filed against another file, and it is printed only if this checker checks that file afterwards: it is lost if this checker already handed that file's diagnostics over or another checker checks it, and never collected for a file that is not type-checked (a JSON file, a declaration file under skipLibCheck). The fingerprints are built from the types' symbols, declaration files (by base name) and positions, type arguments and literal values, not from type ids, so they are comparable between runs and checker counts. Tracing reads only what the types already hold and resolves nothing. When two runs disagree on a TS2590:

  • The same line in both, the error printed in one: it was filed against another file and lost in the other.

  • The same location with a different set: the union itself differed; the same set with a different order: its constituents were ordered differently; for a reduction, the same set and order with a different removed (or a different count at the 100,000 checkpoint): some comparisons answered differently.

  • No line at that location in the run without the error: that checker did not build the type there. Look at its check file lines: if it had checked a file before that printed the same line (same fingerprints), it reused what it cached then. Issue #218 was read this way. The TS2590 at getIntegrationModalData.test.ts(845,3) comes from the cross products that RecordMap.ts also builds; in four default-mode runs at 8 checkers the two files were on different checkers and the error was printed every time:

    run RecordMap.ts on checker error file on checker same checker, RecordMap.ts first? TS2590
    1 4 3 no yes
    2 1 3 no yes
    3 1 4 no yes
    4 4 6 no yes

    On another commit, the checker that checked RecordMap.ts and then execution.ts printed no line at execution.ts(89,23) and no error, while under --checkerAssignment go the two files were on different checkers and execution.ts's checker built the cross products itself and reported the TS2590, as tsgo does.

.types / .symbols comparisons against a cached reference

Generate reference dumps with the Go oracle (tools/oracle/project-types) built from the verified source pin. Keep cached manifests at $TSRS_WORK/project-ref-dump/<types|symbols>/manifest.<kind> outside tracked files. Record the reference commit, project revision and compiler flags with each cache; regenerate it when any changes. The measurements in the earlier project-types note used an older TypeScript pin and do not establish equivalence at the current pin.

A reference-default tsrs-test types-dump (TSRS_FAST unset or 0, with no per-feature overrides) may substitute for a Go dump only after its output has been verified against the pinned Go oracle for the same inputs and baseline kind. Keep $PRIVATE_PROJECT read-only as described below, and keep private paths and cache contents out of tracked files.

  • Quick tier: tsrs-test types-dump -p tsconfig.json --out <dir> --mode types --text none --sample <repo>/target/project-types-sample.txt (run from $PRIVATE_PROJECT), then tools/project-types-compare.mjs $TSRS_WORK/project-ref-dump/types <dir> --subset. tools/project-types-sample.mjs writes the sample list; keep it outside tracked files. The checker has fully checked the program before the walk either way, but the walk itself can create types and assign symbol ids, so a file that differs only in a sample run should be confirmed with a full run.
  • Full tier (pre-landing for checker changes that can affect printing, one at a time): the same without --sample and without --subset.

The pristine private-monorepo checkout is read-only

$PRIVATE_PROJECT belongs to a working checkout of the private monorepo. Two agents have already written into it by accident (a tsbuildinfo from a tsgo-ref run without --incremental false; a bench script's results file). Rules: tsrs emits by default like tsc, so run both tsrs and tsgo-ref there only with --noEmit --incremental false (cwd elsewhere), or with every output path (--outDir, --declarationDir, --tsBuildInfoFile) redirected into your worktree's target/ or /tmp; anything that writes (mutation testing, scripts that create files) uses the disposable clone at $TSRS_WORK/project-clone. Before you finish, run git -C <pristine root> status --short (read-only) and confirm it prints nothing.

Arena recycling checks

TSRS_ARENA_POISON=1 fills freed arena blocks, rewound ranges and released regions with poison and keeps their memory mapped. Run the recycling and language-server lifetime regressions after changes to arena ownership:

cargo build --profile dev -p tsrs_cli -p tsrs_testrunner
cargo test -p tsrs_core
cargo test -p tsrs_cli --test integration -- recycle_checkers free_leaf_files lsp_edit_resolutions

The CLI regressions exercise normal and poison modes. These checks exercise stale accesses; they do not scan the object graph for all references to freed memory.

The alloc-profile feature, heap sampler and reachability census have been removed. Older measurements in the engineering notes describe the instrumentation available at the recorded revision.

What the resident memory is: the memory split

TSRS_MEM_SPLIT=1 (any build) prints on stderr, at parse end, at check end (every checker of the type-check pass is done and its thread still alive, the moment memory peaks with many checkers) and at exit: the thread arenas' capacity, used bytes and the resident bytes below each arena's finger that were never handed out (mincore); the mimalloc heap's live blocks and the resident bytes of its pages (mi_heap_visit_blocks); the checker threads' resident stacks; and on Linux /proc/self/status and /proc/self/smaps summed into the arena reservation, thread stacks, the heap and other anonymous memory, and file-backed pages, with a one-line split arena used + arena unused resident | heap live + heap retained (in pages, outside pages) | stacks | file (crates/tsrs_core/src/memsplit.rs, notes/mem-linux-residency-32.md). TSRS_MEM_SPLIT=purge also has mimalloc purge its freed memory after check end (mi_collect(true)) and prints the split again. tools/perf/slackprobe.sh runs it on the selected GitHub-hosted runner with interleaved timed reps when the disabled .github/workflows/perf-probe.yml.bak is restored to perf-probe.yml (ubuntu-24.04 by default).

Optional build features

For tooling builds, append these arguments to cargo build --profile dev -p tsrs_cli (use --release for performance measurements). Features live on the crate that controls the instrumentation:

Tool Additional build arguments
Creation-site counters -p tsrs_checker --features tsrs_checker/site-counts
Checker work census -p tsrs_checker --features tsrs_checker/work-census
AST sizing --features tsrs_ast/ast-sizing
Shared checker graph --features tsrs_core/shared-graph

The CLI and compiler aliases for these features have been removed. Creation-site counters also live entirely in tsrs_checker; the tsrs_core/site-counts feature has been removed. Selecting tsrs_checker alongside tsrs_cli lets Cargo enable its features without a direct CLI dependency. The system-alloc CLI feature selects the system allocator instead of mimalloc.

Pointer compression is automatic on Unix. Use --features tsrs_core/plain-ptrs to disable it; --no-default-features no longer disables compression, and the compressed-ptrs feature has been removed. Non-Unix targets continue to use plain pointers.

Where checker time goes by source pattern: the work census

A function profile cannot tell which type alias or call site the time belongs to. The work census (crates/tsrs_checker/src/workcensus.rs, notes/perf-checker-algorithms.md) attributes the checker's work to source-level identities: conditional types by root alias (with distribution fan-out and never results), mapped types by declaration and key count, generic calls by callee (and repeats of the same signature + argument types), relations by target symbol and relation kind (cache hit rates, pairs compared under several relations, how the constituent loop for union targets exits), flow walks by function (steps per walk, sampled re-walks), union construction / removeSubtypes / indexed access / keyof by size, export-assignment and initializer types.

It is compiled in only with -p tsrs_checker --features tsrs_checker/work-census (the runtime test alone costs 0.3-0.5% instructions):

CARGO_TARGET_DIR=target/census cargo build --release -p tsrs_cli -p tsrs_checker --features tsrs_checker/work-census
cd <project> && TSRS_WORK_CENSUS=/tmp/census.md target/census/release/tsrs -p . --noEmit --incremental false --extendedDiagnostics --checkers 1

The report (Markdown, written after --extendedDiagnostics) starts with per-category totals, then the top keys per category. Read "key incl" as the time spent under the outermost span of that key (recursion counted once) and "self" as the time minus every nested span; categories overlap, so shares do not add up. TSRS_WORK_CENSUS_TOP=<n> sets the rows per table (default 60); TSRS_WORK_CENSUS_SLOW=<ms> prints every mapped-type resolution slower than that, with the type. Use one checker: with several, each checker's spans are merged and wall time is shared. The census adds about 18% instructions; its span cost is calibrated and subtracted, but patterns made of many short spans (unions, relations, indexed access) still read a little high.

Compilation statistics

Compilation timings are collected only when --diagnostics or --extendedDiagnostics is enabled. Config-read timing requires the flag on the command line, before the config is read. The report-only flow-memo, lazy-list, filesystem-call and reclaimed-page counters have been removed; their old statistics switches no longer report those counts.

The flow memo and its shadow mode

Control flow analysis keeps the result of a sub-walk (the type at a flow node for one reference) across walks (crates/tsrs_checker/src/flowmemo.rs, notes/perf-flow-union-inference.md). It is on by default in both modes, including Go-compatible history; --fast false does not disable it. It must never change output. Environment switches, read once per process:

  • TSRS_FLOW_MEMO=0: off. Every walk is Go's walk. Use it to tell whether a difference comes from the memo.
  • TSRS_FLOW_MEMO=shadow: every memo answer is also walked for real, and the walked result is the one used; a different type, an incomplete result or a different full reference key (the stored key bytes against the walk's) panics with the reference, the flow node, and where the stored answer came from. Output equals TSRS_FLOW_MEMO=0. Run it over the suite (TSRS_FAST=1 TSRS_FLOW_MEMO=shadow tsrs-test run --suite all --baselines types,symbols --panic-summary) and the corpora after any change to the checker that could affect what a flow walk reads.
  • TSRS_FLOW_MEMO_BITS=<8..24>: log2 of the table's slots (default 12, 32 bytes each).

testdata/flow-memo holds the hazard cases (each with tsgo's output); cargo test -p tsrs_cli --test integration flow_memo runs them with the memo on, off and in shadow mode.

The union front cache and its shadow mode: TSRS_UNION_CACHE

getUnionType calls with two or more inputs and no origin first look in a small direct-mapped table per checker (crates/tsrs_checker/src/unioncache.rs; 2^10 slots of 40 bytes, allocated on first use), keyed by the input handles in input order, the reduction mode and the alias (symbol and type arguments), at most 8 words. A call is stored only if it created no type other than the union it returns, instantiated nothing, took neither the template-literal nor the constrained-type-variable reduction, and did not return errorType (the TS2590 path); the module comment has the argument for why a hit then returns exactly what the uncached call would. On with --fast, off under --checkerAssignment go. Diagnostics, .types/.symbols output and the Types/Instantiations counters are the same either way.

variable values effect
TSRS_UNION_CACHE unset (off in Go history, on in fast mode), 0/off, 1/on (on in every mode), shadow shadow also computes the uncached answer at every hit and panics if it is a different type (the panic names the input type ids, the reduction and both answers)
TSRS_UNION_CACHE_STATS 1 at exit, one line on stderr: lookups, hits, stores, why misses were not stored, calls that bypass the cache (shadow mode prints it too)
TSRS_UNION_CACHE_BITS default 10 log2 of the table size. Larger tables hit slightly more often (2^12: 0.03% fewer instructions) but cost up to 2 MiB of RSS per checker (an allocation above mimalloc's medium size)

The indexed-access memo has the same history-dependent default: off in Go history, on in canonical history. TSRS_IA_MEMO=0 / off disables it, 1 / on enables it in either mode, and shadow checks cached answers against an uncached evaluation (implementation).

To audit a change near union construction, run the suite and a corpus with TSRS_UNION_CACHE=shadow: the result trees and the diagnostics must equal a TSRS_UNION_CACHE=0 run, with no panics (tsrs-test run --panic-summary). A shadow failure means a stored call was not a function of its inputs: find which branch of get_union_type_worker_inner it took and either keep it out of the cache (count it in union_front_cache.impure) or fix the state it read.

The inference walk memo and its shadow mode: TSRS_INFER_MEMO

inferTypes looks up its walk in a per-checker map (crates/tsrs_checker/src/infermemo.rs) keyed by the source, the target, the priority, the contravariance and what the walk reads from each inference info (type parameter, candidate lists, priority, topLevel, isFixed, implied arity; at most 8 infos). The value is what the walk wrote: each info's candidate lists, priority and topLevel, and whether it cleared the cached inferred types. A walk is stored only if it took at least 16 inferFromTypes steps (and an earlier walk to the same target did too), created no type, symbol or signature, instantiated nothing, added no diagnostic, took no impure union reduction, read no transient state (the flow memo's taint frame around it), did not start inside an instantiation and, if it started with the instantiation count at 0, checked no expression; the module comment has the argument. On with --fast, off under --checkerAssignment go. It pays where one generic call is inferred many times from the same contextual type (vscode's mapSessionEvents.test.ts: notes/perf-heavy-files-infer-memo.md).

variable values effect
TSRS_INFER_MEMO unset (off in Go history, on in fast mode), 0/off, 1/on (on in every mode), shadow shadow also walks every hit from the same starting state and panics unless the walk again creates nothing but lazy member symbols, has no other effects and ends in the stored outcome (the panic names the type ids and both outcomes)
TSRS_INFER_MEMO_STATS 1 at exit, one line on stderr: lookups, hits, the inferFromTypes steps the hits skipped, stores, and why long walks were not stored (shadow mode prints it too)
TSRS_INFER_MEMO_MIN_STEPS default 16 the shortest walk (in inferFromTypes steps) that is stored; lower stores more walks (more memory, more lookups that pay)

The relater stacks' recursion identities and their shadow mode: TSRS_DEEP_NESTED

Each entry of a relater's source and target stacks keeps its recursion identity once a deeply-nested test has computed it (RelaterStackEntry, is_deeply_nested_on_stack in crates/tsrs_checker/src/relater_1.rs); the first walk to reach an entry computes it as hasMatchingRecursionIdentity would.

variable values effect
TSRS_DEEP_NESTED unset, shadow shadow also makes every such test with a fresh walk (is_deeply_nested_type) and recomputes every kept identity, and panics on a difference (the panic names the type id and the stack length)

Freeing checked leaf files: TSRS_FREE_LEAVES

In fast mode, a CLI --noEmit check with a checker upper bound of at most 16 (MAX_DEFAULT_CHECKERS: above that it costs vscode 2.4-3.9% wall time, so it is off unless TSRS_FREE_LEAVES=1; notes/mem-leaf-regions-cost.md) parses each TypeScript root file whose path predicts a leaf (tests, specs, stories, mocks: PREDICTED_LEAF_PATTERNS) into a region of its own and frees the tree and binder output of each leaf among them as soon as its diagnostics are collected (crates/tsrs_compiler/src/fileregions.rs, notes/mem-free-leaf-files.md). A leaf is a checked TypeScript module that no other file refers to and that exports only its own declarations; on vscode 2,289 of the 2,337 leaves are predicted (99.2% of the leaves' nodes). Other files are parsed into the thread arenas as before. The text, the SourceFile and the diagnostics are kept, so the report is the same. It is off with declaration emit (declaration, composite), --explainFiles, --incremental, --build, watch, Go-compatible history (the default or --checkerAssignment go), in the language server, the API and the test harnesses, under the debug modes that walk files or checker tables after the check (TSRS_FILE_TIMES and the work census), and in builds without compressed pointers (plain-ptrs, non-unix): there a freed region's memory stays mapped and the heap its values owned is reused, so a stale read would see a live object, and nothing would be saved. The upper bound is computed before loading files: one for --singleThreaded, the explicit --checkers value, or the machine-based fast count. A later reduction for a small program does not enable freeing. Asking a program for one freed leaf's diagnostics after the pass panics (the tree is gone); the CLI never does.

variable values effect
TSRS_FREE_LEAVES a comma list of: unset (free with a checker upper bound of at most 16), 1 (free at any count), 0, keep, stats, all 0: no file regions (the layout before). keep (any count): file regions and the leaf marks, nothing freed (what the regions alone cost). all (any count): every TypeScript root file gets a region, not only the predicted ones (frees every leaf, but moves every tree out of the huge-page thread arenas: +6-9% wall time on vscode at 32 checkers on Linux against +2.5-5% predicted). stats: one line on stderr: the leaves freed and the ones the prediction missed (with their share of the leaves' nodes), the bytes their regions used, all file regions' bytes, and the arena address space the run used. Example: TSRS_FREE_LEAVES=all,stats

The variables above only apply when leaf freeing is eligible; TSRS_FREE_LEAVES=1 does not bypass Go history or the other exclusions.

After a change that reads a file after the check pass (a new report, a new whole-program loop in the checker, such as getAlternativeContainingModules's), run a corpus in fast mode with TSRS_ARENA_POISON=1: a freed region is then filled with 0xA5 and kept, so a read of a freed leaf crashes. The output must equal a TSRS_FREE_LEAVES=0 run. cargo test -p tsrs_cli --test integration free_leaf_files covers the checker's loop over every module (testdata/regressions/leaf-alternative-containers) and a structurally identical instantiation after a freed leaf (leaf-structural-instantiation).

Declaration-file member lists parsed on first use: TSRS_LAZY_DTS

Fast mode or TSRS_LAZY_DTS=1 enables lazy declaration parsing for batch and incremental CLI compilation when no declaration file is type-checked (skipLibCheck or noCheck). Non-watch --build also enables it when every built project qualifies (notes/mem-lazy-dts-build.md). It parses each interface, class and type-literal member list and keeps it only as a record (position, parser context, binder state) when nothing in it reaches outside the list: the first reader of the list's nodes or of the owner symbol's members parses and binds it again (crates/tsrs_ast/src/lazylist.rs, parse_member_list_lazily in the parser, bind_lazy_member_list in the binder; notes/mem-lazy-dts-members.md). Before the checkers of a multi-checker pass start, the lists of the global libraries (default and lib files, type reference directives and what they reference) and of interfaces merged into the global scope are forced in parallel, so that the checkers do not wait for one another on them (force_shared_lists; TSRS_LAZY_DTS_SHARED=0 skips the global libraries). Output is the same; --extendedDiagnostics Symbols counts only the lists that were bound. Off in watch, the language server, the API and the test harnesses, and under TSRS_CHECK_SHARED=1.

variable values effect
TSRS_LAZY_DTS unset (follows fast preset), 1 (on when eligible), 0/off (off), force (tsrs-test only) force in tsrs-test: every declaration file's lists are lazy, checked files included, so the checker and the .types / .symbols walks force them all: the result trees must equal a run without it

After a change to what the binder does for members of interfaces, classes or type literals, or to what a member list can contain, run the suite with TSRS_LAZY_DTS=force (also with TS_TEST_PROGRAM_SINGLE_THREADED=false) and a corpus with TSRS_ARENA_POISON=1 (rewound first parses are filled with 0xA5 and never reused); both must equal main. testdata/regressions/lazy-dts-members holds the cases where a lazy list could differ.

Checking a heavy declaration file on several checkers: TSRS_SPLIT_FILES

In the type-check pass with stealing (--fast with more than one checker), a checked declaration file that weighs at least 40% of an average checker's share (80% for default library files) is cut into contiguous statement ranges of about 1/8 of a share (crates/tsrs_compiler/src/splitcheck.rs, notes/perf-next-heavy-files.md). The checker that owns the file checks the first range; each other range is a queue item at the front of another checker's queue, which checks only those statements (Checker::check_source_file_piece) and hands its diagnostics for the file to the owner. The owner adds them to its collection, which deduplicates and sorts, and then runs the file-level steps as usual. A file is split only if all its top-level statements are declarations (TS1036 is reported on the first executable statement of a block only, so two ranges would each report one). It is off with a named assignment (--checkerAssignment, go, random:<seed>), single-threaded, and in every other pass (declaration diagnostics, emit, incremental, the language server).

variable values effect
TSRS_SPLIT_FILES a comma list of: unset (on), 0/off, shadow, force:<k>, stats, stats:<file> shadow: the owner also checks the other checkers' ranges itself, reports the file as one checker finds it, and panics if the split would have reported anything else (a diagnostic another checker found that it did not, or one it found in another checker's range that no other checker did). force:<k>: split every checked declaration file with at least two statements into up to k ranges, whatever its weight (the test mode). stats: one line per pass on stderr (files split, ranges, ranges run by another checker); stats:<file> appends it to a file (the test harnesses keep stderr)
TSRS_SPLIT_MIN_SHARE percent, default 40 which files are split (weight relative to an average checker's share)
TSRS_SPLIT_PIECE_DIVISOR default 8 range size: 1/n of an average checker's share

After a change to what checking a declaration statement reads or writes outside the statement, run the conformance suite in fast mode with forced splits and compare its lists with an unsplit run: TS_TEST_PROGRAM_SINGLE_THREADED=false TSRS_FAST=1 TSRS_SPLIT_FILES=force:3,shadow tsrs-test run --suite all --baselines types,symbols --panic-summary against the same with TSRS_SPLIT_FILES=0 (about 630 declaration files in 330 test programs are split). cargo test -p tsrs_cli --test integration split_check runs testdata/split-check in every mode against tsgo-ref's output.

Data races: ThreadSanitizer (tasks/thread_sanitizer)

P<T> is Send/Sync by decree (tsrs_core::ptr): nothing in the type system stops two threads from writing the same arena value. Build tsrs with ThreadSanitizer and run vp run ci:thread-sanitizer <binary>, which fails on any report: type checks at 2 and 8 checkers, emit, -b on a generated graph of 12 projects, --watch and -b --watch with edits, and a language-server session. Prepare the benchmark projects first, then run the separate task:

node bench/bench.mjs prepare --projects xstate-main,webpack
RUSTFLAGS=-Zsanitizer=thread CARGO_PROFILE_DEV_DEBUG=line-tables-only \
  CARGO_TARGET_DIR=target/thread-sanitizer cargo +nightly-2026-10-04 build --profile dev -p tsrs_cli -Zbuild-std \
  --features tsrs_core/plain-ptrs,tsrs_cli/system-alloc --target "$(rustc -vV | sed -n 's/^host: //p')"
vp run ci:thread-sanitizer target/thread-sanitizer/<host triple>/debug/tsrs

The ThreadSanitizer job in .github/workflows/ci.yml runs this gate on Linux for pull requests, merge groups, pushes to main, and manual CI runs. It uses the pinned nightly with an instrumented standard library and the optimized ci profile, retaining debug locations and symbols. The xstate-main and webpack workloads have a separate preparation cache. Results are uploaded as the thread-sanitizer artifact on success or failure and kept for 14 days; sanitizer failures fail the job.

plain-ptrs because ThreadSanitizer's shadow memory takes the compressed handles' address range; system-alloc because it cannot see mimalloc reuse memory across threads (with mimalloc it reports about 120 races that are not). On Linux, ThreadSanitizer needs sudo sysctl vm.mmap_rnd_bits=28 on kernels that randomize 32 bits. The task rejects an uninstrumented binary and fails on sanitizer reports, abnormal exits, LSP request errors, or timeouts. It continues independent workloads after a failure and never retries a failed workload into a pass. Each invocation retains a new directory under target/thread-sanitizer-results/, including summary.json, the binary SHA-256, commands, relevant environment settings, exit statuses, stdout/stderr, sanitizer reports, and watch/LSP transcripts. summary.json stays incomplete if setup fails or the task is interrupted. Earlier run directories are never replaced. --projects all|a,b, --work-dir DIR, --out-dir DIR, and --timeout SECONDS select the inputs, output parent, and per-workload deadline (900 seconds by default). The default projects remain xstate-main,webpack. vp run ci:thread-sanitizer-summary <logs...> groups reports by their first tsrs frames. tasks/thread_sanitizer/suppressions.txt lists reports that are not tsrs races, each with its reason; anything else is a race to fix.

Profiling

Profile the release profile (fat LTO, one codegen unit) with full debug information and stripping disabled; release binaries also add PGO, and BOLT on Linux (.github/workflows/release.yml). Keep frame pointers for reliable stacks on Linux x86-64, in a separate target directory (Apple arm64 always keeps frame pointers):

CARGO_PROFILE_RELEASE_DEBUG=true CARGO_PROFILE_RELEASE_STRIP=none \
  RUSTFLAGS="-C force-frame-pointers=yes" CARGO_TARGET_DIR=target/profiling \
  cargo build --release -p tsrs_cli
samply record ./target/profiling/release/tsrs -p <project> --noEmit --incremental false

On macOS, prefer xcrun xctrace to locate hotspots. Use Hyperfine 2 with hardware counters for before/after measurements: report single-threaded instructions retired and default-checker peak RSS, plus wall time when claiming a speedup. Keep build settings, hardware, workload and checker counts identical, with warmups and repeated, interleaved runs. See benchmark methodology for the catalog and measurement tools.

For per-function instruction counts on Linux, use valgrind --tool=callgrind --callgrind-out-file=cg.out <tsrs command> on the symbol-bearing release build, then callgrind_annotate --inclusive=no --tree=none cg.out (--tree=none prevents duplicate call-graph counts). The active .github/workflows/benchmark.yml runs benchmark measurement and reporting; auxiliary workflows saved as .yml.bak remain disabled.

Where Go reports an error: an instrumented reference with go build -overlay

Build the Go reference with a patched copy of one Go file, without editing ts-ref: copy the file (e.g. internal/checker/checker.go) to scratch and patch it (e.g. print runtime/debug.Stack() at the error site). Write an overlay JSON that maps both the symlinked and the real path of the file to the copy. Then, in ts-ref/tsc, run GOTOOLCHAIN=auto GOWORK=off go build -overlay <overlay.json> -o <bin> ./cmd/tsrs-oracle-testrunner (the copy of tools/oracle/testrunner/main.go, placed as docs/PORTING.md "Reference tooling" describes) and <bin> diags <regexp> (one JSON line per variant of each test whose file name matches).

How to fix

  • The Go source (ts-ref/tsc/internal/…) is the specification. Find the Go function behind the wrong behavior, read it next to the Rust port (// file.go:LINE origin markers; docs/sigs/checker.txt maps names), and make the Rust do what the Go does. Most bugs are small porting slips: a dropped branch, a swapped argument, intersects vs contains, nil-vs-empty, an unwrap() where Go tolerates nil, evaluation order, a wrong flag constant.
  • Never special-case a test, never paper over a symptom, never "improve" on Go. If Go has a quirk, port the quirk.
  • Fix root causes in shared helpers rather than at one call site; then look for the same slip elsewhere (grep for the pattern) — sibling bugs are common because the same agent ported neighboring functions.
  • When a signature changes, run node tools/sigs-from-rust.mjs to refresh docs/sigs/checker.txt.
  • Keep a short log in notes/fix-<agent-name>.md (root causes found, patterns worth grepping for, things you could not resolve). It is committed with your work and read by the other agents.
  • Do not stop at the first fix. Work through your cluster until it is done or what remains is blocked on something outside your scope — then say exactly what.

Test-runner backend

tsrs-test always uses the Rust compiler and its option tables. The bootstrap compiler / tsoptions Cargo features, --oracle, --options, TSRS_TEST_OPTIONS, and early-port --syntax-only mode have been retired. Use normal conformance runs for diagnostics and --baselines types,symbols,js,jsmap,sourcemap for extra baselines. Go-captured diagnostic rendering remains a unit test; the Go oracle's names and diags commands remain available for comparisons. Its bootstrap options export has been removed.