Skip to content

runtime: tighten recover to direct deferred calls#1918

Closed
cpunion wants to merge 50 commits into
xgo-dev:mainfrom
cpunion:codex/goroot-fixedbugs-recover-coverage
Closed

runtime: tighten recover to direct deferred calls#1918
cpunion wants to merge 50 commits into
xgo-dev:mainfrom
cpunion:codex/goroot-fixedbugs-recover-coverage

Conversation

@cpunion

@cpunion cpunion commented May 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fix recover/defer ownership during panic unwinding.

recover now only succeeds from the deferred function that is actually being run, while panics rethrown through nested defer and range-over-func frames remain recoverable by the correct outer defer.

Examples covered by this PR:

// A directly deferred function value can recover the panic.
defer func() {
	if recover() == nil {
		panic("not recovered")
	}
}()
panic("boom")
// A nested defer inside a deferred function must not recover the outer panic.
defer func() {
	defer func() {
		if recover() != nil {
			panic("nested recover should be nil")
		}
	}()
	if recover() != "outer" {
		panic("outer recover failed")
	}
}()
panic("outer")
// A panic crossing range-over-func/defer frames remains recoverable by the
// correct deferred function, instead of being claimed by an inner frame.
defer func() {
	if recover() != "loop panic" {
		panic("recover owner mismatch")
	}
}()
for range func(yield func(int) bool) {
	defer func() { _ = recover() }()
	yield(1)
} {
	panic("loop panic")
}

Main verification

After #1950 was merged, latest xgo-dev/llgo main still fails the nested defer/recover case above. The temporary LLGo test failed with:

outer recover = <nil>, want outer

The same case passes with go test, so this PR still covers a remaining LLGo recover/defer ownership bug.

@cpunion
cpunion force-pushed the codex/goroot-fixedbugs-recover-coverage branch from 6de90d9 to ad13f0e Compare May 24, 2026 12:52
@codecov-commenter

codecov-commenter commented May 24, 2026

Copy link
Copy Markdown

@cpunion
cpunion marked this pull request as ready for review June 1, 2026 02:30
@cpunion

cpunion commented Jun 6, 2026

Copy link
Copy Markdown
Collaborator Author

Added patch-coverage tests for the recover/direct-deferred-call helpers:

  • ssa/ssa_test.go: covers recover defer token classification for marked function decls, function pointers, non-function values, nil expressions, and the recover builtin helper.
  • cl/cgo_test.go: covers functionUsesRecover nil/direct-recover cases and callMayRecover conservative closure/default paths.

Local verification:

  • go test ./ssa -coverprofile=/tmp/llgo-recover-ssa-full-after.out (passes; ssa/eh.go recover helper functions now report 100% for callRecoverScopedDefer, recoverDeferToken, deferMayRecover, and isRecoverBuiltin)
  • go test ./cl -run 'Test(RecoverCallClassificationHelpers|EmitDo|CompileRangeFuncDeferModule|DeferStackOwner|RangeFuncDefer|NestedRangeFunc|Cgo)' -coverprofile=/tmp/llgo-recover-cl-after2.out (passes; functionUsesRecover and callMayRecover report 100%)
  • go test ./test/go -run TestRecoverFixedbug -count=1 (passes)
  • go run ./cmd/llgo test -run TestRecoverFixedbug -count=1 ./test/go/recover_defer_fixedbugs_test.go (passes)
  • git diff --check (passes)

Note: go run ./cmd/llgo test -run TestRecoverFixedbug -count=1 ./test/go still links the whole test/go package and failed before reaching the recover tests due unrelated generic-local-type undefined symbols, so I used the single recover test file for the focused LLGo runtime/codegen check.

@cpunion
cpunion force-pushed the codex/goroot-fixedbugs-recover-coverage branch 2 times, most recently from 747e47b to a16d798 Compare June 10, 2026 11:30
@cpunion
cpunion force-pushed the codex/goroot-fixedbugs-recover-coverage branch 2 times, most recently from 071f268 to 41dfd44 Compare June 19, 2026 14:44
@cpunion
cpunion force-pushed the codex/goroot-fixedbugs-recover-coverage branch 2 times, most recently from 7156fc2 to 80e2e83 Compare June 29, 2026 02:26
cpunion and others added 13 commits July 2, 2026 20:41
Every llgo-linked executable (linux/darwin, sites enabled) now gets the
prebuilt ftab/findfunctab automatically: internal/build runs
internal/pclnpost.Rewrite after linkMainPkg, and any failure degrades
silently to the first-use construction fallback.

Moves the tool core into internal/pclnpost and hardens it:

- Canonical-record detection by FNV: a record survives when its anchor's
  owning symbol hashes to the record's symbolID (or is the __llgo_stub.
  wrapper of it). The previous one-per-symbolID rule wrongly collapsed a
  function with its stub — they share the target's symbolID by design —
  which broke exact-entry lookups (caught by TestRuntimeLineInfoAndStack
  on Linux). LTO inline copies are now identified exactly: 8.4k/9.5k
  copies removed in the LTO probes.
- Mach-O chained-fixups surgery: unlink the rewritten sections' pointer
  slots from the dyld page chains (repointing predecessors' next links
  and page_start entries) so dyld neither rebases slots inside the new
  table nor skips unrelated fixups after the zeroed stub section, then
  re-sign ad hoc. Without this the table was corrupted at load.
- LTO-safe metadata location: the entry section carries a meta record
  whose relocations hold the addresses of the symbol-index pointer and
  count globals; LTO internalization strips those names from the symbol
  table but relocations always resolve. Runtime skips the meta rows
  (pc==0 / symbolID==0).
- Idempotence guard (already-rewritten binaries are left alone).

Runtime fixes that surfaced during validation:

- materializePrebuiltEntries is now two-phase so concurrent losers wait
  for the winner's store instead of reading a nil entries slice.
- pcLineFrameForPC rejects nearest-below sites whose entry is
  unresolved when the caller knows the function entry, instead of
  leaking a neighboring function's file/line.

Validation: macOS cl (full) + test/go + LLDB 194/194; Linux test/go
TestRuntime suite; probes on both platforms report entries=prebuilt
with first-FuncForPC at 7-21µs (Linux) from 13ms on the original
baseline, and LTO builds drop 8-9.5k inline copies.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…table

On Mach-O, pointer slots that name exported functions — every
__llgo_stub.* wrapper and any exported Go function — are emitted as
chained-fixup BIND nodes, not rebases. The rewriter only decoded rebase
nodes, so all stub records (and some entry records) were dropped as
unowned and never reached the prebuilt ftab; FuncForPC on function
values silently fell back to dladdr (~6µs per fresh pc on darwin).

- Parse the LC_DYLD_CHAINED_FIXUPS imports table and resolve bind
  ordinals to their in-image definitions.
- Match canonical owners against the record symbolID with underscore
  normalization (debug/macho's suffix-shared string table can surface
  one mangling underscore more or less than the source-level name).
- Splice the prebuilt header's base slot back into the fixup chain as a
  live rebase node: dyld writes the slid text base at load, so the
  runtime reads a ready runtime PC with no slide arithmetic (non-PIE
  ELF link-time values already equal runtime addresses).
- LLGO_PCLNPOST=0 escape hatch keeps first-use construction.

Fresh-pc FuncForPC slow path: darwin 6-8µs -> 1.2-1.7µs, linux
6.8µs -> 0.5µs; first-in-process lookup: darwin ~32µs -> ~14µs,
linux ~6.8µs -> ~4µs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pure-compute probes (recursive fib, JSON round-trip, sort.Ints, map
churn) with no runtime introspection, so one harness run covers both
the introspection extremes and what the funcinfo machinery costs code
that never asks for it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Go's pclntab pages are touched by its own runtime (traceback, GC) long
before user code queries it, so its first FuncForPC never pays page-in.
Mirror that: when the prebuilt table is present, init adopts it
(zero-copy, sub-µs), touches the pages the lookup path reads (blob,
funcinfo records, string offsets, strings), runs one synthetic lookup
to warm the code paths, and write-warms the FuncForPC cache pages.

First-in-process FuncForPC: darwin ~17µs -> ~2.8µs, linux ~6.6µs ->
~1.0µs. Startup cost is page-count-bound (tens of µs on stdlib-sized
tables, invisible next to ~3ms process startup; hello-world medians
unchanged). Non-prebuilt binaries stay fully lazy: first-use
construction allocates, which has no place in init, and programs that
never introspect pay nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-depths generates deep_<N> scenarios at configurable call depths;
-bigsizes generates bigfunc scenarios (funcs x statements) whose large
bodies stress statement-level pcline density, mid-function pc
symbolization, and ordinary performance of big method bodies.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cpunion
cpunion force-pushed the codex/goroot-fixedbugs-recover-coverage branch from f867d84 to 8fe0ad3 Compare July 2, 2026 15:03
@cpunion

cpunion commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto #2016 (codex/pclntab-linkphase-p1, which includes #2012) per the review-order plan: #2012#2016 → this line of semantics fixes. Conflicts resolved were additive (context fields, the noinline condition set, and runtime.Panic now calls SavePanicCallerFrames() before the panic-node bookkeeping). Note the PR base is still main, so the diff shows #2012/#2016 commits until those merge.

@cpunion

cpunion commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #2033, which re-expresses the Defer-node recover-ownership model on the stage-5 base (#2023); the funcinfo draft carried here was superseded by #2012/#2016/#2019. #1882's recover-error-value work re-expresses on top of #2033 next.

@cpunion cpunion closed this Jul 6, 2026
cpunion added a commit to cpunion/llgo that referenced this pull request Jul 8, 2026
Re-expresses xgo-dev#1918 on the xgo-dev#2023 base (its remaining ~11k diff lines were
the pre-xgo-dev#2012 funcinfo draft, superseded by the stage-5 chain):

- recover() only succeeds when called directly by a deferred function
  (gc semantics): the panic node records the owning Defer frame at
  rethrow (panicKey/panicNode + GoDeferData), and Recover checks the
  caller is that frame's direct deferred call. Closure wraps carry
  StartRecoverFrameAlias/EndRecoverFrame so method-value and closure
  adapters stay transparent to the ownership check.
- Rethrow keeps the xgo-dev#2023 PanicTraceback hook on the unrecovered path.
- xfail: retire fixedbugs/issue4066 (2m-timeout entries; now runs in
  ~2.7s), fixedbugs/issue73916 and issue73916b (go1.26 recover
  semantics), validated on darwin/arm64 go1.26.

Supersedes xgo-dev#1918.
cpunion added a commit to cpunion/llgo that referenced this pull request Jul 8, 2026
Re-expresses xgo-dev#1918 on the xgo-dev#2023 base (its remaining ~11k diff lines were
the pre-xgo-dev#2012 funcinfo draft, superseded by the stage-5 chain):

- recover() only succeeds when called directly by a deferred function
  (gc semantics): the panic node records the owning Defer frame at
  rethrow (panicKey/panicNode + GoDeferData), and Recover checks the
  caller is that frame's direct deferred call. Closure wraps carry
  StartRecoverFrameAlias/EndRecoverFrame so method-value and closure
  adapters stay transparent to the ownership check.
- Rethrow keeps the xgo-dev#2023 PanicTraceback hook on the unrecovered path.
- xfail: retire fixedbugs/issue4066 (2m-timeout entries; now runs in
  ~2.7s), fixedbugs/issue73916 and issue73916b (go1.26 recover
  semantics), validated on darwin/arm64 go1.26.

Supersedes xgo-dev#1918.
cpunion added a commit to cpunion/llgo that referenced this pull request Jul 8, 2026
Re-expresses xgo-dev#1918 on the xgo-dev#2023 base (its remaining ~11k diff lines were
the pre-xgo-dev#2012 funcinfo draft, superseded by the stage-5 chain):

- recover() only succeeds when called directly by a deferred function
  (gc semantics): the panic node records the owning Defer frame at
  rethrow (panicKey/panicNode + GoDeferData), and Recover checks the
  caller is that frame's direct deferred call. Closure wraps carry
  StartRecoverFrameAlias/EndRecoverFrame so method-value and closure
  adapters stay transparent to the ownership check.
- Rethrow keeps the xgo-dev#2023 PanicTraceback hook on the unrecovered path.
- xfail: retire fixedbugs/issue4066 (2m-timeout entries; now runs in
  ~2.7s), fixedbugs/issue73916 and issue73916b (go1.26 recover
  semantics), validated on darwin/arm64 go1.26.

Supersedes xgo-dev#1918.
cpunion added a commit to cpunion/llgo that referenced this pull request Jul 9, 2026
Re-expresses xgo-dev#1918 on the xgo-dev#2023 base (its remaining ~11k diff lines were
the pre-xgo-dev#2012 funcinfo draft, superseded by the stage-5 chain):

- recover() only succeeds when called directly by a deferred function
  (gc semantics): the panic node records the owning Defer frame at
  rethrow (panicKey/panicNode + GoDeferData), and Recover checks the
  caller is that frame's direct deferred call. Closure wraps carry
  StartRecoverFrameAlias/EndRecoverFrame so method-value and closure
  adapters stay transparent to the ownership check.
- Rethrow keeps the xgo-dev#2023 PanicTraceback hook on the unrecovered path.
- xfail: retire fixedbugs/issue4066 (2m-timeout entries; now runs in
  ~2.7s), fixedbugs/issue73916 and issue73916b (go1.26 recover
  semantics), validated on darwin/arm64 go1.26.

Supersedes xgo-dev#1918.
cpunion added a commit to cpunion/llgo that referenced this pull request Jul 16, 2026
Re-expresses xgo-dev#1918 on the xgo-dev#2023 base (its remaining ~11k diff lines were
the pre-xgo-dev#2012 funcinfo draft, superseded by the stage-5 chain):

- recover() only succeeds when called directly by a deferred function
  (gc semantics): the panic node records the owning Defer frame at
  rethrow (panicKey/panicNode + GoDeferData), and Recover checks the
  caller is that frame's direct deferred call. Closure wraps carry
  StartRecoverFrameAlias/EndRecoverFrame so method-value and closure
  adapters stay transparent to the ownership check.
- Rethrow keeps the xgo-dev#2023 PanicTraceback hook on the unrecovered path.
- xfail: retire fixedbugs/issue4066 (2m-timeout entries; now runs in
  ~2.7s), fixedbugs/issue73916 and issue73916b (go1.26 recover
  semantics), validated on darwin/arm64 go1.26.

Supersedes xgo-dev#1918.
cpunion added a commit to cpunion/llgo that referenced this pull request Jul 17, 2026
Re-expresses xgo-dev#1918 on the xgo-dev#2023 base (its remaining ~11k diff lines were
the pre-xgo-dev#2012 funcinfo draft, superseded by the stage-5 chain):

- recover() only succeeds when called directly by a deferred function
  (gc semantics): the panic node records the owning Defer frame at
  rethrow (panicKey/panicNode + GoDeferData), and Recover checks the
  caller is that frame's direct deferred call. Closure wraps carry
  StartRecoverFrameAlias/EndRecoverFrame so method-value and closure
  adapters stay transparent to the ownership check.
- Rethrow keeps the xgo-dev#2023 PanicTraceback hook on the unrecovered path.
- xfail: retire fixedbugs/issue4066 (2m-timeout entries; now runs in
  ~2.7s), fixedbugs/issue73916 and issue73916b (go1.26 recover
  semantics), validated on darwin/arm64 go1.26.

Supersedes xgo-dev#1918.
cpunion added a commit to cpunion/llgo that referenced this pull request Jul 18, 2026
Re-expresses xgo-dev#1918 on the xgo-dev#2023 base (its remaining ~11k diff lines were
the pre-xgo-dev#2012 funcinfo draft, superseded by the stage-5 chain):

- recover() only succeeds when called directly by a deferred function
  (gc semantics): the panic node records the owning Defer frame at
  rethrow (panicKey/panicNode + GoDeferData), and Recover checks the
  caller is that frame's direct deferred call. Closure wraps carry
  StartRecoverFrameAlias/EndRecoverFrame so method-value and closure
  adapters stay transparent to the ownership check.
- Rethrow keeps the xgo-dev#2023 PanicTraceback hook on the unrecovered path.
- xfail: retire fixedbugs/issue4066 (2m-timeout entries; now runs in
  ~2.7s), fixedbugs/issue73916 and issue73916b (go1.26 recover
  semantics), validated on darwin/arm64 go1.26.

Supersedes xgo-dev#1918.
cpunion added a commit to cpunion/llgo that referenced this pull request Jul 18, 2026
Re-expresses xgo-dev#1918 on the xgo-dev#2023 base (its remaining ~11k diff lines were
the pre-xgo-dev#2012 funcinfo draft, superseded by the stage-5 chain):

- recover() only succeeds when called directly by a deferred function
  (gc semantics): the panic node records the owning Defer frame at
  rethrow (panicKey/panicNode + GoDeferData), and Recover checks the
  caller is that frame's direct deferred call. Closure wraps carry
  StartRecoverFrameAlias/EndRecoverFrame so method-value and closure
  adapters stay transparent to the ownership check.
- Rethrow keeps the xgo-dev#2023 PanicTraceback hook on the unrecovered path.
- xfail: retire fixedbugs/issue4066 (2m-timeout entries; now runs in
  ~2.7s), fixedbugs/issue73916 and issue73916b (go1.26 recover
  semantics), validated on darwin/arm64 go1.26.

Supersedes xgo-dev#1918.
cpunion added a commit to cpunion/llgo that referenced this pull request Jul 19, 2026
Re-expresses xgo-dev#1918 on the xgo-dev#2023 base (its remaining ~11k diff lines were
the pre-xgo-dev#2012 funcinfo draft, superseded by the stage-5 chain):

- recover() only succeeds when called directly by a deferred function
  (gc semantics): the panic node records the owning Defer frame at
  rethrow (panicKey/panicNode + GoDeferData), and Recover checks the
  caller is that frame's direct deferred call. Closure wraps carry
  StartRecoverFrameAlias/EndRecoverFrame so method-value and closure
  adapters stay transparent to the ownership check.
- Rethrow keeps the xgo-dev#2023 PanicTraceback hook on the unrecovered path.
- xfail: retire fixedbugs/issue4066 (2m-timeout entries; now runs in
  ~2.7s), fixedbugs/issue73916 and issue73916b (go1.26 recover
  semantics), validated on darwin/arm64 go1.26.

Supersedes xgo-dev#1918.
cpunion added a commit to cpunion/llgo that referenced this pull request Jul 20, 2026
Re-expresses xgo-dev#1918 on the xgo-dev#2023 base (its remaining ~11k diff lines were
the pre-xgo-dev#2012 funcinfo draft, superseded by the stage-5 chain):

- recover() only succeeds when called directly by a deferred function
  (gc semantics): the panic node records the owning Defer frame at
  rethrow (panicKey/panicNode + GoDeferData), and Recover checks the
  caller is that frame's direct deferred call. Closure wraps carry
  StartRecoverFrameAlias/EndRecoverFrame so method-value and closure
  adapters stay transparent to the ownership check.
- Rethrow keeps the xgo-dev#2023 PanicTraceback hook on the unrecovered path.
- xfail: retire fixedbugs/issue4066 (2m-timeout entries; now runs in
  ~2.7s), fixedbugs/issue73916 and issue73916b (go1.26 recover
  semantics), validated on darwin/arm64 go1.26.

Supersedes xgo-dev#1918.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants