Skip to content

fix(test): make TestContextInCallbacks deterministic - #118

Open
tigerquoll wants to merge 1 commit into
looplab:mainfrom
tigerquoll:fix/flaky-context-in-callbacks-test
Open

fix(test): make TestContextInCallbacks deterministic#118
tigerquoll wants to merge 1 commit into
looplab:mainfrom
tigerquoll:fix/flaky-context-in-callbacks-test

Conversation

@tigerquoll

Copy link
Copy Markdown

Problem

TestContextInCallbacks deadlocks intermittently and takes the whole package down with it: when it loses an internal race, the test blocks forever on a channel receive and the test binary dies on the 10-minute timeout, so every other test's result is discarded too. This is what failed CI on #116 (a PR that doesn't touch fsm_test.go), and it reproduces on main in seconds with:

go test -race -run 'TestContextInCallbacks$' -count=300 -timeout 60s .

Root cause

The test races a free-running goroutine against Event:

ctx, cancel := context.WithCancel(context.Background())
go func() { cancel() }()
err := fsm.Event(ctx, "run")
...
<-enterEndAsyncWorkDone

Event's transition function checks ctx.Err() before running enter-state callbacks. If cancel() fires before that check, the transition aborts early: Event still returns context.Canceled — so the test's error assertion passes — but enter_end never runs, the goroutine that closes enterEndAsyncWorkDone is never spawned, and <-enterEndAsyncWorkDone blocks forever. The goroutine dump in the CI failure confirms it: the test goroutine is parked on that receive and no goroutine is waiting inside the callback.

The intended path only happens when cancel() loses the race, which it usually does on a fast machine and frequently doesn't on a loaded CI runner.

Fix

Test-only. Trigger the cancellation from inside enter_end, after the watcher goroutine is spawned, so it always happens after the transition has begun and can no longer race the context check. The assertions are unchanged.

Verification

  • Before: the 300-iteration command above hangs on main within ~100 iterations and hits the timeout.
  • After: 300/300 iterations pass under -race; full suite passes under -race; go vet clean.

Note: #117 fixes a different (production-facing) consequence of the same early-abort path — the two are independent and don't conflict.

TestContextInCallbacks raced a free-running cancel() goroutine against
Event(). When cancel fired before the transition's ctx.Err() check, the
enter_end callback was skipped, the goroutine that closes
enterEndAsyncWorkDone was never spawned, and the test deadlocked until
the 10m test-binary timeout. Trigger the cancellation from inside
enter_end instead, so it always happens after the transition has begun.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant