Skip to content

runtime/wasm: add single-worker Asyncify scheduler - #2192

Open
cpunion wants to merge 8 commits into
xgo-dev:mainfrom
cpunion:codex/wasm-single-worker-asyncify-scheduler
Open

runtime/wasm: add single-worker Asyncify scheduler#2192
cpunion wants to merge 8 commits into
xgo-dev:mainfrom
cpunion:codex/wasm-single-worker-asyncify-scheduler

Conversation

@cpunion

@cpunion cpunion commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Part of #2152. #2166 is merged and provides the runtime-owned G/M/P boundary used here.

Problem

#2166 moves go statement creation behind runtime.NewProc, but its only execution backend is one pthread per G. js/wasm has no scheduler that can suspend multiple G contexts on one Worker, so a runnable WebAssembly goroutine path is still missing.

The two supported js/wasm entry points also need distinct, unambiguous data models:

  • GOOS=js GOARCH=wasm llgo build keeps Go's 64-bit word model and uses Emscripten Memory64;
  • llgo build -target wasm implies GOOS=js GOARCH=wasm but selects the configured wasm32 model.

They must not require redundant GOOS/GOARCH flags or share incompatible package-cache entries.

Change

  • keep the existing pthread lifecycle in a backend-specific file and preserve its 1:1 behavior;
  • add a js/wasm backend with one M, one P, an allocation-free FIFO run queue, and Emscripten Fiber contexts backed by Asyncify;
  • route runtime.Gosched, park/ready test points, normal return, panic/defer/recover, and runtime.Goexit through that scheduler;
  • preserve compiler/runtime: add //llgo:tls and //llgo:gls package variables #2079 owner teardown by keeping it in the pthread Goexit backend;
  • reclaim a dead G context only after execution has switched to another C stack;
  • resolve -target wasm through its target configuration before platform-dependent output and package loading;
  • use the existing tinygo.wasm target tag and target name for wasm32, without adding another environment variable or build tag;
  • emit raw GOOS=js GOARCH=wasm as wasm64-unknown-emscripten with MEMORY64=1, while the named target remains wasm32-unknown-emscripten;
  • select LP64 C long for Memory64 and 32-bit C long for the configured target;
  • make C ssize_t follow pointer width, avoiding an i32/i64 write signature mismatch under Memory64;
  • allow explicit .js/.mjs output and include Node in the existing Emscripten environment list;
  • select the existing nogc runtime for wasm targets, because BDWGC is unavailable there;
  • correct G/M/P ID allocation for LLGo atomics, whose Add returns the previous value.

The default per-G reservation is one 64 KiB C stack plus one 64 KiB Asyncify area. The main G allocates only its Asyncify area on first switch. No pthread or Worker is created per G.

The Emscripten Fiber API is used only as the raw unwind/rewind context adapter. The scheduler does not use Promise-owned Asyncify flow. #2079 currently maps TLS and GLS to one physical owner; separating GLS per logical G on a multiplexed worker requires a later locality-layout change and is not partially implemented here.

Execution flow

  1. NewProc allocates G state and two stack areas, initializes a Fiber, and appends the G to the local FIFO.
  2. Gosched requeues the current G; park leaves it waiting; exit marks it dead.
  3. The scheduler assigns the shared M/P to the next runnable G and swaps Fiber contexts.
  4. After the new or resumed G is running on its own C stack, it releases the previously retired G context.

Scope

This is the single-Worker execution slice, not complete GOOS=js support. It does not add channels, select, timers, host async I/O, safe-point preemption, GC roots, browser Workers, or a WASI Asyncify scheduler. The existing WASI path is covered only as a build regression.

Memory64 is currently limited to the Emscripten js host. The pinned WASI SDK 25 provides only wasm32 sysroots and libraries, so raw wasip1/wasm retains its existing wasm32 compatibility path.

The independent diff over the merged #2166/main base is +1258/-160 across 38 files. 469 added lines are tests and CI fixtures; the pthread portion is primarily a mechanical move out of the common lifecycle file.

Validation

  • macOS arm64, Go 1.24.2 and Go 1.26.5, Emscripten 4.0.21, Node 25.2.1:
    • llgo build -target wasm produces wasm32 and executes the scheduler fixture;
    • GOOS=js GOARCH=wasm llgo build produces Memory64 and executes the same fixture;
    • each fixture asserts its expected uintptr and C long sizes (4 or 8 bytes);
    • both pass FIFO yield, park/ready, shared M/P ownership, panic/recover, defer, and Goexit checks.
    • both also verify that a worker exiting while main is parked reports a deadlock and terminates with status 2.
  • macOS arm64: focused build/crosscompile/SSA tests, runtime module tests/build, runtime: introduce G/M/P pthread backend #2166 native test/llgoext, raw js/wasm, and raw wasip1/wasm builds pass.
  • Ubuntu arm64 container, Go 1.26.5, --memory=15g --cpus=2: focused build/crosscompile/SSA tests and runtime module tests pass.
  • CI pins Node 25 and runs both wasm32 and Memory64 scheduler artifacts under Node for Go 1.24.2 and Go 1.26.5; the all-target sweep installs the same pinned Emscripten toolchain and also builds the configured wasm target.
  • The reduced test: parallelize llgo test packages #2193 matrix and its in-command parallelism are unchanged. The full single-shard macOS test job reached late test/std packages but was terminated by the 30-minute job cap; this PR raises only that safety cap to 40 minutes. The rerun completes successfully in 37m46s; current main is about 25 minutes.
  • GitHub CI passes on the latest main after compiler/runtime: add //llgo:tls and //llgo:gls package variables #2079, including the full Ubuntu/macOS matrix and Codecov; local J32/J64 scheduler execution and locality/compiler/runtime integration tests also pass.
  • runtime/internal/runqueue has 100% statement coverage; target resolution, type-model selection, target triples, output naming, and both fixture model assertions have focused coverage.

No test or previously runnable path is skipped or ignored.

@cpunion
cpunion force-pushed the codex/wasm-single-worker-asyncify-scheduler branch 3 times, most recently from c241c73 to e32655c Compare July 27, 2026 05:52
@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@cpunion
cpunion marked this pull request as ready for review July 27, 2026 15:06

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: single-worker Asyncify WASM scheduler

Solid, well-structured PR. The G/M/P backend split (pthread / wasm-fiber / baremetal) is cleanly partitioned by mutually-exclusive build tags, the intrusive runqueue is allocation-free and O(1), the emscripten_fiber_t layout is guarded by a _Static_assert, and retired-G reaping is correctly deferred to the next resumed G on every switch path. Test coverage (Node-executed scheduler test, G/M/P ownership tests) is good.

Findings below. The most important is #1 — the new scheduler's fail-fast guards do not actually fail fast, because the compiled fatal is a non-aborting stub. Inline comments carry the concrete locations.

1. (High) fatal() returns instead of aborting — every new scheduler guard falls through

The compiled fatal is the stub at runtime/internal/lib/runtime/.../stubs.go:119, which only prints and returns (the real aborting fatal in panic.go is inside a /* ... */ block spanning lines 260–1412, so it is not built). The new scheduler writes its invariant guards as fatal(...); return and assumes fatal never returns. Because it does return, illegal states continue executing:

  • resumeDeadWasmG (proc_wasm.go:222) unwinds a dead fiber normally after the trailing fatal.
  • casgstatus (proc_atomic.go:39-43) does not apply the CAS on an illegal transition, then every caller proceeds as if the new status took effect.
  • goschedBackend / gopark / goexitBackend return after an empty-runq fatal, leaving a _Gdead/_Gwaiting G as the running context.

Recommend either making the compiled fatal actually abort the process (e.g. abort/c.Exit) or replacing these fatal(...); return sites with an explicit abort. This is pre-existing at the stub, but this PR is where the assumption becomes load-bearing.

2. (Low) goexitBackend reports a false deadlock when a worker exits while main is parked

goexitBackend (proc_wasm.go:198) pops the runq before marking gp dead; if a normal goroutine returns while main is parked (empty runq), it hits the "no goroutines (main called runtime.Goexit) - deadlock!" path even though this is not main and not Goexit. Combined with #1 (the return is reached), the finished goroutine's fiber then unwinds with gp still _Grunning and unreaped. Please confirm the intended "worker exits while main waits" handling.

3. (Low) Inconsistent ID generation: nextGoid/nextMid add +1, nextPid does not

See inline comment on proc_atomic.go. Not currently a functional bug (tests only assert non-zero/unique), but the undocumented asymmetry is a latent hazard — make the three uniform or comment the intent.

4. (Low) Stale/misleading doc on the shared g type

See inline comment on runtime2.go:38. runtime2.go has no build tag, so this g is shared by all backends, yet the comment describes only the pthread per-G M/P model — the exact model the new wasm backend (single shared wasmSched.m/.p) is built to violate. The "when those facilities are added" sentence is also stale: this PR adds suspend/resume.

5. (Info) Gosched doc overstates behavior on pthread

Gosched (proc.go:104) is documented as yielding to another goroutine, but goschedBackend is a no-op on the pthread backend (proc_pthread.go:105). Consider noting that yielding is backend-dependent.

Non-blocking follow-ups

  • No stack/context pool: each go on wasm does 2× AllocRoot (~128KB: 64KB G stack + ≥64KB asyncify stack) + 2× free per goroutine, never pooled. A free-list keyed on the default size would remove steady-state churn under high goroutine turnover.
  • resumeDeadWasmG is a near-duplicate of resumeWasmG but omits the nil-guards on old/next/next.context; consider factoring the shared switch sequence.

View job run

Comment thread runtime/internal/runtime/proc_wasm.go
Comment thread runtime/internal/runtime/proc_atomic.go
Comment thread runtime/internal/runtime/proc_atomic.go
Comment thread runtime/internal/runtime/proc_wasm.go Outdated
Comment thread runtime/internal/runtime/runtime2.go Outdated
@cpunion

cpunion commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review follow-up pushed in 8882b7c9f:

  • fatal now terminates through c.Exit(2), making scheduler/CAS guards fail-fast;
  • worker exit while main is parked is classified as all goroutines are asleep, after marking the worker dead;
  • G/M/P ID origins are documented and asserted as 1/1/0 under the LLGo pre-increment-value atomic semantics;
  • shared g and Gosched documentation now describe backend-specific ownership/yield behavior;
  • the existing wasm32 and Memory64 artifacts each run normal scheduling plus the fatal deadlock path, with no extra artifact build.

Local validation passed on macOS arm64 with Go 1.26.5 and an independent Go 1.24.11 GOROOT, Emscripten 4.0.21, and Node 25.2.1. The exact Go 1.24.2 artifact path remains covered by CI because its locally downloaded GOROOT lives under GOMODCACHE, where Go rejects LLGo source overlays.

@cpunion
cpunion force-pushed the codex/wasm-single-worker-asyncify-scheduler branch from 8882b7c to 0e631e5 Compare July 28, 2026 12:29
@cpunion cpunion changed the title runtime/wasm: add single-worker Asyncify scheduler (based on #2166) runtime/wasm: add single-worker Asyncify scheduler Jul 28, 2026
@cpunion

cpunion commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator Author

Disposition of the two non-blocking review follow-ups:

  • Stack/context pooling is intentionally deferred to the later wasm hardening/performance work. It changes retained-memory, reset, and lifetime behavior; this scheduler foundation keeps allocation and retirement explicit first.
  • resumeDeadWasmG remains separate from resumeWasmG: the dead-G path must never return to or reap on the retiring stack, while the ordinary path resumes and performs deferred reaping. Factoring the small shared sequence would obscure that lifecycle invariant without removing backend state.

cpunion added 2 commits July 29, 2026 07:46
…orker-asyncify-scheduler

# Conflicts:
#	internal/build/source_patch_test.go
…orker-asyncify-scheduler

# Conflicts:
#	runtime/internal/runtime/z_default.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant