bazel/wasm: upgrade wasmtime from 32.0.0 to 42.0.2#30250
Closed
travisdowns wants to merge 1 commit intoredpanda-data:devfrom
Closed
bazel/wasm: upgrade wasmtime from 32.0.0 to 42.0.2#30250travisdowns wants to merge 1 commit intoredpanda-data:devfrom
travisdowns wants to merge 1 commit intoredpanda-data:devfrom
Conversation
github-actions
Bot
added
area/build
area/redpanda
area/wasm
Contributor
There was a problem hiding this comment.
Pull request overview
Upgrades the Bazel-managed Wasmtime integration to Wasmtime v42.0.2, updating the Rust toolchain and adjusting build/runtime glue to match upstream C API and build template changes.
Changes:
- Bump Wasmtime crates from 32.0.0 to 42.0.2 (Cargo.toml / Cargo.lock) and update generated
conf.htemplate substitutions. - Update the Rust toolchain version to Rust 1.91.0 (Wasmtime 42 MSRV).
- Remove deprecated Wasmtime C API async toggle call and introduce a Bazel-side
rust_alloc_shimworkaround dependency.
Reviewed changes
Copilot reviewed 7 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/v/wasm/wasmtime.cc | Removes the removed/deprecated async-support config call from runtime initialization. |
| src/v/wasm/BUILD | Adds the new rust_alloc_shim dependency to the Wasmtime C++ library build. |
| bazel/thirdparty/wasmtime.BUILD | Extends conf.h template substitutions for new Wasmtime 42 feature defines. |
| bazel/thirdparty/rust_alloc_shim.rs | Adds an “empty” Rust crate intended to pull allocator symbols into the final link. |
| bazel/thirdparty/Cargo.toml | Bumps Wasmtime-related crate versions to 42.0.2. |
| bazel/thirdparty/Cargo.lock | Updates the full Rust dependency graph lockfile for the Wasmtime upgrade. |
| bazel/thirdparty/BUILD | Introduces rust_alloc_shim and empty_alloc Bazel targets for allocator-shim workaround. |
| MODULE.bazel | Adds documentation about the allocator-shim workaround and bumps Rust toolchain version. |
Comment on lines
+1
to
+2
| // Empty crate that provides Rust allocator symbols when linked. | ||
| // See https://github.com/bazelbuild/rules_rust/issues/3459 |
There was a problem hiding this comment.
The new rust_alloc_shim crate is completely empty (comments only). If the goal is to reliably pull allocator symbols into the final link, consider adding a minimal reference that forces linking the relevant Rust runtime/alloc components; as-is, the crate itself does not define any allocator symbols and may not achieve the intended workaround under dead-code elimination.
Suggested change
| // Empty crate that provides Rust allocator symbols when linked. | |
| // See https://github.com/bazelbuild/rules_rust/issues/3459 | |
| // Minimal shim crate that forces a reference to Rust heap allocation code. | |
| // See https://github.com/bazelbuild/rules_rust/issues/3459 | |
| // | |
| // Keeping this crate comment-only does not reliably pull allocator-related | |
| // symbols into the final link under dead-code elimination. Define and retain | |
| // a tiny symbol that performs a heap allocation instead. | |
| #[no_mangle] | |
| pub extern "C" fn rust_alloc_shim_force_link() { | |
| drop(Box::new(0u8)); | |
| } | |
| #[used] | |
| static RUST_ALLOC_SHIM_FORCE_LINK: extern "C" fn() = rust_alloc_shim_force_link; |
Comment on lines
356
to
360
| "aarch64-unknown-linux-gnu", | ||
| "x86_64-unknown-linux-gnu", | ||
| ], | ||
| versions = ["1.86.0"], | ||
| versions = ["1.91.0"], | ||
| ) |
There was a problem hiding this comment.
//bazel/thirdparty:empty_alloc is added as part of the allocator-shim workaround, but this rust.toolchain(...) registration doesn’t reference it (and a repo-wide search finds no references). If the intent is to override the toolchain’s allocator library to prevent double-linking, wire empty_alloc into the toolchain configuration here; otherwise consider removing the unused target/comment.
Comment on lines
21
to
25
| cc_library( | ||
| name = "empty_alloc", | ||
| visibility = ["//visibility:public"], | ||
| ) | ||
|
|
There was a problem hiding this comment.
empty_alloc is introduced as part of the allocator shim workaround but appears unused (no references found anywhere in the repo). If it’s required for the workaround, add the missing reference from the Rust toolchain registration; otherwise remove it to avoid an extra public target that has no effect.
Suggested change
| cc_library( | |
| name = "empty_alloc", | |
| visibility = ["//visibility:public"], | |
| ) |
Comment on lines
+11
to
+13
| # The empty cc_library is used as allocator_library in the toolchain | ||
| # registration to prevent rules_rust from double-linking its own | ||
| # allocator shim. |
There was a problem hiding this comment.
The comment claims empty_alloc is used as allocator_library in the toolchain registration to prevent double-linking, but there are no references to //bazel/thirdparty:empty_alloc in the repo. Either wire it into the toolchain configuration or update/remove this comment so it doesn’t describe behavior that isn’t happening.
Suggested change
| # The empty cc_library is used as allocator_library in the toolchain | |
| # registration to prevent rules_rust from double-linking its own | |
| # allocator shim. | |
| # The empty cc_library is kept as a placeholder target for allocator-related | |
| # toolchain integration, but this file does not wire it into any toolchain | |
| # registration. |
Additional changes required by the upgrade: - Bump Rust toolchain from 1.86.0 to 1.91.0 (wasmtime 42 MSRV) - Add PULLEY and ALL_ARCH feature flags to wasmtime.BUILD conf.h template substitutions (new in wasmtime 42) - Remove wasmtime_config_async_support_set() call: async support is now implicit, the separate toggle was deprecated and removed from the C API (bytecodealliance/wasmtime#12371) - Add rust_alloc_shim workaround for Rust >= 1.87 allocator symbol changes that are not yet handled by rules_rust 0.60.0 (bazelbuild/rules_rust#3459, CORE-16131)
travisdowns
force-pushed
the
td-hermetic-wasmtime42
branch
from
fc68ab9 to
7f39714
Compare
Collaborator
CI test resultstest results on build#83524
|
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation: make the Bazel build of
//:redpandabyte-for-bytereproducible. #30187 did this for hwloc, openssl, and libxml2 via local
rules_foreign_ccbuild tweaks, but for the wasmtime / cranelift sideof things we had two local reproducibility patches that were already
fixed upstream and only required a version bump to pick up. Per the
discussion on #30187 (#30187 (comment)),
the cleaner path is to drop the patches and upgrade wasmtime instead —
which is what this PR does.
The minimum wasmtime version that carries both cranelift fixes is v42,
so this is a jump from v32.0.0 to v42.0.2. The upgrade brings along the
following required adjustments:
PULLEYandALL_ARCHfeature flags to the wasmtime.BUILDconf.h template substitutions (new in wasmtime 42).
wasmtime_config_async_support_set()call. Async supportis implicit now — the separate toggle was deprecated and removed
from the C API in Remove need for explicit
Config::async_supportknob bytecodealliance/wasmtime#12371.rust_alloc_shimworkaround. Rust >= 1.87 changedhow allocator shim symbols are emitted when linking Rust static
libraries into C++ binaries via
cc_common.link; rules_rust 0.60.0does not yet handle this (
cc_common_linkis broken in rust1.87.0bazelbuild/rules_rust#3459). The shim canbe removed once rules_rust picks up the fix — tracked in CORE-16131.
Backports Required
Release Notes