Skip to content

bazel: make build byte-for-byte reproducible #30187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
travisdowns merged 2 commits into from
Apr 23, 2026
Merged

bazel: make build byte-for-byte reproducible #30187

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
aded016
bazel/hwloc: make foreign_cc build reproducible
travisdowns Apr 15, 2026
5f19753
bazel/openssl: make foreign_cc build reproducible
travisdowns Apr 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion MODULE.bazel.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions bazel/repositories.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,8 @@ def data_dependency():
http_archive(
name = "openssl",
build_file = "//bazel/thirdparty:openssl.BUILD",
patches = ["//bazel/thirdparty:openssl-reproducible-buildinf.patch"],
patch_args = ["-p1"],
sha256 = "deae7c80cba99c4b4f940ecadb3c3338b13cb77418409238e57d7f31f2a3b736",
strip_prefix = "openssl-3.5.6",
url = "https://vectorized-public.s3.amazonaws.com/dependencies/openssl-3.5.6.tar.gz",
Expand Down
8 changes: 8 additions & 0 deletions bazel/thirdparty/hwloc.BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,17 @@ configure_make(
# Build a static library
"--disable-shared",
"--enable-static",

# Use a fixed runstatedir so the autoconf-derived path doesn't embed
# the sandbox directory into compiled objects.
"--runstatedir=/var/run/hwloc",
],
env = {
"HWLOC_BUILD_JOBS": "$(BUILD_JOBS)",
# Remap the sandbox root in __FILE__ expansions so that inlined
# headers (helper.h, plugins.h) produce deterministic assert strings.
"CFLAGS": "-ffile-prefix-map=$$EXT_BUILD_ROOT=.",
"CXXFLAGS": "-ffile-prefix-map=$$EXT_BUILD_ROOT=.",
},
lib_source = ":srcs",
out_binaries = [
Expand Down
21 changes: 21 additions & 0 deletions bazel/thirdparty/openssl-reproducible-buildinf.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
diff --git a/util/mkbuildinf.pl b/util/mkbuildinf.pl
index abcdef1..1234567 100644
--- a/util/mkbuildinf.pl
+++ b/util/mkbuildinf.pl
@@ -11,6 +11,16 @@ use warnings;
my $platform = pop @ARGV;
my $cflags = join(' ', @ARGV);
$cflags =~ s(\\)(\\\\)g;
+
+# Strip sandbox/build-root absolute paths so the output is reproducible
+# across different Bazel output bases and sandbox instances. Note: the
+# cflags modified here are only embedded as a diagnostic string (shown by
+# `openssl version -a`), not used for actual compilation.
+my $ebr = $ENV{'EXT_BUILD_ROOT'} // '';
+if ($ebr ne '') {
+ $cflags =~ s/\Q$ebr\E\/?/./g;
Copy link
Copy Markdown
Contributor

@rockwotj rockwotj Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this feels sort of sketchy 😀

I suppose our alternatives are to use the bazel build for openssl is undesirable for other reasons? I am not sure if there is a way to achieve this without the regex replace... Do you know what cflags in practice have the build root embedded?

Copy link
Copy Markdown
Member Author

@travisdowns travisdowns Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be clear, since this also sketched me out, I checked and this "cflags" isn't used for compilation (after this point), it's only used to embed the a copy of cflags in the binary probably for diagnostic output (e.g. so --version can tell you what flags you compiled with).

Copy link
Copy Markdown
Contributor

@rockwotj rockwotj Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah a comment to that effect would be helpful TBH

Copy link
Copy Markdown
Member Author

@travisdowns travisdowns Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment in the patch clarifying that the modified cflags are only embedded as a diagnostic string (shown by openssl version -a), not used for compilation.

+}
+
$cflags = "compiler: $cflags";

# Use the value of the envvar SOURCE_DATE_EPOCH, even if it's
4 changes: 4 additions & 0 deletions bazel/thirdparty/openssl.BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,10 @@ configure_make(
}),
env = {
"OPENSSL_BUILD_JOBS": "$(BUILD_JOBS)",
# Pin the build timestamp to epoch 0 for reproducible builds.
# Without this, mkbuildinf.pl embeds the current time into
# crypto/buildinf.h.
"SOURCE_DATE_EPOCH": "0",
},
lib_source = ":srcs",
out_binaries = [
Expand Down
Loading