Relink don't rebuild: add a baseline, sound implementation that can be incrementally improved by susitsm · Pull Request #155871 · rust-lang/rust

susitsm · 2026-04-27T10:33:56Z

This PR implemenets a sound but not too useful implementation of relink don't rebuild with the unstable -Z public-api-hash flag. It currently uses the stable hash of all items in the metadata.

The goal is to give a base implementation that can be used for experimentation. It can be incrementally improved over time by removing or stable sorting items. The PR also adds new test attributes rustc_public_hash_changed and rustc_public_hash_unchanged and an example test using them.

What are non-goals for this PR: a useful, optimized implementation of RDR and public api hashing. That has many more challenges which will each require careful review.

A non exhaustive list of the challenges for the feature I ran into while trying to make the hash useful (this should probably go in a tracking issue, but I'm not aware of one)

Name resolution diagnostics use private items. Importing the private my_crate::func prints private function as the error
Stripped cfg items are included in rmeta for improved diagnostics.
Which types to include? A simple is this public is not enough. RPIT returned types and the private types used in the included mir for generic/inline functions are probably needed.
It might make sense to have multiple hashes. Rmeta generation (cargo check), could use a much simpler hash than codegen, it does not need private types from mir (or any mir at all?)

rustbot · 2026-04-27T10:33:59Z

Some changes occurred in compiler/rustc_attr_parsing

cc @jdonszelmann, @JonathanBrouwer

Some changes occurred in compiler/rustc_hir/src/attrs

cc @jdonszelmann, @JonathanBrouwer

Some changes occurred in compiler/rustc_passes/src/check_attr.rs

cc @jdonszelmann, @JonathanBrouwer

rustbot · 2026-04-27T10:34:03Z

r? @jdonszelmann

rustbot has assigned @jdonszelmann.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: compiler
compiler expanded to 73 candidates
Random selection from 20 candidates

jdonszelmann · 2026-04-28T10:24:39Z

This is a large change, might take me multiple review passes. I'll see if I can do the first today or tomorrow :)

jdonszelmann · 2026-04-28T13:34:09Z

@rustbot author

rustbot · 2026-04-28T13:34:15Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

bjorn3 · 2026-04-28T14:05:28Z

+        let mut public_api_hasher = PublicApiHasher::default();
        let tcx = self.tcx;
        let mut stats: Vec<(&'static str, usize)> = Vec::with_capacity(32);



The encode_crate_deps below should record the public api hash rather than crate hash when doing rdr, right?

View changes since the review

Short answer: you are right, that will improve it while keeping it sound. I will add that to this PR.

Long answer: one of the main goals of RDR is to enable early cutoff, including the public hash of all dependencies goes against that. We should only include public hashes of dependencies we reexport in some way, or better, only include the hash of the part we reexport. Where reexport here can mean pub use, inlinable/generic/const eval mir reachable through local inlinable/generic/const eval mir and some more stuff we are not aware of yet. Doing this correctly and maintainably is likely the single most technically challenging part of getting RDR right.

changed it to use public_api_hash, added some comments. 044529d

…ll encoded data

… rmeta without parents)

…end on public_api_hash instead of crate_hash

rustbot · 2026-04-29T10:12:26Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

susitsm · 2026-04-29T10:13:23Z

@rustbot review

susitsm · 2026-04-29T12:31:31Z

Just realized that there is already a soundness hole: the crate_hash query itself. It won't be updated downstream if only the public hash changes. I'll have to review what exactly it is used for, maybe it is as simple as replacing it with the public hash. Except for proc macros, that might need some work

jdonszelmann · 2026-04-29T13:49:02Z

+    //    should be added here as
+    //    ```
+    //    // FIXME do we need this                      // or a comment about why we need this
+    //    let _ = my_new_field;


this part is outdated now, about the let _ =

View changes since the review

jdonszelmann · 2026-04-29T13:49:59Z

+    //    `hash_crate_root_public_api`" into `encode_my_new_field`
+    // 3. Only remove/change what is hashed in a separate PR. Removing items just from the hash
+    //    should be done with extreme scrutiny. A better way might be to sort the query result
+    //    in its provider, or filter which values we encode. That also helps with rmeta size.


this should maybe be a fixme at the end

View changes since the review

jdonszelmann · 2026-04-29T13:54:16Z

        }
    }

+    // When changed, make sure to update the hashing in `hash_crate_root_public_api`


ideally, we of course made it so the hashing of that and the encoding here share some of the same source. Maybe by implementing a trait, or calling a method here that adds to the RDR hash, or returning a closure, idk. That way you don't have to think about all these comments and nonlocal subtleties

View changes since the review

that, or even a rustc lint at some point that automatically prompts you if you've made a change here and didn't update the other file. not 100% sure yet what the logic there would be, but this, though better, still seems brittle

jdonszelmann · 2026-04-29T13:56:54Z

@rustbot author

cezarbbb

Hello, I have read your code and I think it is very well written, but I have some questions I would like to confirm.

View changes since this review

cezarbbb · 2026-04-30T03:54:37Z

+            template!(List: &[r#"cfg = "...", crate_name = "...""#]),
+            |this, cx, args| {
+                this.items.extend(parse_rdr_fields(cx, args).map(|(cfg, crate_name)| {
+                    (cx.attr_span, RDRFields { cfg, crate_name, changed: false })


Am I misunderstanding something: rustc_public_hash_changed but change = false? Then the code that checks let green = !fields.changed; seems to be reversing this.
In other words, although a change has occurred, green is still set to true?

Good catch! I likely messed this up when rebasing to main and converting to attribute parsers. There is also something wrong with running the tests. cpass revisions don't run them, they need at least bpass. Likely because I based it on the CGU reuse attributes, and inserted the runner code there.

That's also exactly what I wanted to say. It is possible that none of the assertions in your test triggered a fatal error, yet the test passed silently. It's necessary to double check your test:>

Fixed this. I also moved the attribute checking code near #[rustc_clean] checking and added a bfail revision to the test to make sure it fires.

cezarbbb · 2026-04-30T03:55:18Z

+            template!(List: &[r#"cfg = "...", crate_name = "...""#]),
+            |this, cx, args| {
+                this.items.extend(parse_rdr_fields(cx, args).map(|(cfg, crate_name)| {
+                    (cx.attr_span, RDRFields { cfg, crate_name, changed: true })


Same as above.

cezarbbb · 2026-04-30T03:56:44Z

+        return;
+    }
+
+    let green = !fields.changed;


The reversal happened here.

…tributes for testing

…stc_public_hash_unchanged attributes

…when changing some rmeta encoder functions

susitsm · 2026-04-30T18:13:52Z

Did an analysis on the usage of crate_hash by renaming it and everything it touches. It is mostly replaceable by public api hash, but there is a needs_crate_hash function that needs cleanup. It mixes whether HIR hashing is needed or the full crate_hash query is needed. I'll have to open a PR for that before continuing here

…nkov Clean up `TyCtxt::needs_crate_hash` usage and rename it to `needs_hir_hash`. While reviewing `crate_hash` query usage for rust-lang#155871, the `needs_crate_hash` function turned out to be the cause of unnecessary calls to the query. The `needs_crate_hash` name is easy to mistake for the functionality of "needs the crate_hash query". This PR removes the usage of `needs_crate_hash` where it was not appropriate and renames the function to `needs_hir_hash` which better reflects its functionality.

susitsm · 2026-05-04T20:05:01Z

After diving into how the crate hash is used during dependency loading, my conclusion is, with the current resolver, it is not possible to fully skip rustc invocations for libraries where only private parts of upstream dependencies changed. The resolver uses the crate hash of the dependencies saved inside the metadata to load transitive dependencies. Leaving something out of this hash is unsound.

I recommend reading the module level documentation of locator, but here is a short example showing the problem:

my_crate -> dep1 -> transitive_dep_with_private_hash1
         -> dep2 -> transitive_dep_with_private_hash2

The direct dependencies are located by name, but the transitive dependencies are located by name + crate hash of the transitive dependencies loaded from the metadata of dep1/dep2. This allows using different versions of crates by different dependencies. Here the hash is used to uniquely identify the crate, which needs private parts.

The solution(s)

Public api hash includes the stable crate id, it should be fine (for now)

StableCrateId collisions can happen, they are actively checked for during compilation. However, if we have one, rustc terminates. They don't happen in practice, that wouldn't make rustc too reliable. From the docs of StableCrateId:

For a big crate graph with 1000 crates in it, there is
a probability of 1 in 36,890,000,000,000 of a StableCrateId collision.

so it should be fine while this is not stabilized, the amount of miscompilations should be zero. Ofc, this is only true with battle tested build systems like cargo, which use -Cmetadata properly.

Save paths + public hash instead of full hashes

We save the path of where to look for a dependency and use that with the public hash to look up dependencies. This could improve resolver speed, while sacrificing some portability of rmetas (not sure of the real world use case where this would cause a problem.) This would also allow users to replace a dependency with a different one that has the same public api hash, then just relink. Which kind of clicks with the spirit of relink-dont-rebuild.

Add some kind of fast path that only updates the crate and dependency hashes when public hash did not change

The upside to this is that we don't have to touch cargo to make the RDR feature good (would still be an improvement, but a lot less important.) All other tries for this feature included changes to cargo, to skip rustc invocations. This would need to be handled in all alternate build systems as well. If we add this fast path, there is no need to do that. Downside is that I have no idea how hard this is, but seems complicated.

I'm leaning towards save paths + public hash instead of full hashes. Any input is appreciated here.

…nkov Clean up `TyCtxt::needs_crate_hash` usage and rename it to `needs_hir_hash`. While reviewing `crate_hash` query usage for rust-lang#155871, the `needs_crate_hash` function turned out to be the cause of unnecessary calls to the query. The `needs_crate_hash` name is easy to mistake for the functionality of "needs the crate_hash query". This PR removes the usage of `needs_crate_hash` where it was not appropriate and renames the function to `needs_hir_hash` which better reflects its functionality.

rustbot added A-attributes Area: Attributes (`#[…]`, `#![…]`) S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Apr 27, 2026

rustbot assigned jdonszelmann Apr 27, 2026

This comment has been minimized.

Sign in to view

susitsm force-pushed the relink-dont-rebuild-base branch from 0e63248 to 4b676bd Compare April 27, 2026 10:47

jdonszelmann reviewed Apr 28, 2026

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 28, 2026

bjorn3 reviewed Apr 28, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

susitsm force-pushed the relink-dont-rebuild-base branch from 4b676bd to 1682f38 Compare April 29, 2026 09:55

susitsm added 9 commits April 29, 2026 12:04

rdr: add RDRHashes field to CrateRoot

f877fb4

rdr: add public_api_hash unstable option

27034fc

derive HashStable for TargetModifier

5612408

derive HashStable for DeniedPartialMitigation

9db9131

rdr: implement rmeta public api hash as the stable hash of (almost) a…

9132e4b

…ll encoded data

rdr: hash spans as if they had no parents (spans are encoded into the…

1ce1fe3

… rmeta without parents)

rdr: add the public_api_hash query. Queries provided by rmeta now dep…

bc16057

…end on public_api_hash instead of crate_hash

rdr: remove crate_hash from public api hash

280b75e

rdr: add public api hash debug logs

68bf2d4

susitsm force-pushed the relink-dont-rebuild-base branch from 1682f38 to ab0ab17 Compare April 29, 2026 10:12

rustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Apr 29, 2026

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 29, 2026

jdonszelmann reviewed Apr 29, 2026

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 29, 2026

cezarbbb reviewed Apr 30, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

susitsm added 6 commits April 30, 2026 20:04

rdr: add rustc_public_hash_unchanged and rustc_public_hash_changed at…

3170ab0

…tributes for testing

rdr: add incremental test exercising rustc_public_hash_changed and ru…

a5d94e2

…stc_public_hash_unchanged attributes

rdr: use public_api_hash when hashing dependencies

8d4b5db

rdr: elaborate on how EII-s should be included in the public api hash

bd4ea04

rdr: document the process of hashing new fields added to CrateRoot

dcfff37

rdr: add comments indicating that public api hashing must be updated …

981dee4

…when changing some rmeta encoder functions

susitsm force-pushed the relink-dont-rebuild-base branch from ab0ab17 to 981dee4 Compare April 30, 2026 18:04

susitsm mentioned this pull request May 2, 2026

Clean up TyCtxt::needs_crate_hash usage and rename it to needs_hir_hash. #156092

Open

Uh oh!

Conversation

susitsm commented Apr 27, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

A non exhaustive list of the challenges for the feature I ran into while trying to make the hash useful (this should probably go in a tracking issue, but I'm not aware of one)

Uh oh!

rustbot commented Apr 27, 2026

Uh oh!

rustbot commented Apr 27, 2026

Uh oh!

This comment has been minimized.

jdonszelmann commented Apr 28, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jdonszelmann commented Apr 28, 2026

Uh oh!

rustbot commented Apr 28, 2026

Uh oh!

bjorn3 Apr 28, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

rustbot commented Apr 29, 2026

Uh oh!

susitsm commented Apr 29, 2026

Uh oh!

susitsm commented Apr 29, 2026

Uh oh!

jdonszelmann Apr 29, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jdonszelmann Apr 29, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jdonszelmann Apr 29, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jdonszelmann commented Apr 29, 2026

Uh oh!

cezarbbb left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

susitsm commented Apr 30, 2026

Uh oh!

susitsm commented May 4, 2026

The solution(s)

Public api hash includes the stable crate id, it should be fine (for now)

susitsm commented Apr 27, 2026 •

edited by rustbot

Loading

bjorn3 Apr 28, 2026 •

edited by rustbot

Loading

jdonszelmann Apr 29, 2026 •

edited by rustbot

Loading

jdonszelmann Apr 29, 2026 •

edited by rustbot

Loading

jdonszelmann Apr 29, 2026 •

edited by rustbot

Loading

cezarbbb left a comment •

edited by rustbot

Loading