-
Notifications
You must be signed in to change notification settings - Fork 192
deps: Update mu_msvm to v26.0.3 release #3421
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
maheeraeron
wants to merge
5
commits into
microsoft:main
Choose a base branch
from
maheeraeron:user/maheeraeron/uefi-update
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+5
−5
Open
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,147 @@ | ||
| #!/usr/bin/env python3 | ||
| """Helper for updating fetchzip/fetchurl SRI hashes in the .nix files in this folder. | ||
|
maheeraeron marked this conversation as resolved.
Outdated
|
||
|
|
||
| Given one or more URLs, this script invokes ``nix-prefetch-url --unpack`` to | ||
| download and hash the unpacked archive contents (matching what ``fetchzip`` | ||
| does), then converts the resulting Nix base32 hash into the SRI format | ||
| (``sha256-<base64>=``) used by these .nix files. | ||
|
maheeraeron marked this conversation as resolved.
Outdated
|
||
|
|
||
| Usage: | ||
| # Prefetch one or more URLs and print SRI hashes. | ||
| ./update_hashes.py <url> [<url> ...] | ||
|
|
||
| # Convert already-computed Nix base32 hashes to SRI without re-downloading. | ||
| ./update_hashes.py --convert <nix32-hash> [<nix32-hash> ...] | ||
|
|
||
| Requires ``nix-prefetch-url`` on PATH (``sudo apt install nix-bin`` on Ubuntu / | ||
| WSL). On a multi-user Nix install you may need ``sudo`` to access the daemon | ||
| socket; in that case prefix the command with ``sudo``. | ||
|
|
||
| Example: | ||
| sudo ./update_hashes.py \\ | ||
| https://github.com/microsoft/mu_msvm/releases/download/v26.0.3/RELEASE-X64-VS2022-artifacts.tar.gz \\ | ||
| https://github.com/microsoft/mu_msvm/releases/download/v26.0.3/RELEASE-AARCH64-CLANGPDB-artifacts.tar.gz | ||
| """ | ||
|
|
||
| # Copyright (c) Microsoft Corporation. | ||
| # Licensed under the MIT License. | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import argparse | ||
| import base64 | ||
| import shutil | ||
| import subprocess | ||
| import sys | ||
|
|
||
| # Nix base32 alphabet: omits 'e', 'o', 'u', 't' to avoid spelling words. | ||
| _NIX32_ALPHABET = "0123456789abcdfghijklmnpqrsvwxyz" | ||
|
|
||
|
|
||
| def nix32_to_bytes(s: str, hashlen: int = 32) -> bytes: | ||
| """Decode a Nix base32 string (the format printed by ``nix-prefetch-url``) | ||
| into its raw bytes. | ||
|
|
||
| Nix base32 encodes characters in reverse order vs. position: char ``n`` of | ||
| the encoded string holds bits ``[5n .. 5n+4]`` of the hash, but the string | ||
| itself is reversed before encoding, so the most-significant bits appear | ||
| first when read left-to-right. | ||
| """ | ||
| if len(s) != (hashlen * 8 - 1) // 5 + 1: | ||
| raise ValueError( | ||
| f"unexpected nix32 length {len(s)} for {hashlen}-byte hash" | ||
| ) | ||
|
|
||
| out = bytearray(hashlen) | ||
| # Reverse the string so character index 0 corresponds to the lowest bits. | ||
| for n, c in enumerate(reversed(s)): | ||
| try: | ||
| digit = _NIX32_ALPHABET.index(c) | ||
| except ValueError as e: | ||
| raise ValueError(f"invalid nix32 character: {c!r}") from e | ||
|
maheeraeron marked this conversation as resolved.
Outdated
|
||
| b = 5 * n | ||
| i, j = b // 8, b % 8 | ||
| out[i] |= (digit << j) & 0xFF | ||
| if i + 1 < hashlen: | ||
| out[i + 1] |= (digit >> (8 - j)) & 0xFF | ||
| return bytes(out) | ||
|
|
||
|
|
||
| def nix32_to_sri(nix32: str) -> str: | ||
| """Convert a Nix base32 sha256 hash to SRI (``sha256-<base64>``) format.""" | ||
| raw = nix32_to_bytes(nix32) | ||
| return "sha256-" + base64.b64encode(raw).decode("ascii") | ||
|
|
||
|
|
||
| def prefetch(url: str) -> str: | ||
| """Download ``url`` via ``nix-prefetch-url --unpack`` and return the hash. | ||
|
|
||
| The returned value is the Nix base32 string printed on the last line of | ||
| ``nix-prefetch-url``'s stdout. | ||
| """ | ||
| if shutil.which("nix-prefetch-url") is None: | ||
| sys.exit( | ||
| "error: nix-prefetch-url not found on PATH.\n" | ||
| "Install it with: sudo apt install nix-bin" | ||
| ) | ||
|
|
||
| result = subprocess.run( | ||
| ["nix-prefetch-url", "--unpack", "--type", "sha256", url], | ||
| check=True, | ||
| capture_output=True, | ||
| text=True, | ||
| ) | ||
| # nix-prefetch-url prints progress on stderr and the hash as the last | ||
| # non-empty line of stdout. | ||
| lines = [line for line in result.stdout.splitlines() if line.strip()] | ||
| if not lines: | ||
| sys.exit(f"error: nix-prefetch-url produced no output for {url}") | ||
| return lines[-1].strip() | ||
|
maheeraeron marked this conversation as resolved.
Outdated
|
||
|
|
||
|
|
||
| def main() -> int: | ||
| parser = argparse.ArgumentParser( | ||
| description=( | ||
| "Prefetch URLs and print SRI hashes suitable for fetchzip in the " | ||
| ".nix files in this folder." | ||
| ) | ||
| ) | ||
| parser.add_argument( | ||
| "args", | ||
| nargs="*", | ||
| metavar="URL_OR_HASH", | ||
| help=( | ||
| "URLs to prefetch with `nix-prefetch-url --unpack`, or, with " | ||
| "--convert, Nix base32 hashes to convert to SRI." | ||
| ), | ||
| ) | ||
| parser.add_argument( | ||
| "--convert", | ||
| action="store_true", | ||
| help=( | ||
| "Treat positional arguments as Nix base32 hashes and convert " | ||
| "them to SRI without re-downloading." | ||
| ), | ||
| ) | ||
| parsed = parser.parse_args() | ||
|
|
||
| if not parsed.args: | ||
| parser.print_help() | ||
| return 2 | ||
|
|
||
| if parsed.convert: | ||
| for h in parsed.args: | ||
| print(nix32_to_sri(h)) | ||
| return 0 | ||
|
|
||
| for url in parsed.args: | ||
| print(f"# {url}", file=sys.stderr) | ||
| nix32 = prefetch(url) | ||
| sri = nix32_to_sri(nix32) | ||
| print(sri) | ||
|
|
||
| return 0 | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main()) | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
probably real
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are the hashes coming from the sha256 entries here?:
https://github.com/microsoft/mu_msvm/releases/tag/v26.0.3
RELEASE-X64-VS2022 is:
sha256:d8e320f89f0e4871e5dfac6433dde62c4ac96988a192fcd713f12a96ff6c07a5RELEASE-AARCH64-CLANGPDB is:
sha256:255fbd69f1cf1d3d769d8bad20e1533fe1fcde3436f53bd94d8d387fbbad70f6The format looks different than what is on this file right now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like it comes from some nix tool that you can get in linux, but there is some more conversion needed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
right you need to build nix locally otherwise this update will break it. if you're not ready to update this, you should hold off.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ran
sudo nix-shell --puretwice after emtpying those hash strings. The first one failed for x64 with:And the second time fails for arm64 (after populating x64):
These hashes match the state of what's in the PR today, so should be good to go