handy: cross-platform frontendDeps FOD + bindgenHook#1
handy: cross-platform frontendDeps FOD + bindgenHook#1philocalyst merged 10 commits intophilocalyst:handyfrom
Conversation
|
Thanks for this! Unfortunately.. Nixpkgs would not accept the TS, so you'd have to open ANOTHER PR for Handy, and then we could do it exactly like how Opencode does: bun install \
--cpu="*" \
--frozen-lockfile \
--filter ./packages/app \
--filter ./packages/desktop \
--filter ./packages/opencode \
--ignore-scripts \
--no-progress \
--os="*"
bun --bun ./nix/scripts/canonicalize-node-modules.ts
bun --bun ./nix/scripts/normalize-bun-binaries.tsThat would be it, however; I can handle the rest going forward, thanks so much for the effort. If you want credit, you can update this PR when it merges, otherwise I can just re-implement. |
|
Agreed, moved the scripts into the Handy repo: cjpais/Handy#1256 Once it's merged, the nixpkgs package.nix can reference them from src: bun --bun .nix/scripts/canonicalize-node-modules.ts
bun --bun .nix/scripts/normalize-bun-binaries.ts |
3548978 to
1988b96
Compare
|
@philocalyst — big update, rebased cleanly on top of your latest two commits ( TL;DR: bun's isolated install is now bit-reproducible within a platform, and the FOD hash is real. End-to-end What changed since the last review
Temporary
|
|
If possible CI runners for all platforms would be the preference, throwing fakeHash would be the right option until then, however.
If the runners are meant to be for when Handy updates, the most idiomatic thing to do would be add a custom update script, which nixpkgs would accept and then we don't need to maintain any infrastructure ourselves. There's plenty of examples in nixpkgs of this ! CI would be good for testing up until merge though..
▰▰▰▰▰
Miles Wirth 🙃
… From: Evgeny Khudoba ***@***.***>
Sent: 15 April 2026 06:28
To: philocalyst/nixpkgs ***@***.***>
Cc: Miles Wirht ***@***.***>, Mention ***@***.***>
Subject: Re: [philocalyst/nixpkgs] handy: cross-platform frontendDeps FOD + bindgenHook (PR #1)
xilec left a comment (philocalyst/nixpkgs#1)
@philocalyst — big update, rebased cleanly on top of your latest two commits (`cctools fix` and `added structured attrs`) and swapped the frontendDeps approach.
**TL;DR:** bun's isolated install is now bit-reproducible within a platform, and the FOD hash is real. End-to-end `nix-build -A handy` works on NixOS x86_64, and the helper workflow produced matching CI hashes for both `ubuntu-latest` and `macos-latest`.
### What changed since the last review
1. **Dropped `bun install --cpu="*" --os="*"`.** It can't produce a single cross-platform hash: even with every native variant downloaded, bun's per-package `.bin/<tool>` symlink still points at the host-matching binary (esbuild, rollup, tauri-cli, oxide, lightningcss, ...). One FOD hash per system is the only thing that holds up.
2. **Added a post-install orchestrator** at `.nix/scripts/normalize-install.ts` in the Handy source tree, called from the buildPhase after `bun install`. It runs three small passes:
- `canonicalize-node-modules.ts` — rebuild `.bun/node_modules/` symlinks in sorted order.
- `heal-peer-dep-bins.ts` — add missing `.bin/<peer>` entries around circular peer dependencies (`browserslist ↔ update-browserslist-db`, `eslint ↔ eslint-utils`). This was the non-obvious one: bun's installer races on those pairs and silently drops random `.bin/` entries, so the tree wasn't even stable between two runs on the same box. Closest upstream tracker: [oven-sh/bun#28147](oven-sh/bun#28147).
- `normalize-bun-binaries.ts` — rebuild each per-package `.bin/` in sorted order.
The healed entries match what bun would have produced without the race, so if bun ever fixes it upstream the pass becomes a no-op and the hash is unchanged.
3. **Filled in real hashes** for the two systems we can actually compute:
- `x86_64-linux`: `sha256-tJ6LK99dELOiR0BcsTRTt/vLyNamntujLxhBy5Xl/lc=`
- `aarch64-darwin`: `sha256-DQbogNBQ9izK5GPmoOudqiB2lJvct1vZI2U5lp3WFy8=`
`x86_64-darwin` and `aarch64-linux` are still `lib.fakeHash` — GH Actions `macos-latest` is Apple Silicon, and there's no ARM Linux runner available, so I left a TODO explaining what's needed.
4. **Helper workflow** `handy-frontend-hashes.yml` is the same shape as before: rewrites the hashes to `lib.fakeHash`, builds on ubuntu-latest + macos-latest, captures hashes into the job summary, and rebuilds for a sanity check. Triggered by push to the branch or `workflow_dispatch`. For future Handy bumps, one push is enough to get fresh hashes for both platforms.
### Temporary `src` override
`src` currently points at `xilec/Handy` at commit `681c6a9` (the head of cjpais/Handy#1256) so the orchestrator scripts are actually present in the source tree. Once #1256 lands and a Handy release containing the scripts is cut (0.8.3 or whatever comes next), this reverts to `owner = "cjpais"; tag = "v${finalAttrs.version}"` and the comment goes away. Flagged inline with a clear `# TEMPORARY:` block so it won't be forgotten.
### What I'd appreciate from you
- A look at the overall shape — particularly the buildPhase invoking `bun --bun "$PWD/.nix/scripts/normalize-install.ts"` after the usual bun install.
- When you have a moment on a Mac, the full end-to-end build (`nix-build -A handy`). I was only able to verify the hash side on CI and ran the full build locally on Linux; the `Handy.app` wrapping on darwin is your territory.
- If there's a reason to prefer a different strategy for the missing `x86_64-darwin` / `aarch64-linux` hashes (throwing vs `lib.fakeHash` vs something else), I'm happy to adjust.
Thanks for the patience on this one — took a while to figure out the race was real and not something I was causing with my own scripts.
--
Reply to this email directly or view it on GitHub:
#1 (comment)
You are receiving this because you were mentioned.
Message ID: ***@***.***>
|
|
Follow-up update, three small changes on top:
About the failing
|
`bindgenHook` reads `libc-cflags` directly from the cc-wrapper, which already contains the correct `.dev` include paths, so we can drop both the manual `LIBCLANG_PATH` / `BINDGEN_EXTRA_CLANG_ARGS` block and the `llvmPackages` input. Less surface area and less chance of the libc-vs-libc.dev mixup that usually lurks behind hand-rolled bindgen env on NixOS.
`bun install --linker=isolated` is not bit-reproducible on its own: symlink creation order in `.bun/node_modules/` and in each package's `.bin/` directory is not stable, and a timing race in bun's installer silently drops some `.bin/<peer>` entries around circular peer dependencies (e.g. `browserslist ↔ update-browserslist-db`, `eslint ↔ eslint-utils`). See `oven-sh/bun#28147` for the closest upstream tracker. Handy ships a tiny post-install orchestrator at `.nix/scripts/normalize-install.ts` (added in cjpais/Handy#1256) that runs three passes — canonicalize, heal, normalize — producing a stable, idempotent `node_modules/` tree. The buildPhase invokes that single entry point right after `bun install`. Within a platform the tree is now deterministic, but bun still only downloads the host-matching native binaries (esbuild, rollup, tauri-cli, lightningcss, tailwindcss-oxide, ...), so the hash differs between systems. Store one hash per `hostPlatform.system`; missing platforms `throw` rather than fall back to a placeholder. Currently populated: x86_64-linux sha256-tJ6LK99dELOiR0BcsTRTt/vLyNamntujLxhBy5Xl/lc= aarch64-linux sha256-S+dX6ZVgv9dexxIHoa5PxP7e0nxf/d7cKUGty5eEi8A= aarch64-darwin sha256-DQbogNBQ9izK5GPmoOudqiB2lJvct1vZI2U5lp3WFy8= `x86_64-darwin` is the only hole: GitHub Actions no longer hosts a free `macos-13` Intel image, so we cannot compute it without a self-hosted or paid runner. The throw-on-missing surfaces that fact immediately when the build is requested there. Also expose `frontendDeps` via `passthru` so the update script and other tooling can build it directly without dragging in the full handy compile. `src` is temporarily pinned to the HEAD commit of cjpais/Handy#1256 so the orchestrator scripts are actually present in the source tree; reverts to `tag = "v${finalAttrs.version}"` once that PR merges and a Handy release containing the scripts is cut.
2865096 to
852007e
Compare
852007e to
5ed3837
Compare
|
@philocalyst — applied your feedback, thanks. Rewrote the branch as three clean commits on top of your latest (
Three of four systems have real hashes:
Once a self-hosted or paid Intel Mac is available, a single On the helper CI workflowKept it out of the upstream PR per your preference. The computation was useful for proving reproducibility during development, but it lives on a separate About the failing
|
`nix-update-script` alone cannot keep the per-platform
`frontendDepsHashes` table in sync: it can refresh the entry for the
current host via `--subpackage frontendDeps`, but it does not reset
the other platforms' entries, so after a version bump the stale
hashes from the previous release stay in place and eventually pass
a hash check on any host that just rebuilds without running the
script. The custom update script at
`pkgs/by-name/ha/handy/update.sh` plugs that gap:
1. `nix-update handy` bumps `version`, `src.hash`, `cargoHash`.
2. On a version change, every `frontendDepsHashes` entry is reset
to `lib.fakeHash` so the package throws on any host until a
fresh hash is computed there.
3. `handy.passthru.frontendDeps` is rebuilt on the current host,
the "got:" line is parsed, and the matching entry is written
back with the real value.
Running the script on each target host (or via remote builders)
refreshes the table one entry at a time, without silently reusing
stale data from the previous release. The update flow is pure shell
+ `sed` + `awk`, wrapped in a `nix-shell` shebang so the required
tooling (`nix-update`, `nix`, ...) is fetched automatically when
`maintainers/scripts/update.nix --argstr package handy` is invoked.
5ed3837 to
26b6ef9
Compare
|
@philocalyst — quick note on the failing From my side I think this is ready to merge. All four |
|
Needs an update as your PR was merged. I would also check if it's possible to use the default update hook with some custom params. The code I would say is over-commented which is a no-go.. too |
|
Trimmed comments in package.nix and update.sh (9935d04). Checked Ok. Waiting on cjpais/Handy#1256 merge. |
9935d04 to
9b621cf
Compare
|
cjpais/Handy#1256 is merged. Once cjpais cuts a Handy v0.8.3 release, I'll flip |
|
Sweet! BTW the Darwin build is still stubbed...
Asked around at Handy discord and they had no advice :(
▰▰▰▰▰
Miles Wirth 🙃
… From: Evgeny Khudoba ***@***.***>
Sent: 19 April 2026 07:09
To: philocalyst/nixpkgs ***@***.***>
Cc: Miles Wirht ***@***.***>, Mention ***@***.***>
Subject: Re: [philocalyst/nixpkgs] handy: cross-platform frontendDeps FOD + bindgenHook (PR #1)
xilec left a comment (philocalyst/nixpkgs#1)
cjpais/Handy#1256 is merged.
Once cjpais cuts a Handy v0.8.3 release, I'll flip `src` from the pinned `rev` back to `tag = "v${finalAttrs.version}"`.
--
Reply to this email directly or view it on GitHub:
#1?email_source=notifications&email_token=A3MQBNBAUIAQ2VO2VPDA2AD4WTMYDA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMRXGYYDONBZHAYKM4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJNLQOJPWG33NNVSW45C7N5YGK3S7MNWGSY3L#issuecomment-4276074980
You are receiving this because you were mentioned.
Message ID: ***@***.***>
|
Backports Darwin-specific build logic from philocalyst/nixpkgs#1's package.nix into flake.nix (conditional on isDarwin: swift, cctools, cargo-tauri.hook, .app bundling, FOD frontendDeps, rpath patch) and adds a macos-latest CI workflow that runs `nix build .#handy` and diagnoses the resulting binary (file/otool/nm/size + try-run) to reproduce the cargo-tauri stub-binary issue from NixOS/nixpkgs#507754. Debug branch only — not intended for upstream.
apple-sdk_26's setup hook exposes FoundationModels.framework via SDKROOT, so build.rs compiles the real apple_intelligence.swift instead of the fallback stub — enabling Apple Intelligence at runtime on macOS 26+.
|
So you're really going for it! This is getting impressive, actually, wow. So, still no. But my guess is you can push the binary size past 3.7mb (maybe 10mb for safety) that would be the signal to look for. So you could add that check in CI, you could get a good feedback loop going. If you manage to get it up, I'll do everything in my power to get it merged quickly, this is very very cool of you. |
Without this swiftc flag the app exits with 0 immediately on nixpkgs Darwin stdenv, because the open-source ld64 picks Swift's synthetic `_main` over Rust's. The fix is one line in src-tauri/build.rs. Cargo.lock, package.json and bun.lock are unchanged between the old and new src rev, so cargoHash and frontendDepsHashes remain valid.
|
@philocalyst Quick clarification — I think two different "stubs" are getting conflated here:
I just pushed |
|
I got:
structuredAttrs is enabled
Running phase: unpackPhase
unpacking source archive /nix/store/mhwji9pjvbdnd0jviwiy13sp0k8fpmx7-source
source root is source
Executing cargoSetupPostUnpackHook
Finished cargoSetupPostUnpackHook
Running phase: patchPhase
patching file src-tauri/build.rs
Hunk #3 FAILED at 171.
1 out of 3 hunks FAILED -- saving rejects to file src-tauri/build.rs.rej
For full logs, run:
nix log /nix/store/ii3ac1dnwy2ynlkysnx2f789pz0ymadq-handy-0.8.2.drv
Last time you asked I ran the binary and it did nothing, so that's what I'm referring to.
▰▰▰▰▰
Miles Wirth 🙃
… From: Evgeny Khudoba ***@***.***>
Sent: 20 April 2026 20:38
To: philocalyst/nixpkgs ***@***.***>
Cc: Miles Wirht ***@***.***>, Mention ***@***.***>
Subject: Re: [philocalyst/nixpkgs] handy: cross-platform frontendDeps FOD + bindgenHook (PR #1)
xilec left a comment (philocalyst/nixpkgs#1)
Quick clarification — I think two different "stubs" are getting conflated here:
1. **Swift `_main` stub** — the actual startup bug. Fixed in cjpais/Handy#1316.
2. **apple-sdk** — needed to enable AI post-processing on Mac via the built-in Apple Intelligence framework.
I just pushed `d4eecf0c6` that pins `src` to the HEAD of #1316 so you can `nix-build` and verify directly. Could you give it another try on your Mac?
--
Reply to this email directly or view it on GitHub:
#1?email_source=notifications&email_token=A3MQBNACMUZOBXLLSXGYTMT4W3UKNA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMRYGU3TMNBXGEZ2M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJNLQOJPWG33NNVSW45C7N5YGK3S7MNWGSY3L#issuecomment-4285764713
You are receiving this because you were mentioned.
Message ID: ***@***.***>
|
Hunk 3 of the patch failed to apply because cjpais/Handy#1316 added `-parse-as-library` (with explanatory comment) right where the patch expected `-target` to follow `"swiftc"`. Regenerated against current src; the substantive change (drop xcrun wrapper, call swiftc directly) is unchanged.
|
@philocalyst Patch was the issue, not the source — Hunk 3 of If it works for you this time, would you mind dropping a quick note on cjpais/Handy#1316 confirming the fix? An independent macOS verification would help cjpais review and merge it faster. |
|
WOW!!! Yes!!! It works :) Overjoyed. |
vulkan-loader and onnxruntime are already in the wrapped binary's
RPATH via auto-rpath from buildInputs (DT_NEEDED → stdenv ld
wrapper). The explicit --prefix LD_LIBRARY_PATH was a no-op.
Verified with patchelf --print-rpath after nix-build: both
${onnxruntime}/lib and ${vulkan-loader}/lib remain in RPATH.
The SDKROOT/SWIFTC env-var fallbacks in build.rs are now part of cjpais/Handy#1316 alongside -parse-as-library, so the local patch is no longer needed. Bump src.rev to the new HEAD of NixOS#1316.
d5b276d to
e2b5cd3
Compare
cjpais/Handy#1316 (-parse-as-library swiftc fix + SDKROOT/SWIFTC env-var fallbacks) shipped in v0.8.3, so revert `src` from the in-flight rev pin back to `tag = "v${finalAttrs.version}"`. bun.lock unchanged between v0.8.2 and v0.8.3, so frontendDepsHashes remain valid; cargoHash bumped due to the version field in src-tauri/Cargo.toml.
|
@philocalyst v0.8.3 just shipped, so I've reverted |
|
@philocalyst the failing From my side this PR looks ready to merge. |
|
All good! Now onto merging into nixpkgs! THANK YOU for you effort. |
Summary
Three commits, rebased on top of your latest
handy-cross-platform-depsstate:handy: replace manual bindgen env with rustPlatform.bindgenHook— drops the hand-rolledLIBCLANG_PATH/BINDGEN_EXTRA_CLANG_ARGSblock and thellvmPackagesinput.bindgenHookreads the rightlibc-cflagsfrom the cc-wrapper automatically, so this is a pure cleanup.handy: reproducible bun frontendDeps via normalize-install.ts— turnsfrontendDepsinto a per-system fixed-output derivation, calls Handy's new.nix/scripts/normalize-install.tsorchestrator right afterbun install --linker=isolated, and stores one hash perhostPlatform.system. Missing systemsthrowinstead of falling back so unsupported builds fail fast with a pointer at the update script.handy: add passthru.updateScript with custom per-platform refresh— a smallupdate.shnext topackage.nixthat wrapsnix-updateand additionally keeps the per-platformfrontendDepsHashestable honest on version bumps. Pure bash +sed+awk, self-contained via anix-shellshebang.Why
bun install --linker=isolatedis not bit-reproducible out of the box, even with a frozen lockfile and a pinned bun version:.bun/node_modules/and in each package's private.bin/is not stable across hosts or filesystems.browserslist ↔ update-browserslist-db,eslint ↔ eslint-utils) hit a timing race in bun's installer: the consumer's wait on the provider is skipped to avoid a deadlock, and whichever side finishes last silently drops one or more.bin/<peer>symlinks. Same bun version + same lockfile + same platform can produce different.bin/sets between runs. Closest upstream tracker: oven-sh/bun#28147.Neither of these can be coerced away with install flags. The fix lives in the Handy repo as three small TypeScript passes, orchestrated from a single entry point that this package calls after
bun install:canonicalize-node-modules.ts— rebuilds.bun/node_modules/symlinks in sorted order, preserving bun's chosen targets.heal-peer-dep-bins.ts— for every package, reads itspeerDependencies, finds each peer'sbinfield, and adds any missing.bin/<name>entries bun's race dropped. Idempotent: becomes a no-op if bun ever fixes the race upstream.normalize-bun-binaries.ts— rebuilds every per-package.bin/in sorted order, preserving bun's targets.Having the scripts in the Handy source tree (rather than vendored into this package) matches how
opencodehandles the same class of problem, and means any bun/Handy bump can update the scripts in one place. They live behind cjpais/Handy#1256.About the single cross-platform hash idea
We originally tried
bun install --cpu="*" --os="*"to fetch all platform variants and get one universal hash. It doesn't work: bun's per-package.bin/<tool>entries point at host-matching native binaries (@esbuild/linux-x64vs@esbuild/darwin-arm64,@rollup/rollup-linux-*vs@rollup/rollup-darwin-*, same for@tailwindcss/oxide-*,@tauri-apps/cli-*,lightningcss-*), so even with every archive downloaded the.bin/targets still diverge. One hash per system is the only sensible shape.Note on bun2nix
Still not in nixpkgs (tracked in NixOS#376299). The TODO comment pointing at that issue is preserved.
Test plan
nix-build -A handy.passthru.frontendDepsis reproducible and idempotent;nix-build -A handysucceeds end-to-end with full Vulkan GPU acceleration.ubuntu-latest(x86_64-linux),ubuntu-24.04-arm(aarch64-linux),macos-latest(aarch64-darwin) — all three produce stable FOD hashes that match the values committed here.x86_64-darwinhash — GitHub Actions retired the freemacos-13Intel pool ("macos-13-us-default is not supported"). Filling it in needs either a self-hosted Intel Mac or a paid runner.frontendDepsHashes.x86_64-darwinis deliberately missing from the table so the evaluationthrows on that system with a pointer at the update script.Hashes
frontendDepsHashesentrysha256-tJ6LK99dELOiR0BcsTRTt/vLyNamntujLxhBy5Xl/lc=sha256-S+dX6ZVgv9dexxIHoa5PxP7e0nxf/d7cKUGty5eEi8A=sha256-DQbogNBQ9izK5GPmoOudqiB2lJvct1vZI2U5lp3WFy8=Update flow
passthru.updateScript = ./update.sh;— a custom script in the package directory, not anix-update-scriptwrapper.nix-update-scriptalone cannot keep the per-platformfrontendDepsHashestable in sync: it can refresh the entry for the current host, but it does not reset the other platforms' entries, so after a version bump the stale hashes from the previous release silently pass a hash check on any host that just rebuilds without running the update.The custom script:
nix-update handybumpsversion,src.hash,cargoHash.frontendDepsHashesentry is reset tolib.fakeHashso every unrefreshed hostthrows loudly instead of silently reusing a stale hash.handy.passthru.frontendDepsis rebuilt on the current host, the"got:"line is parsed, and the matching entry is written back with the real value.Running the script again on each target host (or over a remote builder) fills in the rest of the table one entry at a time. Pure
bash+sed+awk, wrapped in anix-shellshebang somaintainers/scripts/update.nix --argstr package handyinvokes it as normal — no extra tooling for the maintainer to install.No in-tree CI workflow needed.
Related
srcis temporarily pinned to the HEAD commit of that PR (rev = "681c6a9…",owner = "cjpais"); reverts totag = "v${finalAttrs.version}"once Add hdevtools NixOS/nixpkgs#1256 merges and a Handy release containing the scripts is cut.