feat(android): apply binary patch updates - #154
Merged
Conversation
Binary patch updates need HDiffPatch's patch applier and zstd's decompressor compiled into the native library, so the minimal set of sources for applying a patch is vendored here. Only the apply side is vendored: libHDiffPatch/HPatch plus zstd's common and decompress directories. zstd's amd64 assembly implementation is left out because the applier builds with ZSTD_DISABLE_ASM=1. The upstream directory layout is kept because the HDiffPatch headers include each other by repository path. The tree lives at the repository root rather than under android/ or ios/ because both platforms compile the same sources: iOS through the podspec at the root, Android through externalNativeBuild. One shared copy leaves nothing to drift.
…lier HDiffPatch takes the decompressor as a plugin. Upstream ships a demo header implementing a dozen codecs, which would pull in headers for codecs the applier never sees, so this is a standalone hpatch_TDecompress implementation for zstd - the only codec CodePush patches are compressed with. The window it accepts is capped at the 2^24 the generation options use, so a corrupted frame header asking for a wider window is rejected before anything is allocated. apply_patch_host is the reference applier built from those sources for the development machine. It loads the base bundle and the patch into memory and writes the restored bundle sequentially, which is the memory contract the Android and iOS appliers follow, and it maps each failure to its own exit code so a caller can tell a corrupt patch from a mismatched base.
Adds the CLI side of the codec - generatePatch and applyPatch spawn hdiffz and hpatchz with the fixed options that define the patch format - together with a test suite that pins the format down against real bytes and real binaries. The fixtures are committed and can be regenerated deterministically by scripts/binary-patch/generate-fixtures.mjs, so the tests verify the patch bytes that ship rather than bytes produced on the fly. hdiffz and hpatchz are built from upstream sources by scripts/binary-patch/build-hdiffpatch.sh, which the suite runs on demand when the tools are missing. Two properties are pinned down because callers have to handle them: a patch carries no checksum of the base data, and its zstd streams carry no content checksums. Applying a patch to a different base of the same size, or applying a patch whose body is corrupted, can therefore report success and still produce the wrong bytes. Verifying the base and target hashes stays the caller's responsibility. The applier sources ship in the npm package so both platforms can build them, minus the host harness, which only exists to run this suite.
An absolute --output-path produced an absolute bundle directory, which the './' prefix turned into a path below the current working directory.
`bundle` and `release` accept --binary-bundle-path, the JS bundle of the target binary. With it, `release` publishes two artifacts per platform: the full bundle named after its packageHash, and `<packageHash>-patch.zip`, which carries the target bundle only as a patch against the binary's bundle plus a `codepush-binary-patch.json` manifest. Every other file is copied unchanged, so applying the patch and dropping the two patch-only files reproduces the full contents byte for byte - and therefore the same packageHash. The Hermes compilation is aligned with the base bundle through `-base-bytecode` when the app's own compiler advertises the flag, which is what keeps the patch small. A compiler without the flag only warns; a compilation that fails with it fails the release, since the base is then wrong. Both archive sizes and the saving are printed before anything is uploaded, and the full bundle is uploaded before the patch. A failed upload of either leaves the release history untouched. Carrying the patch URL in the release history is deliberately not part of this change.
The release action read `options.bundleName`, but commander stores the `-j, --js-bundle-name` flag as `options.jsBundleName`, so a custom JS bundle name never reached `release()` and the platform default was always used. Optionality is now expressed in the types instead of asserted away, so "not given" cannot pass for a name again.
The suites that generate real patches each built the tools in their own `beforeAll`, so a suite without that hook - the release flow suite - failed on a machine with no `.hdiffpatch-tools` yet, and two workers could run the same build at the same time. A jest global setup builds them once before any worker starts, which also removes the duplicated helper from the two suites that had one.
A patch is only worth publishing when it is smaller than the archive it replaces, and the CLI runs unattended in CI, so what happens otherwise is decided up front instead of being left to whoever reads the summary. `skip`, the default, warns, records the skip in the summary and releases the full bundle alone. `fail` stops the release before either artifact is uploaded, so the release history stays untouched. Equal sizes count as oversized: a patch that saves nothing still costs a client an extra download and an apply step.
The Babel config of this repository has no JSX transform - an app bundling the library transforms it with the React Native preset - so a test cannot load src/CodePush.js. Compile that one file with TypeScript instead, which turns the JSX of the decorator into React.createElement calls and leaves the rest of the module to the same downlevelling Babel would have applied.
A release published with --binary-bundle-path uploads a patch archive whose URL was logged and then dropped. Record it in the release history entry of that release, and carry it from the fetched history through the update check to the metadata the native module is handed when it downloads the update. A release without a patch says nothing about one: the field is absent from the history entry and from the update, so a release history written before binary patches existed keeps behaving exactly as it did.
A release published with a binary patch offers two archives of the same update: the full one, and a patch of the JS bundle against the bundle inside the app binary. Download the patch when the release has one, apply it to the binary's bundle and put the restored bundle where the archive left a patch, which leaves contents identical to the full archive - and therefore the folder hash check that follows unchanged. Nothing about a patch is trusted. The manifest is checked before anything is read, the paths it points at have to stay inside the archive, the base bundle is hashed before the patch is applied and the restored bundle is hashed afterwards, and the restored bytes only reach the update contents once both hashes match. A patch carries no checksum of what it produces, so those two hashes are the only thing standing between a corrupted patch and a broken app. Any failure along that path downloads the full archive instead, and does so exactly once: the fallback download is not allowed to take the patch path, so it has no patch failure of its own to fall back from and needs no counter or stored state to say so. Each failure is logged with the reason it fell back, which is what a rollout is judged from. The applier itself is the shared C code the host build compiles too, reached from CMake where it lives rather than copied in.
The android client now installs an update from its patch archive and falls back to the full one, so say so where patch bundles are documented, and record what that adds to a consumer's android build: the NDK and CMake, which compile the applier. The note about installing updates against the binary's bundle being something android cannot do is no longer true, so it now explains why the update check still does not carry the binary's hash.
Three ways an update could go wrong on the patch path, found by testing the download and install steps together instead of apart: An archive wraps its files in a single directory, and a manifest's paths are relative to that directory, so the manifest was looked for one level above where it is. Every patch update would have fallen back to the full archive. A patch URL that answers with something other than an archive - an error page served with a 200, say - took the branch that treats a download as a bare JS bundle and moves it into the package folder under the update's hash, with no patch applied and no hash ever checked, and then reported success so nothing fell back. The patch path now refuses anything that is not an archive. Applying a patch is the one path that holds a whole bundle in memory, and an OutOfMemoryError there escaped the fallback, the native module's error handling and the download task, leaving the promise unsettled - the worst answer to the failure the fallback exists for. It is now absorbed like any other patch failure, and the bundle is read into a buffer sized from the asset instead of one that grows into a second copy of itself. The download and install steps are now covered together, over a real socket with real archives, with only the applier's two seams stubbed.
HDiffPatch is pinned to v5.1.3 but its zstd dependency was cloned from the fork's default branch, so any upstream push silently changed the generator and could break a build that had been green. Fetch the pinned commit instead, and record it next to the zstd entry in the third party notices.
`hdiffz` is the one prerequisite a binary patch release has that npm install does not provide, and the error a missing tool raises names `scripts/binary-patch/build-hdiffpatch.sh` - a path the published package did not carry. Add the script to `files`, and document the one-time build, what it needs, where it installs, and the `HDIFFPATCH_TOOLS_DIR` override in both release guides.
floydkim
marked this pull request as ready for review
August 17, 2026 10:38
A manifest is untrusted input, and the size it names is what the restore reserves before reading a byte of the patch. The old 512 MiB ceiling was far above anything a release can produce: a large Hermes bundle stays under 50 MB. A manifest that does exceed the bound costs nothing beyond the full archive being downloaded instead, so 128 MiB leaves ample room while cutting what a bad manifest can ask for.
Requirements lists what an app has to have in place, and by its own wording that bullet asked for nothing: the Android Gradle Plugin installs the NDK and CMake versions it is missing. The binary patch section of the CLI README still says the library builds native code and what that needs, which is where the note belongs.
…y-patch-cli-artifacts
…h-android-apply # Conflicts: # cli/README.ko.md # cli/README.md
…h-android-apply # Conflicts: # cli/README.ko.md # cli/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fourth PR of the binary differential OTA series (stacked on
feat: add optional binary patch metadata— merge that first).The previous PR delivered
binaryPatchDownloadUrlto native download metadata without consuming it. This PR makes Android consume it: when the field is present, the client downloads the patch archive, applies the patch against the bundle embedded in the app binary, and installs the reconstructed contents — verifying them at every step. Any failure at any patch stage falls back to the existing full archive exactly once, and the fallback is structural (the full path re-enters through an internal call that cannot reach the patch branch — no counters, no persisted state). iOS is the next PR.A package without the field takes the untouched legacy path; the existing full zip, non-zip, and
hotcodepush.jsondiff flows are unchanged downstream.Trust model: the patch archive is treated as unverified input until proven otherwise.
algorithm/formatVersion, and path confinement (no entry may escape the archive) are checked first, then a size bound and available disk space.baseBundleHashbefore anything is applied.targetBundleSizeandtargetBundleHashbefore it is moved into place — this post-hash is the only defense against corrupted patch bodies (the compressed stream carries no content checksum), so it is never skipped. The existing folder-hash verification then confirms the finalpackageHash, same as a full download.base_hash_mismatch,invalid_manifest,unsupported_format,patch_apply_failed,target_verification_failed,base_bundle_unavailable) through the existing CodePush log channel, and a success logs the apply duration — the signal for validating on a staging identifier before enabling the URL in production. Temp files live under<CodePushPath>/binary-patch/and are cleaned on success, fallback, and failure, including stale leftovers from a killed process.Changes
android/app/src/main/cpp/) — a single JNI entry point (HDiffPatchNative.applyPatch) plus a CMake target that compiles the shared C sources atcpp/binarypatch/(HDiffPatch apply path, zstd decompress set, and the project's zstd adapter) by relative path — nothing is re-vendored underandroid/.ZSTD_DISABLE_ASM=1,_IS_USED_MULTITHREAD=0, no assembly files. The wrapper validates the patch header against the caller's expectations, applies with sequential writes to a temp file, and returns distinct error codes that Java maps to fallback reasons.CodePushUpdateManager— exactly two branch points: download-URL selection at the top ofdownloadPackage, and patch application right after unzip, beside the existing diff-manifest detection. The manager now takes the applicationContext(single construction site) to read the embedded bundle viaAssetManagerstraight into memory — no disk copy. A patch download that turns out not to be a zip archive (e.g. an error page served with HTTP 200) is rejected asinvalid_manifestand falls back, instead of flowing into the raw-bundle path.OutOfMemoryErroron the patch path — the only path holding a whole bundle in memory — is caught and treated as a patch failure so the streaming full download still runs; the base-bundle buffer is pre-sized to cut the allocation peak.CodePushBinaryPatch/BinaryPatchResult— the validation pipeline above as a self-contained, JVM-testable unit behind two seams (base-bundle provider, patch applier). Archive contents are resolved the same way the CLI lays them out (everything nested under a single root directory whose name is part ofpackageHash) — the integration tests caught that reading the manifest at the unzip root would have quietly sent every patch to the fallback.externalNativeBuild/CMake wiring in the library module. Consumer builds now need NDK + CMake, which React Native apps generally already satisfy; documented in the README.CodePushBinaryPatchTestdrives real files, real zips, and real SHA-256 (expected hashes computed independently of the production hash code);CodePushUpdateManagerDownloadTestgoes end-to-end through a real loopback HTTP server, the real download/unzip/restore/folder-hash pipeline, with only the two native seams stubbed — covering patch install, non-archive responses, out-of-memory fallback, and request-order assertions ([patch, full]). A package-private constructor exists solely to inject the seams.src/CodePush.jsclaiming Android cannot install binary-based updates now describes the actual behavior.Measurements
25 MB-class bytecode pair (base 22.5 MB / target 22.8 MB, patch 212 KB, target aligned with
-base-bytecode), applied through the same shared C sources this PR compiles:On Android the transient total is ≈ +33 MB (the base bundle as a Java-heap
byte[]at 22.5 MB plus ~4 MB of native decoder cache and buffers), released when the install completes. Apply time is negligible next to download time.* Simulator numbers are host-CPU approximations; a physical-device iOS measurement is an open follow-up.
Test plan
Mutation-checked: reverting the target-hash check, the fallback branch, or the fix commit's guards fails exactly the tests that cover them. On-device end-to-end scenarios (real patch bytes on a real archive) land with the E2E PR later in the series.