Add experimental pnpm lockfile invalidation

change/change-39d2ed64-e3c5-4189-a32d-a0c18f5ea28a.json

@@ -0,0 +1,39 @@
+{
+  "changes": [
+    {
+      "packageName": "@lage-run/lockfile",
+      "type": "patch",
+      "dependentChangeType": "patch",
+      "comment": "New experimental package that analyzes a pnpm lockfile (`lockfileVersion 9.x`) to compute a precise per-package dependency-closure signature, so cache hashing and `--since` filtering can avoid treating a lockfile change as a repo-wide invalidation.",
+      "email": "email not defined"
+    },
+    {
+      "packageName": "@lage-run/config",
+      "type": "patch",
+      "dependentChangeType": "patch",
+      "comment": "Add experimental opt-in `experimentalLockfileInvalidation` config option (`{ packageManager: \"pnpm\" }`) for smarter, per-package lockfile invalidation.",
+      "email": "email not defined"
+    },
+    {
+      "packageName": "@lage-run/hasher",
+      "type": "minor",
+      "dependentChangeType": "patch",
+      "comment": "`TargetHasher` accepts an experimental `experimentalLockfileInvalidation` option. When enabled, external dependency invalidation uses a precise per-package pnpm lockfile closure signature instead of the resolved dependency list, so only packages whose closure changed get a new cache key. Unsupported lockfiles conservatively hash their complete content into every target.",
+      "email": "email not defined"
+    },
+    {
+      "packageName": "@lage-run/cli",
+      "type": "patch",
+      "dependentChangeType": "patch",
+      "comment": "Support the experimental `experimentalLockfileInvalidation` config option so that `--since` filtering and cache hashing only invalidate packages actually affected by a pnpm lockfile change, instead of the whole graph. Only pnpm (`lockfileVersion 9.x`) is supported; other package managers/versions fall back to blanket invalidation.",
+      "email": "email not defined"
+    },
+    {
+      "packageName": "workspace-tools",
+      "type": "patch",
+      "dependentChangeType": "patch",
+      "comment": "Add `getMergeBase` git helper to resolve the merge-base commit SHA between two refs.",
+      "email": "email not defined"
+    }
+  ]
+}

docs/docs/guides/cache.md

@@ -27,3 +27,44 @@ lage build --reset-cache
 ## Cache Options
 
 Caching capability is provided by `backfill`. All of the configuration under the `cacheOptions` key is passed to `backfill`. For the complete documentation of `cacheOptions`, see the [`backfill` configuration documentation](https://www.npmjs.com/package/backfill#configuration).
+
+## Experimental: smarter lockfile invalidation
+
+A common cause of poor cache hit rates in PR builds is the lockfile. By default, `lage` treats any
+change to the lockfile as a repo-wide change, which invalidates every package's cache. Since the
+pnpm lockfile changes frequently in PRs, this means a single dependency bump can cause a full
+rebuild and total cache misses — even for packages that were not affected by the change.
+
+The experimental `experimentalLockfileInvalidation` config option makes `lage` analyze the lockfile
+and only invalidate the packages whose resolved dependency closure actually changed:
+
+```js title="/lage.config.js"
+const config = {
+  cacheOptions: {
+    environmentGlob: ["package.json", "lage.config.js", "pnpm-lock.yaml"]
+  },
+  repoWideChanges: ["pnpm-lock.yaml"],
+  experimentalLockfileInvalidation: { packageManager: "pnpm" }
+};
+```
+
+`lage` automatically removes `pnpm-lock.yaml` from global environment and repo-wide matches while
+the feature owns it, including matches from wildcard globs. You do not need to remove existing
+lockfile entries from those options.
+
+`lage` computes a stable per-package signature that captures each workspace project's entire
+resolved external dependency graph (using a memoized Merkle hash of the lockfile's shared dependency
+DAG, so the added cost is roughly proportional to the size of the lockfile — computed once per run,
+not per package). Only packages whose signature changed get a new cache key, so unaffected packages
+keep their cache hits, including reuse from the [remote cache](./remote-cache.md) across PR branches.
+
+**Only pnpm is supported**, and only the latest lockfile format (`lockfileVersion 9.x`). This
+feature depends on pnpm's strict, deterministic lockfile, which precisely describes every project's
+resolved dependency graph (including peer-dependency resolution). Package managers with looser
+lockfiles (npm, yarn) do not offer the same guarantees and are not supported. For any unsupported
+package manager or lockfile version, `lage` warns and falls back to blanket invalidation. In that
+fallback, every cache key includes the raw lockfile content, so builds are never under-invalidated.
+Top-level pnpm settings and metadata also intentionally invalidate every package.
+
+See the [configuration reference](../reference/config.md#experimental-smarter-lockfile-invalidation)
+for details.

docs/docs/guides/remote-cache.md

@@ -124,3 +124,32 @@ const config = {
 };
 module.exports = config;
 ```
+
+## Improving remote cache hit rates on lockfile changes
+
+Remote cache is most valuable in PR builds, where one branch can reuse cache entries produced by
+another. However, by default any lockfile change invalidates every package's cache key, so a single
+dependency bump in a PR causes remote cache misses across the entire repo.
+
+If you use pnpm, the experimental `experimentalLockfileInvalidation` option makes cache keys change
+only for the packages whose resolved dependency closure actually changed, so unaffected packages
+continue to hit the remote cache:
+
+```js title="/lage.config.js"
+const config = {
+  cacheOptions: {
+    environmentGlob: ["package.json", "lage.config.js", "pnpm-lock.yaml"]
+  },
+  repoWideChanges: ["pnpm-lock.yaml"],
+  experimentalLockfileInvalidation: { packageManager: "pnpm" }
+};
+```
+
+`lage` automatically removes the lockfile from global cache inputs while applying its per-package
+signature, so existing exact or wildcard environment globs do not defeat the optimization.
+
+Only pnpm (latest `lockfileVersion 9.x`) is supported; unsupported package managers or lockfile
+versions fall back to raw lockfile content in every cache key. See the
+[caching guide](./cache.md#experimental-smarter-lockfile-invalidation) and the
+[configuration reference](../reference/config.md#experimental-smarter-lockfile-invalidation) for
+details.

docs/docs/guides/scopes.md

@@ -53,3 +53,32 @@ In fact, this is so useful that `lage` has a special syntactic sugar for it:
 ## syntactic sugar for --scope build-tools --no-dependents
 lage build --to build-tools
 ```
+
+## Experimental: smarter lockfile invalidation with `--since`
+
+When you run `lage --since <ref>`, `lage` normally treats a lockfile change as a repo-wide change
+(via `repoWideChanges`) and runs **every** package. In PR builds the pnpm lockfile changes often, so
+this defeats the purpose of `--since`.
+
+The experimental `experimentalLockfileInvalidation` config option teaches `--since` to diff the old
+and new lockfile and only include the packages whose resolved dependency closure actually changed:
+
+```js title="/lage.config.js"
+const config = {
+  repoWideChanges: ["pnpm-lock.yaml"],
+  experimentalLockfileInvalidation: { packageManager: "pnpm" }
+};
+```
+
+With this enabled, a lockfile change that only affects a couple of packages will only cause those
+packages (and their dependents) to run under `--since`, instead of the entire graph. When the
+lockfile is unchanged, this adds no lockfile parsing work. `lage` automatically ignores the lockfile
+in repo-wide matches while analyzing it, including wildcard matches. Staged and unstaged lockfile
+edits receive the same precise analysis as committed changes.
+
+**Only pnpm is supported** (latest `lockfileVersion 9.x`), because it depends on pnpm's strict,
+deterministic lockfile. Unsupported package managers or lockfile versions safely fall back to the
+previous blanket behavior. Missing, added, deleted, malformed, or globally significant lockfile
+changes also fall back to all packages. See the
+[configuration reference](../reference/config.md#experimental-smarter-lockfile-invalidation) and the
+[caching guide](./cache.md#experimental-smarter-lockfile-invalidation) for details.

docs/docs/reference/config.md

@@ -117,3 +117,66 @@ const config = {
 
 module.exports = config;
 ```
+
+## Experimental: smarter lockfile invalidation
+
+By default, any change to your package manager's lockfile (e.g. `pnpm-lock.yaml`) is treated as a
+repo-wide change: it invalidates **every** package's cache and, when using `--since`, forces
+**every** package to run. This is safe but expensive — in PR builds the lockfile changes often, and
+a single dependency bump ends up rebuilding the whole graph and missing the cache (including the
+remote cache) for packages that were not actually affected.
+
+The `experimentalLockfileInvalidation` option makes `lage` analyze the lockfile to determine
+**exactly which workspace packages had their resolved dependency closure changed**, and only those
+packages (and their dependents) are invalidated. Everything else keeps its cache hits.
+
+```js title="/lage.config.js"
+/** @type {import("lage").ConfigFileOptions} */
+const config = {
+  // ...
+  experimentalLockfileInvalidation: {
+    // Only "pnpm" is supported today.
+    packageManager: "pnpm"
+  }
+};
+```
+
+When enabled, `lage` takes ownership of `pnpm-lock.yaml` handling. It excludes the lockfile from
+`repoWideChanges` and `cacheOptions.environmentGlob` matches, including wildcard matches, so existing
+configuration does not need to change:
+
+```js title="/lage.config.js"
+const config = {
+  cacheOptions: {
+    environmentGlob: ["package.json", "lage.config.js", "pnpm-lock.yaml"]
+  },
+  repoWideChanges: ["pnpm-lock.yaml"],
+  experimentalLockfileInvalidation: { packageManager: "pnpm" }
+};
+```
+
+Changes to top-level pnpm settings and metadata (such as overrides, patched dependencies, and unknown
+future fields) still invalidate every package because they can affect the entire install. Staged and
+unstaged lockfile edits are analyzed precisely. A missing, deleted, newly added, malformed, or
+unsupported lockfile safely uses the blanket fallback.
+
+### Supported package managers
+
+Only **pnpm** is supported, and only the **latest pnpm lockfile format (`lockfileVersion 9.x`)**.
+
+This feature relies on the lockfile being **strict and deterministic** — that is, it must fully and
+unambiguously describe each workspace project's entire resolved dependency graph (including
+peer-dependency resolution). pnpm's lockfile provides exactly this via its `importers` and
+`snapshots` sections, which is what lets `lage` compute a precise per-package signature. Package
+managers with looser or less deterministic lockfiles (npm, yarn) do not provide the same guarantees,
+so they are intentionally not supported here.
+
+For anything unsupported — a different package manager, an older pnpm lockfile version, or a lockfile
+that cannot be parsed — `lage` logs a warning and **safely falls back to blanket invalidation**. The
+raw lockfile content is included in every cache key, and `--since` runs every package, so builds never
+silently under-invalidate.
+
+:::caution Experimental
+This option is experimental and may change. It is opt-in and has no effect on other `lage` commands
+when disabled. See the [caching guide](../guides/cache.md) for more details.
+:::

packages/cli/package.json

@@ -26,6 +26,7 @@
     "@lage-run/cache": "workspace:^",
     "@lage-run/config": "workspace:^",
     "@lage-run/hasher": "workspace:^",
+    "@lage-run/lockfile": "workspace:^",
     "@lage-run/logger": "workspace:^",
     "@lage-run/reporters": "workspace:^",
     "@lage-run/rpc": "workspace:^",

packages/cli/src/__tests__/createTargetGraph.test.ts

@@ -110,6 +110,41 @@ describe("createTargetGraph", () => {
       },
     ]);
   });
+
+  it.each([
+    { repoWideChanged: false, expectedStagedTarget: true },
+    { repoWideChanged: true, expectedStagedTarget: false },
+  ])("uses staged targets only when changes are not repo-wide", async ({ repoWideChanged, expectedStagedTarget }) => {
+    const packageInfos: PackageInfos = {
+      foo: stubPackage({ name: "foo", scripts: ["lint"] }),
+    };
+    const targetGraph = await createTargetGraph({
+      logger: createLogger(),
+      root: ROOT,
+      dependencies: false,
+      dependents: false,
+      enableTargetConfigMerging: true,
+      enablePhantomTargetOptimization: false,
+      ignore: [],
+      pipeline: {
+        lint: {
+          stagedTarget: {},
+        },
+      },
+      repoWideChanges: ["pnpm-lock.yaml"],
+      scope: [],
+      since: "main",
+      outputs: [],
+      tasks: ["lint"],
+      packageInfos,
+      priorities: [],
+      changedFiles: ["pnpm-lock.yaml"],
+      filteredPackages: ["foo"],
+      repoWideChanged,
+    });
+
+    expect(targetGraph.targets.has("Δlint")).toBe(expectedStagedTarget);
+  });
 });
 
 const ROOT = path.resolve("/fake/root");