Skip to content

Commit ad59d3a

Browse files
committed
v2: bound the layer cache with max-cache-size-mb
Port the max-cache-size-mb input from v1 to release/v2. When set, buildkitd gets an extra size-bounded GC policy (all = true, keepBytes = N) after the age-based keepDuration policy, and the post step prunes the cache down to the limit with buildctl prune --all --keep-storage before the sticky disk is committed. The prune is bounded by a timeout, never fails the step, and reports the reclaimed bytes in the job lifecycle. Closes #125
1 parent 06dd9c8 commit ad59d3a

12 files changed

Lines changed: 474 additions & 19 deletions

‎README.md‎

Lines changed: 25 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -48,16 +48,17 @@ This action follows semantic versioning and supports multiple referencing patter
4848

4949
## Inputs
5050

51-
| Name | Description | Required | Default |
52-
| ---------------------- | ---------------------------------------------------------------------------------------------- | -------- | -------------- |
53-
| `buildx-version` | Buildx version (e.g., v0.23.0, latest) | No | `v0.23.0` |
54-
| `buildkit-version` | BuildKit version to install (e.g., v0.16.0, v0.18.0) | No | System default |
55-
| `platforms` | List of target platforms for build (e.g., linux/amd64,linux/arm64) | No | |
56-
| `nofallback` | If true, fail the action if Blacksmith builder setup fails | No | `false` |
57-
| `github-token` | GitHub token for GitHub API access | No | |
58-
| `skip-integrity-check` | Deprecated: the bbolt database integrity check has been removed; this input has no effect | No | `false` |
59-
| `driver-opts` | List of additional driver-specific options (e.g., env.VARIABLE=value) | No | |
60-
| `max-parallelism` | Maximum number of concurrent BuildKit RUN steps. Defaults to the number of vCPUs on the runner | No | |
51+
| Name | Description | Required | Default |
52+
| ---------------------- | -------------------------------------------------------------------------------------------------------------------------- | -------- | -------------- |
53+
| `buildx-version` | Buildx version (e.g., v0.23.0, latest) | No | `v0.23.0` |
54+
| `buildkit-version` | BuildKit version to install (e.g., v0.16.0, v0.18.0) | No | System default |
55+
| `platforms` | List of target platforms for build (e.g., linux/amd64,linux/arm64) | No | |
56+
| `nofallback` | If true, fail the action if Blacksmith builder setup fails | No | `false` |
57+
| `github-token` | GitHub token for GitHub API access | No | |
58+
| `skip-integrity-check` | Deprecated: the bbolt database integrity check has been removed; this input has no effect | No | `false` |
59+
| `driver-opts` | List of additional driver-specific options (e.g., env.VARIABLE=value) | No | |
60+
| `max-parallelism` | Maximum number of concurrent BuildKit RUN steps. Defaults to the number of vCPUs on the runner | No | |
61+
| `max-cache-size-mb` | Maximum size of the layer cache for this `cache-key`, in MB (e.g., 51200 for 50 GB). If not set, only age-based GC applies | No | |
6162

6263
## Example Workflows
6364

@@ -91,6 +92,20 @@ This action follows semantic versioning and supports multiple referencing patter
9192

9293
buildkitd runs with garbage collection enabled and reclaims cache entries that have not been used for 8 days. There is no need to run `docker buildx prune` or `buildctl prune` inside the job; if you do, `prune --all --keep-storage` only removes layers that are no longer referenced, in least-recently-used order.
9394

95+
To also cap the size of each cache, set `max-cache-size-mb`. buildkitd then garbage collects least recently used layers beyond the limit (right after it starts, so an oversized cache is trimmed at the start of the job, and after builds), and the post step prunes the cache down to the limit before it is saved. Layers still referenced by other cached layers are never removed.
96+
97+
```yaml
98+
- uses: useblacksmith/setup-docker-builder@v2
99+
with:
100+
cache-key: services/api/Dockerfile
101+
max-cache-size-mb: "51200" # keep at most 50 GB of layers for this cache-key
102+
- uses: useblacksmith/build-push-action@v2
103+
with:
104+
file: services/api/Dockerfile
105+
push: true
106+
tags: user/app:latest
107+
```
108+
94109
### Custom Docker commands
95110

96111
```yaml

‎action.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,13 @@ inputs:
3939
max-parallelism:
4040
description: "Maximum number of concurrent BuildKit RUN steps. Defaults to the number of vCPUs on the runner."
4141
required: false
42+
max-cache-size-mb:
43+
description: >-
44+
Maximum size of the Docker layer cache for this cache-key, in MB
45+
(e.g., 51200 for 50 GB). Least recently used layers beyond the limit are
46+
garbage collected during the job and pruned before the cache is saved.
47+
If not set, only the age-based garbage collection applies.
48+
required: false
4249
runs:
4350
using: node24
4451
main: dist/index.js

‎dist/index.js‎

Lines changed: 5 additions & 5 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎dist/index.js.map‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎src/driver-opts.test.ts‎

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,12 @@ import { describe, it, expect, vi, beforeEach } from "vitest";
22
import * as core from "@actions/core";
33
import * as fs from "fs";
44
import { startBuildkitd } from "./setup_builder";
5-
import { buildkitdConfigFromServer } from "./server-config";
5+
import {
6+
buildkitdConfigFromServer,
7+
DEFAULT_BUILDKITD_CONFIG,
8+
withMaxCacheSize,
9+
} from "./server-config";
10+
import * as TOML from "@iarna/toml";
611
import { execa } from "execa";
712

813
vi.mock("@actions/core");
@@ -227,4 +232,27 @@ describe("buildkitd.toml GC policy", () => {
227232
expect(toml).toContain("all = true");
228233
expect(toml).toContain("max-parallelism = 4");
229234
});
235+
236+
it("writes a size-bounded policy after the age-based one when max-cache-size-mb is set", async () => {
237+
await startBuildkitd(
238+
4,
239+
"tcp://127.0.0.1:1234",
240+
undefined,
241+
undefined,
242+
withMaxCacheSize(DEFAULT_BUILDKITD_CONFIG, 51200),
243+
);
244+
245+
const toml = writtenBuildkitdToml();
246+
// keepBytes must be a TOML integer: older buildkitd releases decode it
247+
// into an int64 and reject floats.
248+
expect(toml).toMatch(/keepBytes = 51_?200_?000_?000\n/);
249+
const parsed = TOML.parse(toml) as {
250+
worker: { oci: { gc: boolean; gcpolicy: TOML.JsonMap[] } };
251+
};
252+
expect(parsed.worker.oci.gc).toBe(true);
253+
expect(parsed.worker.oci.gcpolicy).toEqual([
254+
{ keepDuration: "192h", all: true },
255+
{ keepBytes: 51_200_000_000, all: true },
256+
]);
257+
});
230258
});

‎src/main.ts‎

Lines changed: 51 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ import {
1919
logBuildCacheContents,
2020
logBuildkitdLogTail,
2121
logDatabaseHashes,
22+
pruneBuildkitCacheToSize,
2223
writeDockerContainerBuildkitdTomlFile,
2324
parseFstrimBytes,
2425
fstrimFailureOutcome,
@@ -41,7 +42,12 @@ import {
4142
} from "./build-telemetry";
4243
import { resolveRemoteBuilderPlatforms } from "./platform-utils";
4344
import { checkPreviousStepFailures } from "./step-checker";
44-
import { runPreCommitHooks, type PrepareCommitResponse } from "./server-config";
45+
import {
46+
parseMaxCacheSizeMB,
47+
runPreCommitHooks,
48+
withMaxCacheSize,
49+
type PrepareCommitResponse,
50+
} from "./server-config";
4551
import {
4652
Metric_MetricType,
4753
BuilderMode,
@@ -205,6 +211,7 @@ export interface Inputs {
205211
"github-token": string;
206212
"driver-opts": string[];
207213
"max-parallelism": number | null;
214+
"max-cache-size-mb": number | null;
208215
}
209216

210217
async function getInputs(): Promise<Inputs> {
@@ -232,6 +239,9 @@ async function getInputs(): Promise<Inputs> {
232239
quote: false,
233240
}),
234241
"max-parallelism": maxParallelism,
242+
"max-cache-size-mb": parseMaxCacheSizeMB(
243+
core.getInput("max-cache-size-mb"),
244+
),
235245
};
236246
}
237247

@@ -387,7 +397,10 @@ async function startBlacksmithBuilder(
387397
parallelism,
388398
buildkitdPath,
389399
inputs["driver-opts"],
390-
stickyDiskSetup.buildkitdConfig,
400+
withMaxCacheSize(
401+
stickyDiskSetup.buildkitdConfig,
402+
inputs["max-cache-size-mb"],
403+
),
391404
);
392405
} finally {
393406
const buildkitdDurationMs = Date.now() - buildkitdStartTime;
@@ -405,6 +418,7 @@ async function startBlacksmithBuilder(
405418
stateHelper.setBuilderMode("blacksmith-remote");
406419
stateHelper.setFstrimTimeoutSecs(stickyDiskSetup.fstrimTimeoutSecs);
407420
stateHelper.setFstrimMinExtentBytes(stickyDiskSetup.fstrimMinExtentBytes);
421+
stateHelper.setMaxCacheSizeMB(inputs["max-cache-size-mb"]);
408422

409423
return { addr: buildkitdAddr, exposeId: stickyDiskSetup.exposeId };
410424
} catch (error) {
@@ -424,6 +438,35 @@ async function startBlacksmithBuilder(
424438
}
425439
}
426440

441+
/**
442+
* Prunes the build cache to the user's max-cache-size-mb before the sticky
443+
* disk is committed. Skipped when no limit is set or when this job's changes
444+
* will not be committed anyway.
445+
*/
446+
async function maybePruneBuildCache(
447+
buildkitdAddr: string,
448+
lifecycle: JobLifecycle,
449+
): Promise<void> {
450+
const maxCacheSizeMB = stateHelper.getMaxCacheSizeMB();
451+
if (maxCacheSizeMB === null) {
452+
return;
453+
}
454+
if (!stateHelper.getExposeId() || stateHelper.getCommitEarlyDenyReason()) {
455+
core.info(
456+
"Skipping BuildKit cache prune: the sticky disk will not be committed for this job",
457+
);
458+
return;
459+
}
460+
461+
const result = await pruneBuildkitCacheToSize(buildkitdAddr, maxCacheSizeMB);
462+
lifecycle.pruneTriggered = true;
463+
lifecycle.pruneBytes = result.reclaimedBytes;
464+
await reporter.reportMetric(
465+
Metric_MetricType.BPA_V2_PRUNE_BYTES,
466+
result.reclaimedBytes,
467+
);
468+
}
469+
427470
/**
428471
* Shuts down buildkitd only if this action instance started it.
429472
* When setup is called multiple times in one job, only the first
@@ -492,6 +535,12 @@ async function maybeShutdownBuildkitd(
492535
lifecycle,
493536
);
494537

538+
// Enforce max-cache-size-mb on what gets committed. buildkitd's size-bounded
539+
// GC policy may not have run since the last build finished (it is throttled
540+
// to once a minute), so prune explicitly, after this job's history records
541+
// have been exported and deleted.
542+
await maybePruneBuildCache(buildkitdAddr, lifecycle);
543+
495544
const buildkitdShutdownStartTime = Date.now();
496545
const wasRunning = await shutdownBuildkitd();
497546
const buildkitdShutdownDurationMs = Date.now() - buildkitdShutdownStartTime;

‎src/server-config.test.ts‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@ import * as core from "@actions/core";
33
import {
44
buildkitdConfigFromServer,
55
DEFAULT_BUILDKITD_CONFIG,
6+
parseMaxCacheSizeMB,
7+
withMaxCacheSize,
68
} from "./server-config";
79

810
vi.mock("@actions/core", () => ({
@@ -49,3 +51,65 @@ describe("buildkitdConfigFromServer", () => {
4951
},
5052
);
5153
});
54+
55+
describe("parseMaxCacheSizeMB", () => {
56+
beforeEach(() => {
57+
vi.clearAllMocks();
58+
});
59+
60+
it("returns null without warning when the input is not set", () => {
61+
expect(parseMaxCacheSizeMB("")).toBeNull();
62+
expect(parseMaxCacheSizeMB(" ")).toBeNull();
63+
expect(core.warning).not.toHaveBeenCalled();
64+
});
65+
66+
it("parses a positive integer number of MB", () => {
67+
expect(parseMaxCacheSizeMB("51200")).toBe(51200);
68+
expect(parseMaxCacheSizeMB(" 409600 ")).toBe(409600);
69+
expect(parseMaxCacheSizeMB("1")).toBe(1);
70+
});
71+
72+
it.each(["0", "-1", "1.5", "50GB", "1e5", "abc", "99999999999"])(
73+
"warns and ignores invalid value %s",
74+
(value) => {
75+
expect(parseMaxCacheSizeMB(value)).toBeNull();
76+
expect(core.warning).toHaveBeenCalledWith(
77+
expect.stringContaining("Invalid max-cache-size-mb value"),
78+
);
79+
},
80+
);
81+
});
82+
83+
describe("withMaxCacheSize", () => {
84+
it("returns the config unchanged when no limit is set", () => {
85+
expect(withMaxCacheSize(DEFAULT_BUILDKITD_CONFIG, null)).toBe(
86+
DEFAULT_BUILDKITD_CONFIG,
87+
);
88+
});
89+
90+
it("appends a size-bounded policy after the age-based default", () => {
91+
const config = withMaxCacheSize(DEFAULT_BUILDKITD_CONFIG, 51200);
92+
93+
expect(config.gc).toBe(true);
94+
expect(config.gcPolicy).toEqual([
95+
{ keepDuration: "192h", all: true },
96+
{ all: true, keepBytes: 51_200_000_000 },
97+
]);
98+
// The shared default is not mutated.
99+
expect(DEFAULT_BUILDKITD_CONFIG.gcPolicy).toEqual([
100+
{ keepDuration: "192h", all: true },
101+
]);
102+
});
103+
104+
it("keeps the backend keepDuration and gc flag", () => {
105+
const config = withMaxCacheSize(buildkitdConfigFromServer(72n), 1024);
106+
expect(config.gcPolicy).toEqual([
107+
{ keepDuration: "72h", all: true },
108+
{ all: true, keepBytes: 1_024_000_000 },
109+
]);
110+
111+
const gcOff = withMaxCacheSize({ gc: false, gcPolicy: [] }, 1024);
112+
expect(gcOff.gc).toBe(false);
113+
expect(gcOff.gcPolicy).toEqual([{ all: true, keepBytes: 1_024_000_000 }]);
114+
});
115+
});

‎src/server-config.ts‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,64 @@ export function buildkitdConfigFromServer(
9999
};
100100
}
101101

102+
// `max-cache-size-mb` uses the same decimal megabytes as
103+
// `buildctl prune --keep-storage` and the v1 input: 1 MB = 1,000,000 bytes.
104+
export const BYTES_PER_MB = 1_000_000;
105+
106+
/**
107+
* Parses the `max-cache-size-mb` input. Returns the limit in MB, or null when
108+
* the input is empty or invalid (invalid values only warn, so a typo never
109+
* fails the build). Zero is rejected: buildkit treats a zero byte limit as
110+
* "no limit" in gcpolicy but as "delete everything" in `prune --keep-storage`.
111+
*/
112+
export function parseMaxCacheSizeMB(input: string): number | null {
113+
const value = input.trim();
114+
if (value === "") {
115+
return null;
116+
}
117+
118+
const mb = /^\d+$/.test(value) ? Number(value) : Number.NaN;
119+
if (!Number.isSafeInteger(mb * BYTES_PER_MB) || mb <= 0) {
120+
core.warning(
121+
`Invalid max-cache-size-mb value '${input}', ignoring. Must be a positive integer number of MB (e.g. 51200 for 50 GB).`,
122+
);
123+
return null;
124+
}
125+
126+
return mb;
127+
}
128+
129+
/**
130+
* Adds a size-bounded GC policy to a buildkitd config. The existing policies
131+
* (the age-based keepDuration, possibly set by the backend) still run first;
132+
* the extra `all = true, keepBytes = N` policy then evicts least recently used
133+
* cache records until the cache fits in N bytes. buildkitd runs GC shortly
134+
* after it starts and after builds, so an oversized cache restored from the
135+
* sticky disk is trimmed at the start of the job as well. The `gc` flag is
136+
* left as is; the post step's prune enforces the limit either way.
137+
*
138+
* `keepBytes` is used rather than the newer `maxUsedSpace` because every
139+
* buildkit release accepts it; on buildkit >= v0.17 it maps to
140+
* `reservedSpace`, which caps usage the same way when no other space limit
141+
* is set on the policy.
142+
*/
143+
export function withMaxCacheSize(
144+
config: BuildkitdConfig,
145+
maxCacheSizeMB: number | null,
146+
): BuildkitdConfig {
147+
if (maxCacheSizeMB === null) {
148+
return config;
149+
}
150+
151+
return {
152+
...config,
153+
gcPolicy: [
154+
...(config.gcPolicy ?? []),
155+
{ all: true, keepBytes: maxCacheSizeMB * BYTES_PER_MB },
156+
],
157+
};
158+
}
159+
102160
/**
103161
* Runs an ordered list of pre-commit hooks. Returns whether the commit
104162
* should proceed based on hook results and their failure modes.

0 commit comments

Comments
 (0)