Skip to content

Commit 4f830b3

Browse files
Add @useblacksmith/mocha beside the Vitest plugin in one pnpm workspace, so Mocha projects can run only the spec files a change affects (#1)
* Turn the repository into a pnpm workspace with the Vitest plugin under packages/vitest The code both plugins use (fs and child recording, the git anchor, the Actions cache client, the shard hash) moves to packages/shared, which is never published: each plugin compiles it into its own dist/. Releases are cut per package, from <package>-v<version> tags. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Add @useblacksmith/mocha: run only the Mocha spec files a change affects A preload records import edges, fs reads, child processes, worker threads and socket endpoints per spec file and per test; a wrapper command and a .mocharc.cjs helper select from that record. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * mocha: keep test fixtures off the CI job's real Actions cache, and print only JSON on stdout for affected --json On a Blacksmith runner ACTIONS_RESULTS_URL is set, so the fixture repositories (identical commits, identical keys) restored each other's records. With --json the cache log lines now go to stderr. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vitest: restore the git-ignored fixture file the first commit dropped test/fixture-affected/src/generated/value.json is tracked in useblacksmith/vitest despite the fixture's own .gitignore; the copy into this repository skipped it, which failed 68 tests in CI. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * mocha: a lockfile or config change drops every stored test record The change reran everything, but a test the run left out (--grep, --bail) kept its old pass and was skipped next time. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vitest: bring in useblacksmith/vitest main up to v0.6.5 (shadow marked deprecated and dropped from the README) The first commit here was taken before vitest main had #26 and the v0.6.5 release. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * mocha: remove the --shadow flag The Vitest plugin has deprecated its shadow option; the Mocha plugin should not start with one. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * mocha: refuse an option the command does not know run --shadow, now removed, was silently ignored and ran the affected selection; so was any mistyped option. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
1 parent 66fc37c commit 4f830b3

175 files changed

Lines changed: 5032 additions & 2148 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 6 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -16,28 +16,16 @@ concurrency:
1616
jobs:
1717
check:
1818
runs-on: blacksmith-2vcpu-ubuntu-2404
19-
strategy:
20-
fail-fast: false
21-
matrix:
22-
# '' is the locked version; 4.1 keeps no dependency record, so the
23-
# plugin's own is what runs there.
24-
vitest: ['', '4.1.1', '4.1']
25-
name: check (vitest ${{ matrix.vitest || 'locked' }})
2619
steps:
2720
- uses: actions/checkout@v6
2821
with:
2922
persist-credentials: false
3023
- uses: actions/setup-node@v6
3124
with:
3225
node-version-file: .nvmrc
33-
cache: npm
34-
cache-dependency-path: package-lock.json
35-
- run: npm ci
36-
- if: matrix.vitest != ''
37-
run: npm install --no-save "vitest@$VERSION" "@vitest/browser-playwright@$VERSION" "@vitest/coverage-v8@$VERSION"
38-
env:
39-
VERSION: ${{ matrix.vitest }}
40-
- run: npx playwright install --with-deps chromium
41-
- run: npm run check:types
42-
- run: npm run build
43-
- run: npm test
26+
- run: corepack enable # pnpm, at the version package.json pins
27+
- run: pnpm install --frozen-lockfile
28+
- run: pnpm --filter @useblacksmith/vitest exec playwright install --with-deps chromium
29+
- run: pnpm run check:types
30+
- run: pnpm run build
31+
- run: pnpm test

‎.github/workflows/release.yml‎

Lines changed: 19 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,16 @@
11
name: Release
22

3-
# Stages the package on npm when a v<version> tag is pushed; a maintainer then
4-
# reviews and releases it with `npm stage approve <id>` (2FA prompt). The
5-
# tag must match package.json, and a version already on the registry is never
6-
# republished. Uses npm trusted publishing (OIDC), so no token is stored; the
3+
# Stages one package on npm when a <package>-v<version> tag is pushed
4+
# (vitest-v0.6.5, mocha-v0.1.0); a maintainer then reviews and releases it
5+
# with `npm stage approve <id>` (2FA prompt). The tag must match the
6+
# package's package.json, and a version already on the registry is never
7+
# republished. Uses npm trusted publishing (OIDC), so no token is stored; each
78
# package's trusted publisher must point at this repo and workflow file.
89
on:
910
push:
1011
tags:
11-
- "v*"
12+
- "vitest-v*"
13+
- "mocha-v*"
1214

1315
permissions:
1416
contents: read
@@ -27,23 +29,27 @@ jobs:
2729
- uses: actions/setup-node@v6
2830
with:
2931
node-version-file: .nvmrc
30-
cache: npm
31-
cache-dependency-path: package-lock.json
3232
registry-url: https://registry.npmjs.org
33-
- run: npm ci
34-
- run: npx playwright install --with-deps chromium
35-
- run: npm run check:types
36-
- run: npm test
33+
- run: corepack enable # pnpm, at the version package.json pins
34+
- run: pnpm install --frozen-lockfile
35+
- name: Pick the package the tag names
36+
run: echo "PACKAGE=${GITHUB_REF_NAME%%-v*}" >> "$GITHUB_ENV"
37+
- if: env.PACKAGE == 'vitest'
38+
run: pnpm --filter @useblacksmith/vitest exec playwright install --with-deps chromium
39+
- run: pnpm --filter "./packages/${PACKAGE}" --filter ./packages/shared run check:types
40+
- run: pnpm --filter "./packages/${PACKAGE}" run test
3741
- name: Check the tag against package.json and the registry
42+
working-directory: packages/${{ env.PACKAGE }}
3843
run: |
3944
name=$(node -p "require('./package.json').name")
4045
version=$(node -p "require('./package.json').version")
41-
if [ "${GITHUB_REF_NAME}" != "v${version}" ]; then
42-
echo "tag ${GITHUB_REF_NAME} does not match package.json version ${version}" >&2
46+
if [ "${GITHUB_REF_NAME}" != "${PACKAGE}-v${version}" ]; then
47+
echo "tag ${GITHUB_REF_NAME} does not match ${name} version ${version}" >&2
4348
exit 1
4449
fi
4550
if npm view "${name}@${version}" version >/dev/null 2>&1; then
4651
echo "${name}@${version} is already published" >&2
4752
exit 1
4853
fi
4954
- run: npm stage publish --provenance --access public # prepack runs the build
55+
working-directory: packages/${{ env.PACKAGE }}

‎.gitignore‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
node_modules/
2-
test/fixture-runs-*/
2+
packages/vitest/test/fixture-runs-*/
33
dist/
44
out/

‎.npmrc‎

Lines changed: 0 additions & 3 deletions
This file was deleted.

‎README.md‎

Lines changed: 20 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -1,80 +1,27 @@
1-
# @useblacksmith/vitest
1+
# Blacksmith test plugins
22

3-
Speed up vitest execution by only running tests impacted by a change, with automatic caching when run on Blacksmith.
3+
Plugins that run only the tests a change affects. Each records what every
4+
test file depends on while it runs, and on later runs skips the files whose
5+
dependencies have not changed. On Blacksmith runners the record is kept in the
6+
Actions cache between jobs.
47

5-
> **Alpha.** The plugin is under active development; options and the cache format may change between releases.
6-
7-
## Usage
8-
9-
```sh
10-
npm install --save-dev @useblacksmith/vitest
11-
```
12-
13-
```ts
14-
// vitest.config.ts
15-
import { defineConfig } from 'vitest/config'
16-
import blacksmith from '@useblacksmith/vitest'
17-
18-
export default defineConfig({
19-
plugins: [blacksmith()],
20-
})
21-
```
22-
23-
With the plugin, vitest only runs tests impacted by changes since the last successful run. `--shard=i/n` works as usual; the plugin assigns files to shards itself so each shard's cache stays stable.
24-
25-
Requires Vitest 4.1.1 or later 4.x, or 5.1 or newer. From 5.1 the plugin selects from Vitest's own record of what each test file loaded; on 4.x, which keeps none, it records that itself.
26-
27-
### Vite+
28-
29-
Vite+ runs Vitest under `vp test`, so the plugin is loaded from the shared `vite.config.ts`:
30-
31-
```ts
32-
// vite.config.ts
33-
import 'vite-plus/test/config'
34-
import { defineConfig } from 'vite-plus'
35-
import blacksmith from '@useblacksmith/vitest'
36-
37-
export default defineConfig({
38-
plugins: [blacksmith()],
39-
})
40-
```
41-
42-
`vp test run` then runs the affected selection, including with `--shard`, test file filters, and `--config <path> --dir .` from a workspace package. Requires `vite-plus` 1.5.0 or newer, the first release that ships Vitest 5.1.
43-
44-
## Running locally
45-
46-
The same selection applies locally: a run reruns what changed on disk since the last successful run, committed or not. To run every test locally and select only in CI:
47-
48-
```ts
49-
plugins: [blacksmith({ runAllTests: !process.env.CI })],
50-
```
51-
52-
## How it works
53-
54-
The plugin uses Vitest's dependency tracking: each run records which modules and files every test depends on, and the next run compares the checkout against that record to calculate the impacted test set; a test the last successful run did not cover always runs. The record is stored in the Actions cache when run on Blacksmith, so runs on different runners pick up where the last successful run left off.
55-
56-
Each run ends with a line saying how much it skipped: `[blacksmith] filtered out 512 of 540 test files as unaffected, ran 28`.
57-
58-
## Options
59-
60-
| option | default | effect |
61-
|---|---|---|
62-
| `runAllTests` | `false` | Run every test instead of the affected selection. |
63-
| `shadow` | `false` | Run every test, but log what the affected selection would have run and which failing test files it would have skipped, i.e. failures the selection would have missed. Use it to evaluate the plugin on a repo before relying on the selection. |
64-
| `dependencies` | `{}` | Dependencies the tracking cannot see, as globs of test files mapped to the glob(s) of the files they depend on, relative to the config root (`../shared/**` reaches outside a package): `{ 'tests/**/*.spec.ts': 'fixtures/**' }`. A change to a matching file reruns the mapped tests, git-ignored files (generated code, build output) included; a glob that matches no file in the repository warns. |
65-
66-
## Caveats
67-
68-
- **Only dependencies the test reaches through node are tracked.** `import`, `require`, and `fs` are tracked automatically, in the test process, in every node process it starts through `child_process` and in every `worker_threads` worker (each gets a preloaded recorder; their own children inherit it). A child or worker records what it has loaded as it goes, so one the test kills or terminates still counts, up to the end of its test file. For any child, the command and the arguments that name files are recorded too. A child started with `env: { ..., NODE_OPTIONS: undefined }`, or a worker with `env: SHARE_ENV`, gets no recorder, only its command line. A file the test, or a node child it started, wrote before reading it is the test's own output and is not tracked (a git-ignored directory the test copies into and reads back, say); what it was produced from is. A file a non-node program wrote and the test then read, or a module the test wrote and then imported, is still tracked, and reruns the test on every fresh checkout where it is absent. Files read by `globalSetup`, by non-node programs, or build outputs a child runs in place of their sources are not; declare them with the `dependencies` option. In browser mode, vitest itself reports nothing, so the plugin records the modules the browser loaded from the project's Vite module graph instead; files a browser test fetches at run time are not tracked.
69-
- **Shards must be stable for maximal efficacy.** The plugin assigns test files to shards by a hash of their path, so a file stays in its shard as others are added or removed. A custom `test.sequence.sequencer` is honoured, but every test that moves to another shard reruns there.
70-
- **Each workflow job keeps its own record.** A job only skips tests that an earlier run of the same job in the same workflow file passed, since another job may run them under a different `env` or setup. The first run of a new or renamed job runs everything.
71-
- **A run with coverage runs every test.** With `--changed`, Vitest measures coverage for the changed files alone, so thresholds would pass without covering anything. The plugin logs a warning and selects nothing. A run that fails only its coverage thresholds vouches for no test.
8+
| Package | Test runner |
9+
| --- | --- |
10+
| [`@useblacksmith/vitest`](packages/vitest) | Vitest |
11+
| [`@useblacksmith/mocha`](packages/mocha) | Mocha |
7212

7313
## Development
7414

15+
A pnpm workspace. `packages/shared` holds the code both plugins use (file-read
16+
recording, the git anchor, the cache client); it is not published, each
17+
plugin compiles it into its own `dist/`.
18+
7519
```sh
76-
npm ci
77-
npm run check:types
78-
npm run build # dist/ (not committed; built on publish)
79-
npm test # runs the fixtures under test/ with a fake backend
20+
corepack enable
21+
pnpm install
22+
pnpm run check:types
23+
pnpm run build
24+
pnpm test
8025
```
26+
27+
Releases are cut per package: `pnpm run release <vitest|mocha> <major|minor|patch>`.

0 commit comments

Comments
 (0)