diff --git a/SECURITY.md b/SECURITY.md index b09425b688..deb7bcab28 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -48,6 +48,12 @@ The daemon also supports an optional shared-secret password (set via `auth.passw Connected clients are trusted operators of the daemon user. File previews follow that authority: a preview request may read any regular file the daemon process can read, while keeping path normalization and symlink checks in the daemon file service. Workspace-relative paths remain a UI convenience, not a security boundary. +An explicit `symlink ` entry in a repository's .worktreeinclude intentionally gives a +Paseo-created worktree live access to that source-checkout file or directory. It is useful for +local dependencies and caches, but it weakens the usual worktree isolation: agents and lifecycle +scripts can modify the source through the link. Paseo validates entries and refuses traversal or +destination-link escapes, but the linked source is a deliberate shared-data boundary. + If you expose the daemon beyond loopback, such as by binding to `0.0.0.0`, forwarding it through a tunnel or reverse proxy, or publishing it from a Docker container, you are responsible for restricting and securing that access. Setting a password is strongly recommended in that case. In Docker, the official image runs the daemon and agents as the non-root diff --git a/docs/development.md b/docs/development.md index 9eed78a00e..5960332584 100644 --- a/docs/development.md +++ b/docs/development.md @@ -29,6 +29,7 @@ Root checkout dev is intentionally split across terminals: - **Repo dev scripts** default to `$ROOT/.dev/paseo-home`, where `$ROOT` is the current checkout or worktree root. This keeps all dev state scoped to the checkout instead of the packaged desktop app. - **`npm run cli -- ...`** runs through the same dev-home wrapper as the dev scripts, so the in-repo CLI automatically targets the current checkout's `.dev/paseo-home` and configured dev daemon endpoint. - **Paseo-created worktrees** seed `$PASEO_WORKTREE_PATH/.dev/paseo-home` from `$PASEO_SOURCE_CHECKOUT_PATH/.dev/paseo-home` by copying durable JSON metadata. Runtime files like pid files, sockets, and logs are not copied. +- **Paseo-created worktrees** read `.worktreeinclude` from the live source checkout before creation. Bare paths and `copy ` copy a snapshot into the new worktree; `symlink ` creates a live source link. Missing paths, malformed entries, unsafe paths, incompatible include overlaps, destination conflicts, unavailable platform links, and ordinary read/write failures are skipped individually and reported in the daemon log, so the rest of the plan still runs. A source symlink is allowed only when its resolved target remains inside the active source checkout. If that checkout is itself Paseo-managed, its own paths remain eligible while other managed worktree paths stay protected; `copy` snapshots that resolved target, while `symlink` links directly to it. Hard links are ordinary files. Each include is staged before it is committed; Paseo aborts creation only if it cannot safely clean up partial materialization state (or Git/worktree setup itself fails). Materialization finishes before `worktree.setup` runs. - **This repo's worktree setup** also best-effort seeds `packages/app/ios` and the newest `.dev/ios-build` entry from the source checkout so iOS simulator services can reuse native project and Xcode cache state when it is safe enough to do so. Override knobs: diff --git a/packages/server/src/server/worktree-session.ts b/packages/server/src/server/worktree-session.ts index 5573a945f8..0fdff84b91 100644 --- a/packages/server/src/server/worktree-session.ts +++ b/packages/server/src/server/worktree-session.ts @@ -605,6 +605,17 @@ export async function createPaseoWorktreeWorkflow( const workspace = createdWorktree.workspace; const setupContinuation = options?.setupContinuation ?? { kind: "workspace" }; + if (createdWorktree.created && createdWorktree.worktree.worktreeIncludeSummary?.skipped.length) { + dependencies.sessionLogger.warn( + { + materialized: createdWorktree.worktree.worktreeIncludeSummary.materialized, + skipped: createdWorktree.worktree.worktreeIncludeSummary.skipped, + worktreePath: createdWorktree.worktree.worktreePath, + }, + "Worktree include completed with skipped entries", + ); + } + setTimeout(() => { if (input.firstAgentContext) { dependencies.autoNameWorkspaceBranchForFirstAgent({ diff --git a/packages/server/src/utils/worktree-include.test.ts b/packages/server/src/utils/worktree-include.test.ts new file mode 100644 index 0000000000..5327b79aad --- /dev/null +++ b/packages/server/src/utils/worktree-include.test.ts @@ -0,0 +1,502 @@ +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { + existsSync, + chmodSync, + lstatSync, + linkSync, + mkdirSync, + mkdtempSync, + readdirSync, + readFileSync, + readlinkSync, + realpathSync, + rmSync, + symlinkSync, + writeFileSync, +} from "fs"; +import { tmpdir } from "os"; +import { dirname, join, relative } from "path"; +import { isPlatform } from "../test-utils/platform.js"; +import { materializeWorktreeIncludePlan, readWorktreeIncludePlan } from "./worktree-include.js"; + +describe("worktree include planning", () => { + let tempDir: string; + let sourceRoot: string; + + beforeEach(() => { + tempDir = mkdtempSync(join(tmpdir(), "worktree-include-test-")); + sourceRoot = join(tempDir, "source"); + mkdirSync(sourceRoot); + }); + + afterEach(() => { + rmSync(tempDir, { recursive: true, force: true }); + }); + + it("copies bare entries and accepts explicit copy and symlink modes", async () => { + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + [".env.local", ".cache/**", "symlink shared-state", "copy packages/*/.runtime.env", ""].join( + "\n", + ), + ); + writeFileSync(join(sourceRoot, ".env.local"), "source\n"); + mkdirSync(join(sourceRoot, ".cache"), { recursive: true }); + writeFileSync(join(sourceRoot, ".cache", "state.txt"), "cache\n"); + mkdirSync(join(sourceRoot, "shared-state"), { recursive: true }); + writeFileSync(join(sourceRoot, "shared-state", "state.txt"), "shared\n"); + mkdirSync(join(sourceRoot, "packages", "api"), { recursive: true }); + writeFileSync(join(sourceRoot, "packages", "api", ".runtime.env"), "api\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toHaveLength(4); + expect(plan.materializations).toEqual( + expect.arrayContaining([ + expect.objectContaining({ mode: "copy", relativePath: ".env.local", sourceKind: "file" }), + expect.objectContaining({ mode: "copy", relativePath: ".cache", sourceKind: "directory" }), + expect.objectContaining({ + mode: "symlink", + relativePath: "shared-state", + sourceKind: "directory", + }), + expect.objectContaining({ + mode: "copy", + relativePath: "packages/api/.runtime.env", + sourceKind: "file", + }), + ]), + ); + }); + + it("skips invalid and conflicting entries while retaining safe entries", async () => { + writeFileSync(join(sourceRoot, "shared"), "source\n"); + writeFileSync(join(sourceRoot, ".env"), "source\n"); + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + ["../outside", "symlink", "shared", "symlink shared", ".env", ""].join("\n"), + ); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([ + expect.objectContaining({ mode: "copy", relativePath: ".env", sourceKind: "file" }), + ]); + expect(plan.skipped).toEqual( + expect.arrayContaining([ + expect.objectContaining({ lineNumber: 1, raw: "../outside", reason: "invalid" }), + expect.objectContaining({ lineNumber: 2, raw: "symlink", reason: "invalid" }), + expect.objectContaining({ lineNumber: 3, raw: "shared", reason: "conflict" }), + expect.objectContaining({ lineNumber: 4, raw: "symlink shared", reason: "conflict" }), + ]), + ); + }); + + it("skips missing entries while retaining existing literal and glob matches", async () => { + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + [".env", ".env.local", "config/*.env", "missing/**", ""].join("\n"), + ); + writeFileSync(join(sourceRoot, ".env"), "source\n"); + mkdirSync(join(sourceRoot, "config"), { recursive: true }); + writeFileSync(join(sourceRoot, "config", "runtime.env"), "runtime\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([ + expect.objectContaining({ mode: "copy", relativePath: ".env", sourceKind: "file" }), + expect.objectContaining({ + mode: "copy", + relativePath: "config/runtime.env", + sourceKind: "file", + }), + ]); + expect(plan.skipped).toEqual( + expect.arrayContaining([ + expect.objectContaining({ raw: ".env.local", reason: "missing" }), + expect.objectContaining({ raw: "missing/**", reason: "missing" }), + ]), + ); + }); + + it("skips directory copies that overlap protected worktree paths", async () => { + const protectedWorktreeRoot = join(sourceRoot, ".dev", "paseo-home", "worktrees", "project"); + mkdirSync(protectedWorktreeRoot, { recursive: true }); + writeFileSync(join(sourceRoot, ".worktreeinclude"), ".dev/**\n"); + + const plan = await readWorktreeIncludePlan({ + sourceRoot, + excludedSourceRoots: [protectedWorktreeRoot], + }); + + expect(plan.materializations).toEqual([]); + expect(plan.skipped).toEqual([expect.objectContaining({ raw: ".dev/**", reason: "unsafe" })]); + }); + + it("allows includes from a source worktree inside the protected worktree root", async () => { + const managedWorktreesRoot = join(tempDir, "paseo-home", "worktrees", "project"); + sourceRoot = join(managedWorktreesRoot, "source-worktree"); + mkdirSync(sourceRoot, { recursive: true }); + writeFileSync(join(sourceRoot, ".worktreeinclude"), ".env\n"); + writeFileSync(join(sourceRoot, ".env"), "source\n"); + + const plan = await readWorktreeIncludePlan({ + sourceRoot, + excludedSourceRoots: [managedWorktreesRoot], + }); + + expect(plan.materializations).toEqual([ + expect.objectContaining({ relativePath: ".env", sourceKind: "file" }), + ]); + expect(plan.skipped).toEqual([]); + }); + + it.skipIf(isPlatform("win32"))( + "skips protected paths reached through a symlink alias", + async () => { + const sourceAlias = join(tempDir, "source-alias"); + symlinkSync(sourceRoot, sourceAlias, "dir"); + mkdirSync(join(sourceRoot, ".dev", "paseo-home", "worktrees", "project"), { + recursive: true, + }); + writeFileSync(join(sourceRoot, ".worktreeinclude"), ".dev/**\n"); + + const plan = await readWorktreeIncludePlan({ + sourceRoot, + excludedSourceRoots: [join(sourceAlias, ".dev", "paseo-home", "worktrees", "project")], + }); + + expect(plan.materializations).toEqual([]); + expect(plan.skipped).toEqual([expect.objectContaining({ raw: ".dev/**", reason: "unsafe" })]); + }, + ); + + it.skipIf(isPlatform("win32"))( + "skips external source links while retaining safe entries", + async () => { + const outsidePath = join(tempDir, "outside.txt"); + writeFileSync(outsidePath, "outside\n"); + writeFileSync(join(sourceRoot, "safe.txt"), "safe\n"); + symlinkSync(outsidePath, join(sourceRoot, "linked.txt")); + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + ["safe.txt", "linked.txt", ""].join("\n"), + ); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([ + expect.objectContaining({ relativePath: "safe.txt", sourceKind: "file" }), + ]); + expect(plan.skipped).toEqual([ + expect.objectContaining({ raw: "linked.txt", reason: "unsafe" }), + ]); + }, + ); + + it.skipIf(isPlatform("win32"))( + "does not scan unrelated directories for bounded globs", + async () => { + writeFileSync(join(sourceRoot, ".worktreeinclude"), "packages/*/.runtime.env\n"); + mkdirSync(join(sourceRoot, "packages", "api"), { recursive: true }); + writeFileSync(join(sourceRoot, "packages", "api", ".runtime.env"), "api\n"); + const unreadableDirectory = join(sourceRoot, "unrelated"); + mkdirSync(unreadableDirectory); + chmodSync(unreadableDirectory, 0o000); + + try { + const plan = await readWorktreeIncludePlan({ sourceRoot }); + expect(plan.materializations).toEqual([ + expect.objectContaining({ relativePath: "packages/api/.runtime.env" }), + ]); + } finally { + chmodSync(unreadableDirectory, 0o700); + } + }, + ); + + it.skipIf(isPlatform("win32") || process.getuid?.() === 0)( + "skips inaccessible entries while retaining safe paths", + async () => { + const inaccessibleDirectory = join(sourceRoot, "private"); + mkdirSync(inaccessibleDirectory); + writeFileSync(join(inaccessibleDirectory, "secret.txt"), "secret\n"); + writeFileSync(join(sourceRoot, "safe.txt"), "safe\n"); + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + ["private/**", "private/*", "safe.txt", ""].join("\n"), + ); + chmodSync(inaccessibleDirectory, 0o000); + + try { + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([ + expect.objectContaining({ relativePath: "safe.txt", sourceKind: "file" }), + ]); + expect(plan.skipped).toEqual( + expect.arrayContaining([ + expect.objectContaining({ raw: "private/**", reason: "materialization" }), + expect.objectContaining({ raw: "private/*", reason: "materialization" }), + ]), + ); + } finally { + chmodSync(inaccessibleDirectory, 0o700); + } + }, + ); +}); + +describe.skipIf(isPlatform("win32"))("worktree include materialization", () => { + let tempDir: string; + let sourceRoot: string; + let worktreeRoot: string; + + beforeEach(() => { + tempDir = mkdtempSync(join(tmpdir(), "worktree-include-materialize-test-")); + sourceRoot = join(tempDir, "source"); + worktreeRoot = join(tempDir, "worktree"); + mkdirSync(sourceRoot); + mkdirSync(worktreeRoot); + }); + + afterEach(() => { + rmSync(tempDir, { recursive: true, force: true }); + }); + + it("copies snapshots, links shared paths, and is idempotent", async () => { + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + ["copy.txt", "copy-dir/**", "symlink linked.txt", "symlink linked-dir"].join("\n"), + ); + writeFileSync(join(sourceRoot, "copy.txt"), "copy-v1\n"); + mkdirSync(join(sourceRoot, "copy-dir"), { recursive: true }); + writeFileSync(join(sourceRoot, "copy-dir", "state.txt"), "copy-dir-v1\n"); + writeFileSync(join(sourceRoot, "linked.txt"), "linked-v1\n"); + mkdirSync(join(sourceRoot, "linked-dir"), { recursive: true }); + writeFileSync(join(sourceRoot, "linked-dir", "state.txt"), "linked-dir-v1\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + expect(lstatSync(join(worktreeRoot, "copy.txt")).isSymbolicLink()).toBe(false); + expect(lstatSync(join(worktreeRoot, "copy-dir")).isSymbolicLink()).toBe(false); + expect(lstatSync(join(worktreeRoot, "linked.txt")).isSymbolicLink()).toBe(true); + expect(lstatSync(join(worktreeRoot, "linked-dir")).isSymbolicLink()).toBe(true); + + writeFileSync(join(sourceRoot, "copy.txt"), "copy-v2\n"); + writeFileSync(join(sourceRoot, "copy-dir", "state.txt"), "copy-dir-v2\n"); + writeFileSync(join(sourceRoot, "linked.txt"), "linked-v2\n"); + writeFileSync(join(sourceRoot, "linked-dir", "state.txt"), "linked-dir-v2\n"); + + expect(readFileSync(join(worktreeRoot, "copy.txt"), "utf8")).toBe("copy-v1\n"); + expect(readFileSync(join(worktreeRoot, "copy-dir", "state.txt"), "utf8")).toBe("copy-dir-v1\n"); + expect(readFileSync(join(worktreeRoot, "linked.txt"), "utf8")).toBe("linked-v2\n"); + expect(readFileSync(join(worktreeRoot, "linked-dir", "state.txt"), "utf8")).toBe( + "linked-dir-v2\n", + ); + + await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + expect(readFileSync(join(worktreeRoot, "copy.txt"), "utf8")).toBe("copy-v2\n"); + expect(readFileSync(join(worktreeRoot, "copy-dir", "state.txt"), "utf8")).toBe("copy-dir-v2\n"); + }); + + it("skips a destination parent symlink without writing through it", async () => { + mkdirSync(join(sourceRoot, "config"), { recursive: true }); + writeFileSync(join(sourceRoot, "config", "local.json"), "{}\n"); + writeFileSync(join(sourceRoot, "safe.txt"), "safe\n"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "config/local.json\nsafe.txt\n"); + + const outsideRoot = join(tempDir, "outside"); + mkdirSync(outsideRoot); + symlinkSync(outsideRoot, join(worktreeRoot, "config")); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + expect(existsSync(join(outsideRoot, "local.json"))).toBe(false); + expect(readFileSync(join(worktreeRoot, "safe.txt"), "utf8")).toBe("safe\n"); + expect(result).toMatchObject({ materialized: 1 }); + expect(result.skipped).toEqual([ + expect.objectContaining({ raw: "config/local.json", reason: "conflict" }), + ]); + expect(readdirSync(worktreeRoot)).not.toContain( + expect.stringMatching(/^\.paseo-worktreeinclude-/), + ); + }); + + it("copies resolved source links and creates direct live links", async () => { + const targetPath = join(sourceRoot, "target.txt"); + const targetDirectoryPath = join(sourceRoot, "target-directory"); + writeFileSync(targetPath, "v1\n"); + mkdirSync(targetDirectoryPath); + writeFileSync(join(targetDirectoryPath, "state.txt"), "v1\n"); + symlinkSync(targetPath, join(sourceRoot, "copy-link.txt")); + symlinkSync(targetPath, join(sourceRoot, "live-link.txt")); + symlinkSync(targetDirectoryPath, join(sourceRoot, "copy-directory-link"), "dir"); + symlinkSync(targetDirectoryPath, join(sourceRoot, "live-directory-link"), "dir"); + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + [ + "copy-link.txt", + "symlink live-link.txt", + "copy-directory-link", + "symlink live-directory-link", + "", + ].join("\n"), + ); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + const copiedPath = join(worktreeRoot, "copy-link.txt"); + const linkedPath = join(worktreeRoot, "live-link.txt"); + const copiedDirectoryPath = join(worktreeRoot, "copy-directory-link"); + const linkedDirectoryPath = join(worktreeRoot, "live-directory-link"); + const canonicalWorktreeRoot = realpathSync(worktreeRoot); + expect(result).toMatchObject({ materialized: 4, skipped: [] }); + expect(lstatSync(copiedPath).isSymbolicLink()).toBe(false); + expect(lstatSync(linkedPath).isSymbolicLink()).toBe(true); + expect(lstatSync(copiedDirectoryPath).isSymbolicLink()).toBe(false); + expect(lstatSync(linkedDirectoryPath).isSymbolicLink()).toBe(true); + expect(readlinkSync(linkedPath)).toBe( + relative(dirname(join(canonicalWorktreeRoot, "live-link.txt")), realpathSync(targetPath)), + ); + expect(readlinkSync(linkedDirectoryPath)).toBe( + relative( + dirname(join(canonicalWorktreeRoot, "live-directory-link")), + realpathSync(targetDirectoryPath), + ), + ); + expect(realpathSync(linkedPath)).toBe(realpathSync(targetPath)); + expect(realpathSync(linkedDirectoryPath)).toBe(realpathSync(targetDirectoryPath)); + + writeFileSync(targetPath, "v2\n"); + writeFileSync(join(targetDirectoryPath, "state.txt"), "v2\n"); + expect(readFileSync(copiedPath, "utf8")).toBe("v1\n"); + expect(readFileSync(linkedPath, "utf8")).toBe("v2\n"); + expect(readFileSync(join(copiedDirectoryPath, "state.txt"), "utf8")).toBe("v1\n"); + expect(readFileSync(join(linkedDirectoryPath, "state.txt"), "utf8")).toBe("v2\n"); + }); + + it("treats hard links as ordinary files", async () => { + const targetPath = join(sourceRoot, "target.txt"); + const hardLinkPath = join(sourceRoot, "hard-link.txt"); + writeFileSync(targetPath, "v1\n"); + linkSync(targetPath, hardLinkPath); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "hard-link.txt\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + const copiedPath = join(worktreeRoot, "hard-link.txt"); + expect(lstatSync(copiedPath).isSymbolicLink()).toBe(false); + expect(readFileSync(copiedPath, "utf8")).toBe("v1\n"); + }); + + it("skips a source link retargeted outside the checkout after planning", async () => { + const insidePath = join(sourceRoot, "inside.txt"); + const linkedPath = join(sourceRoot, "linked.txt"); + const outsidePath = join(tempDir, "outside.txt"); + writeFileSync(insidePath, "inside\n"); + writeFileSync(outsidePath, "outside\n"); + symlinkSync(insidePath, linkedPath); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "symlink linked.txt\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + rmSync(linkedPath); + symlinkSync(outsidePath, linkedPath); + + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + expect(existsSync(join(worktreeRoot, "linked.txt"))).toBe(false); + expect(result.skipped).toEqual([ + expect.objectContaining({ raw: "symlink linked.txt", reason: "unsafe" }), + ]); + }); + + it("skips a linked directory that exposes an external nested link", async () => { + const sharedPath = join(sourceRoot, "shared"); + const outsidePath = join(tempDir, "outside.txt"); + mkdirSync(sharedPath); + writeFileSync(outsidePath, "outside\n"); + symlinkSync(outsidePath, join(sharedPath, "outside.txt")); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "symlink shared\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([]); + expect(plan.skipped).toEqual([ + expect.objectContaining({ raw: "symlink shared", reason: "unsafe" }), + ]); + }); + + it("allows a linked directory with internal nested links", async () => { + const sharedPath = join(sourceRoot, "shared"); + const targetPath = join(sourceRoot, "shared-target"); + mkdirSync(sharedPath); + mkdirSync(targetPath); + writeFileSync(join(targetPath, "state.txt"), "source\n"); + symlinkSync(targetPath, join(sharedPath, "target"), "dir"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "symlink shared\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + expect(result).toMatchObject({ materialized: 1, skipped: [] }); + expect(readFileSync(join(worktreeRoot, "shared", "target", "state.txt"), "utf8")).toBe( + "source\n", + ); + }); + + it("skips a source removed after planning", async () => { + writeFileSync(join(sourceRoot, ".worktreeinclude"), "runtime.env\n"); + const sourcePath = join(sourceRoot, "runtime.env"); + writeFileSync(sourcePath, "source\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + rmSync(sourcePath); + + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + expect(existsSync(join(worktreeRoot, "runtime.env"))).toBe(false); + expect(result.skipped).toEqual([ + expect.objectContaining({ raw: "runtime.env", reason: "missing" }), + ]); + }); +}); + +describe.skipIf(!isPlatform("win32"))("worktree include Windows directory links", () => { + let tempDir: string; + let sourceRoot: string; + let worktreeRoot: string; + + beforeEach(() => { + tempDir = mkdtempSync(join(tmpdir(), "worktree-include-windows-test-")); + sourceRoot = join(tempDir, "source"); + worktreeRoot = join(tempDir, "worktree"); + mkdirSync(sourceRoot); + mkdirSync(worktreeRoot); + }); + + afterEach(() => { + rmSync(tempDir, { recursive: true, force: true }); + }); + + it("uses a live directory link and removes it without touching the source", async () => { + mkdirSync(join(sourceRoot, "shared-state")); + writeFileSync(join(sourceRoot, "shared-state", "state.txt"), "source-v1\n"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "symlink shared-state\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + writeFileSync(join(sourceRoot, "shared-state", "state.txt"), "source-v2\n"); + expect(readFileSync(join(worktreeRoot, "shared-state", "state.txt"), "utf8")).toBe( + "source-v2\n", + ); + + rmSync(worktreeRoot, { recursive: true, force: true }); + expect(readFileSync(join(sourceRoot, "shared-state", "state.txt"), "utf8")).toBe("source-v2\n"); + }); +}); diff --git a/packages/server/src/utils/worktree-include.ts b/packages/server/src/utils/worktree-include.ts new file mode 100644 index 0000000000..c0631661ab --- /dev/null +++ b/packages/server/src/utils/worktree-include.ts @@ -0,0 +1,1316 @@ +import { type Dirent, type Stats } from "fs"; +import { + copyFile, + cp, + lstat, + mkdir, + mkdtemp, + readFile, + readdir, + realpath, + rename, + rm, + rmdir, + symlink, +} from "fs/promises"; +import { + basename as pathBasename, + dirname, + isAbsolute, + join, + relative, + resolve, + win32, +} from "path"; +import { areEquivalentPaths, isPathInsideRoot } from "./path.js"; + +const WORKTREE_INCLUDE_FILE_NAME = ".worktreeinclude"; + +export type WorktreeIncludeMode = "copy" | "symlink"; +type WorktreeIncludeSourceKind = "file" | "directory"; +type WorktreeIncludeErrorCode = + | "conflict" + | "invalid_entry" + | "missing_source" + | "source_changed" + | "unsupported_source" + | "windows_symlink_unavailable"; + +export type WorktreeIncludeSkipReason = + | "conflict" + | "invalid" + | "materialization" + | "missing" + | "source_changed" + | "unsafe"; + +interface WorktreeIncludeEntry { + lineNumber: number; + mode: WorktreeIncludeMode; + raw: string; + relativePath: string; +} + +export interface WorktreeIncludeMaterialization { + lineNumber: number; + mode: WorktreeIncludeMode; + raw: string; + relativePath: string; + sourceKind: WorktreeIncludeSourceKind; +} + +export interface WorktreeIncludeSkippedEntry { + lineNumber: number; + message: string; + raw: string; + reason: WorktreeIncludeSkipReason; +} + +export interface WorktreeIncludeSummary { + materialized: number; + skipped: WorktreeIncludeSkippedEntry[]; +} + +export interface WorktreeIncludePlan { + excludedSourceRoots: string[]; + materializations: WorktreeIncludeMaterialization[]; + skipped: WorktreeIncludeSkippedEntry[]; + sourceRoot: string; +} + +export interface ReadWorktreeIncludePlanOptions { + excludedSourceRoots?: string[]; + sourceRoot: string; +} + +export interface MaterializeWorktreeIncludePlanOptions { + plan: WorktreeIncludePlan; + worktreeRoot: string; +} + +interface ResolvedWorktreeIncludeMaterialization { + materialization: WorktreeIncludeMaterialization; + sourcePath: string; +} + +interface StagedWorktreeIncludeMaterialization { + directoryPath: string; + entryPath: string; +} + +interface ParsedWorktreeIncludeEntries { + entries: WorktreeIncludeEntry[]; + skipped: WorktreeIncludeSkippedEntry[]; +} + +interface NormalizedWorktreeIncludeMaterializations { + materializations: WorktreeIncludeMaterialization[]; + skipped: WorktreeIncludeSkippedEntry[]; +} + +interface WorktreeIncludeCandidateCollection { + candidates: string[]; + errorsByPattern: Map; +} + +export class WorktreeIncludeError extends Error { + constructor( + public readonly code: WorktreeIncludeErrorCode, + message: string, + ) { + super(message); + this.name = "WorktreeIncludeError"; + } +} + +class WorktreeIncludeCleanupError extends Error { + constructor( + message: string, + public readonly cleanupError: unknown, + ) { + super(message); + this.name = "WorktreeIncludeCleanupError"; + } +} + +export async function readWorktreeIncludePlan( + options: ReadWorktreeIncludePlanOptions, +): Promise { + const sourceRoot = await realpath(options.sourceRoot); + const { entries, skipped } = await readWorktreeIncludeEntries(sourceRoot); + if (entries.length === 0) { + return { + sourceRoot, + excludedSourceRoots: [], + materializations: [], + skipped, + }; + } + + const excludedSourceRoots = ( + await Promise.all((options.excludedSourceRoots ?? []).map(canonicalizeExistingPathPrefix)) + ).filter((candidate) => !isPathInsideRoot(candidate, sourceRoot)); + const candidatePatterns = entries + .filter( + (entry) => entry.relativePath.includes("*") && getRecursiveDirectoryPath(entry) === null, + ) + .map((entry) => entry.relativePath); + const candidateCollection = + candidatePatterns.length > 0 + ? await collectWorktreeIncludeCandidates({ + sourceRoot, + excludedSourceRoots, + patterns: candidatePatterns, + }) + : { candidates: [], errorsByPattern: new Map() }; + const materializations: WorktreeIncludeMaterialization[] = []; + + for (const entry of entries) { + if (candidateCollection.errorsByPattern.has(entry.relativePath)) { + skipped.push( + toSkippedEntry(entry, candidateCollection.errorsByPattern.get(entry.relativePath)), + ); + continue; + } + + const matchedPaths = resolveEntryMatches({ entry, candidates: candidateCollection.candidates }); + if (matchedPaths.length === 0) { + skipped.push(toSkippedEntry(entry, noMatchError(entry))); + continue; + } + + for (const relativePath of matchedPaths) { + try { + const resolved = await resolveSourceMaterialization({ + entry, + excludedSourceRoots, + relativePath, + sourceRoot, + }); + materializations.push(resolved.materialization); + } catch (error) { + if (!isWorktreeIncludeMaterializationError(error)) { + throw error; + } + skipped.push(toSkippedEntry(entry, error)); + } + } + } + + const normalized = normalizeMaterializations(materializations); + + return { + excludedSourceRoots, + materializations: normalized.materializations, + skipped: [...skipped, ...normalized.skipped], + sourceRoot, + }; +} + +async function canonicalizeExistingPathPrefix(path: string): Promise { + const absolutePath = resolve(path); + const missingSegments: string[] = []; + let existingPath = absolutePath; + + while (true) { + try { + return join(await realpath(existingPath), ...missingSegments); + } catch (error) { + if (getErrorCode(error) !== "ENOENT" && getErrorCode(error) !== "ENOTDIR") { + throw error; + } + + const parentPath = dirname(existingPath); + if (parentPath === existingPath) { + return absolutePath; + } + missingSegments.unshift(pathBasename(existingPath)); + existingPath = parentPath; + } + } +} + +export async function materializeWorktreeIncludePlan( + options: MaterializeWorktreeIncludePlanOptions, +): Promise { + const skipped: WorktreeIncludeSkippedEntry[] = []; + let materialized = 0; + if (options.plan.materializations.length === 0) { + return { materialized, skipped }; + } + + const worktreeRoot = await realpath(options.worktreeRoot); + for (const materialization of options.plan.materializations) { + const entry = { + lineNumber: materialization.lineNumber, + mode: materialization.mode, + raw: materialization.raw, + relativePath: materialization.relativePath, + }; + let resolved: ResolvedWorktreeIncludeMaterialization; + try { + resolved = await resolveSourceMaterialization({ + entry, + excludedSourceRoots: options.plan.excludedSourceRoots, + relativePath: materialization.relativePath, + sourceRoot: options.plan.sourceRoot, + }); + } catch (error) { + if (isSkippableWorktreeIncludeError(error)) { + skipped.push(toSkippedEntry(entry, error)); + continue; + } + throw error; + } + if (resolved.materialization.sourceKind !== materialization.sourceKind) { + skipped.push( + toSkippedEntry( + entry, + new WorktreeIncludeError( + "source_changed", + `Source for .worktreeinclude entry '${materialization.raw}' changed type before it could be materialized`, + ), + ), + ); + continue; + } + + let staged: StagedWorktreeIncludeMaterialization | null = null; + let createdDestinationParents: string[] = []; + try { + const destinationPath = getDestinationPath({ + worktreeRoot, + relativePath: resolved.materialization.relativePath, + }); + if ( + !(await preflightDestination({ + worktreeRoot, + resolved, + })) + ) { + materialized++; + continue; + } + + staged = await stageMaterialization({ + destinationPath, + resolved, + worktreeRoot, + }); + createdDestinationParents = await ensureDestinationParent({ + worktreeRoot, + relativePath: resolved.materialization.relativePath, + }); + if ( + !(await preflightDestination({ + worktreeRoot, + resolved, + })) + ) { + await cleanupStagingDirectory(staged.directoryPath); + staged = null; + await cleanupCreatedDestinationParents(createdDestinationParents); + materialized++; + continue; + } + + const destinationStats = await lstatIfExists(destinationPath); + if (resolved.materialization.mode === "copy" && destinationStats !== null) { + await copyStagedMaterializationToExistingDestination({ + destinationPath, + resolved, + staged, + }); + } else { + await rename(staged.entryPath, destinationPath); + } + + await cleanupStagingDirectory(staged.directoryPath); + staged = null; + materialized++; + } catch (error) { + if (staged !== null) { + await cleanupStagingDirectory(staged.directoryPath); + } + await cleanupCreatedDestinationParents(createdDestinationParents); + if (isWorktreeIncludeMaterializationError(error)) { + skipped.push(toSkippedEntry(entry, error)); + continue; + } + throw error; + } + } + + return { materialized, skipped }; +} + +async function stageMaterialization(options: { + destinationPath: string; + resolved: ResolvedWorktreeIncludeMaterialization; + worktreeRoot: string; +}): Promise { + const directoryPath = await mkdtemp(join(options.worktreeRoot, ".paseo-worktreeinclude-")); + const entryPath = join(directoryPath, "entry"); + try { + if (options.resolved.materialization.mode === "copy") { + await copyMaterializationPath({ + destinationPath: entryPath, + sourceKind: options.resolved.materialization.sourceKind, + sourcePath: options.resolved.sourcePath, + }); + } else { + await createMaterializationSymlink({ + destinationPath: entryPath, + linkDestinationPath: options.destinationPath, + resolved: options.resolved, + }); + } + return { directoryPath, entryPath }; + } catch (error) { + await cleanupStagingDirectory(directoryPath); + throw error; + } +} + +async function copyStagedMaterializationToExistingDestination(options: { + destinationPath: string; + resolved: ResolvedWorktreeIncludeMaterialization; + staged: StagedWorktreeIncludeMaterialization; +}): Promise { + const backupPath = join(options.staged.directoryPath, "backup"); + const sourceKind = options.resolved.materialization.sourceKind; + await copyMaterializationPath({ + destinationPath: backupPath, + sourceKind, + sourcePath: options.destinationPath, + }); + try { + await copyMaterializationPath({ + destinationPath: options.destinationPath, + sourceKind, + sourcePath: options.staged.entryPath, + }); + } catch (error) { + await restoreCopiedDestination({ + backupPath, + destinationPath: options.destinationPath, + sourceKind, + }); + throw error; + } +} + +async function copyMaterializationPath(options: { + destinationPath: string; + sourceKind: WorktreeIncludeSourceKind; + sourcePath: string; +}): Promise { + if (options.sourceKind === "file") { + await copyFile(options.sourcePath, options.destinationPath); + return; + } + await cp(options.sourcePath, options.destinationPath, { + recursive: true, + force: true, + dereference: false, + }); +} + +async function restoreCopiedDestination(options: { + backupPath: string; + destinationPath: string; + sourceKind: WorktreeIncludeSourceKind; +}): Promise { + try { + const destinationStats = await lstatIfExists(options.destinationPath); + if (destinationStats !== null) { + if ( + destinationStats.isSymbolicLink() || + (options.sourceKind === "file" && !destinationStats.isFile()) || + (options.sourceKind === "directory" && !destinationStats.isDirectory()) + ) { + throw new Error("destination changed while restoring a failed materialization"); + } + await rm(options.destinationPath, { + recursive: destinationStats.isDirectory(), + force: true, + }); + } + await rename(options.backupPath, options.destinationPath); + } catch (error) { + throw new WorktreeIncludeCleanupError( + `Unable to restore .worktreeinclude destination '${options.destinationPath}' after a failed materialization`, + error, + ); + } +} + +async function cleanupStagingDirectory(directoryPath: string): Promise { + try { + await rm(directoryPath, { recursive: true, force: true }); + } catch (error) { + throw new WorktreeIncludeCleanupError( + `Unable to clean up .worktreeinclude staging directory '${directoryPath}'`, + error, + ); + } +} + +async function cleanupCreatedDestinationParents(createdPaths: string[]): Promise { + for (const path of createdPaths.toReversed()) { + try { + await rmdir(path); + } catch (error) { + const code = getErrorCode(error); + if (code === "ENOENT" || code === "ENOTEMPTY" || code === "EEXIST") { + continue; + } + throw new WorktreeIncludeCleanupError( + `Unable to clean up .worktreeinclude destination directory '${path}'`, + error, + ); + } + } +} + +async function readWorktreeIncludeEntries( + sourceRoot: string, +): Promise { + let contents: string; + try { + contents = await readFile(join(sourceRoot, WORKTREE_INCLUDE_FILE_NAME), "utf8"); + } catch (error) { + if (getErrorCode(error) === "ENOENT") { + return { entries: [], skipped: [] }; + } + throw error; + } + + const entries: WorktreeIncludeEntry[] = []; + const skipped: WorktreeIncludeSkippedEntry[] = []; + + for (const [index, sourceLine] of contents.split(/\r?\n/).entries()) { + const lineNumber = index + 1; + const line = sourceLine.trim(); + if (line.length === 0) { + continue; + } + + if (line.startsWith("#")) { + continue; + } + + try { + entries.push(parseWorktreeIncludeEntry({ line, lineNumber })); + } catch (error) { + if (!isSkippableWorktreeIncludeError(error)) { + throw error; + } + skipped.push(toSkippedEntry({ lineNumber, raw: line }, error)); + } + } + + return { entries, skipped }; +} + +function parseWorktreeIncludeEntry(options: { + line: string; + lineNumber: number; +}): WorktreeIncludeEntry { + let mode: WorktreeIncludeMode = "copy"; + let path = options.line; + const separatorIndex = options.line.search(/\s/); + + if (separatorIndex === -1) { + if (options.line === "copy" || options.line === "symlink") { + throw new WorktreeIncludeError( + "invalid_entry", + `.worktreeinclude ${options.line} entry on line ${options.lineNumber} requires a path`, + ); + } + } else { + const verb = options.line.slice(0, separatorIndex); + if (verb === "copy" || verb === "symlink") { + mode = verb; + path = options.line.slice(separatorIndex).trim(); + } + } + + return { + lineNumber: options.lineNumber, + mode, + raw: options.line, + relativePath: normalizeRelativePath({ entry: path, lineNumber: options.lineNumber }), + }; +} + +function normalizeRelativePath(options: { entry: string; lineNumber: number }): string { + const fail = (reason: string): never => { + throw new WorktreeIncludeError( + "invalid_entry", + `Invalid .worktreeinclude entry '${options.entry}' on line ${options.lineNumber}: ${reason}`, + ); + }; + + if ( + options.entry.includes("\0") || + options.entry.startsWith("/") || + options.entry.startsWith("\\") || + isAbsolute(options.entry) || + win32.isAbsolute(options.entry) || + /^[A-Za-z]:/.test(options.entry) + ) { + fail("absolute paths are not allowed"); + } + + const segments = options.entry + .split(/[\\/]+/) + .filter((segment) => segment.length > 0 && segment !== "."); + if (segments.length === 0) { + fail("path must not be empty"); + } + + for (const segment of segments) { + if (segment === "..") { + fail("parent-directory segments are not allowed"); + } + if (segment.toLowerCase() === ".git") { + fail("git metadata cannot be materialized"); + } + if (segment.includes(":") || /[. ]$/.test(segment) || isWindowsReservedSegment(segment)) { + fail("path is not portable to Windows"); + } + } + + return segments.join("/"); +} + +function isWindowsReservedSegment(segment: string): boolean { + const basename = segment.split(".", 1)[0]?.toLowerCase() ?? ""; + return /^(con|prn|aux|nul|com[1-9]|lpt[1-9])$/.test(basename); +} + +function getRecursiveDirectoryPath(entry: WorktreeIncludeEntry): string | null { + const segments = entry.relativePath.split("/"); + if ( + segments.length < 2 || + segments.at(-1) !== "**" || + segments.slice(0, -1).some((segment) => segment.includes("*")) + ) { + return null; + } + return segments.slice(0, -1).join("/"); +} + +async function collectWorktreeIncludeCandidates(options: { + excludedSourceRoots: string[]; + patterns: string[]; + sourceRoot: string; +}): Promise { + const candidates: string[] = []; + const errorsByPattern = new Map(); + + async function visit( + directoryPath: string, + directorySegments: string[], + patterns: string[], + ): Promise { + let entries: Dirent[]; + try { + entries = await readdir(directoryPath, { withFileTypes: true }); + } catch (error) { + if (getErrorCode(error) === null) { + throw error; + } + for (const pattern of patterns) { + errorsByPattern.set(pattern, error); + } + return; + } + for (const entry of entries) { + if (entry.name.toLowerCase() === ".git") { + continue; + } + + const pathSegments = [...directorySegments, entry.name]; + const sourcePath = join(options.sourceRoot, ...pathSegments); + if ( + options.excludedSourceRoots.some((excludedRoot) => + isPathInsideRoot(excludedRoot, sourcePath), + ) + ) { + continue; + } + + const relativePath = pathSegments.join("/"); + if (patterns.some((pattern) => worktreeIncludeGlobMatches(pattern, relativePath))) { + candidates.push(relativePath); + } + const descendantPatterns = patterns.filter((pattern) => + canGlobMatchDescendant(pattern, pathSegments), + ); + if (entry.isDirectory() && descendantPatterns.length > 0) { + await visit(sourcePath, pathSegments, descendantPatterns); + } + } + } + + await visit(options.sourceRoot, [], options.patterns); + return { candidates: candidates.sort(), errorsByPattern }; +} + +function canGlobMatchDescendant(pattern: string, directorySegments: string[]): boolean { + const patternSegments = pattern.split("/"); + const cache = new Map(); + + function match(patternIndex: number, directoryIndex: number): boolean { + const cacheKey = `${patternIndex}:${directoryIndex}`; + const cached = cache.get(cacheKey); + if (cached !== undefined) { + return cached; + } + + const patternSegment = patternSegments[patternIndex]; + let result: boolean; + if (directoryIndex === directorySegments.length) { + result = patternSegment !== undefined; + } else if (patternSegment === "**") { + result = match(patternIndex + 1, directoryIndex) || match(patternIndex, directoryIndex + 1); + } else { + const directorySegment = directorySegments[directoryIndex]; + result = + patternSegment !== undefined && + segmentGlobMatches(patternSegment, directorySegment) && + match(patternIndex + 1, directoryIndex + 1); + } + + cache.set(cacheKey, result); + return result; + } + + return match(0, 0); +} + +function resolveEntryMatches(options: { + candidates: string[]; + entry: WorktreeIncludeEntry; +}): string[] { + if (!options.entry.relativePath.includes("*")) { + return [options.entry.relativePath]; + } + + const recursiveDirectoryPath = getRecursiveDirectoryPath(options.entry); + if (recursiveDirectoryPath !== null) { + return [recursiveDirectoryPath]; + } + + return options.candidates.filter((candidate) => + worktreeIncludeGlobMatches(options.entry.relativePath, candidate), + ); +} + +function worktreeIncludeGlobMatches(pattern: string, candidate: string): boolean { + const patternSegments = pattern.split("/"); + const candidateSegments = candidate.split("/"); + const cache = new Map(); + + function match(patternIndex: number, candidateIndex: number): boolean { + const cacheKey = `${patternIndex}:${candidateIndex}`; + const cached = cache.get(cacheKey); + if (cached !== undefined) { + return cached; + } + + const patternSegment = patternSegments[patternIndex]; + let result: boolean; + if (patternSegment === undefined) { + result = candidateIndex === candidateSegments.length; + } else if (patternSegment === "**") { + result = + match(patternIndex + 1, candidateIndex) || + (candidateIndex < candidateSegments.length && match(patternIndex, candidateIndex + 1)); + } else { + const candidateSegment = candidateSegments[candidateIndex]; + result = + candidateSegment !== undefined && + segmentGlobMatches(patternSegment, candidateSegment) && + match(patternIndex + 1, candidateIndex + 1); + } + + cache.set(cacheKey, result); + return result; + } + + return match(0, 0); +} + +function segmentGlobMatches(pattern: string, value: string): boolean { + const escaped = pattern.replace(/[|\\{}()[\]^$+?.]/g, "\\$&").replace(/\*/g, "[^/]*"); + return new RegExp(`^${escaped}$`).test(value); +} + +async function resolveSourceMaterialization(options: { + entry: WorktreeIncludeEntry; + excludedSourceRoots: string[]; + relativePath: string; + sourceRoot: string; +}): Promise { + const requestedSourcePath = join(options.sourceRoot, ...options.relativePath.split("/")); + const sourcePath = await realpathSourcePath(requestedSourcePath, options.entry); + assertSourcePathIsSafe({ + entry: options.entry, + excludedSourceRoots: options.excludedSourceRoots, + sourcePath, + sourceRoot: options.sourceRoot, + }); + + const sourceKind = await getCanonicalSourceKind({ + entry: options.entry, + sourcePath, + }); + if (sourceKind === "directory") { + if (options.entry.mode === "copy") { + await assertCopyDirectorySafe({ + entry: options.entry, + sourcePath, + }); + } else { + await assertSymlinkDirectorySafe({ + entry: options.entry, + excludedSourceRoots: options.excludedSourceRoots, + sourcePath, + sourceRoot: options.sourceRoot, + }); + } + } + + return { + materialization: { + lineNumber: options.entry.lineNumber, + mode: options.entry.mode, + raw: options.entry.raw, + relativePath: options.relativePath, + sourceKind, + }, + sourcePath, + }; +} + +function assertSourcePathIsSafe(options: { + entry: WorktreeIncludeEntry; + excludedSourceRoots: string[]; + sourcePath: string; + sourceRoot: string; +}): void { + if (!isPathInsideRoot(options.sourceRoot, options.sourcePath)) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} resolves outside the source checkout`, + ); + } + + const excludedRoot = options.excludedSourceRoots.find( + (candidate) => + isPathInsideRoot(options.sourcePath, candidate) || + isPathInsideRoot(candidate, options.sourcePath), + ); + if (excludedRoot === undefined) { + return; + } + + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} overlaps with a protected worktree path`, + ); +} + +async function getCanonicalSourceKind(options: { + entry: WorktreeIncludeEntry; + sourcePath: string; +}): Promise { + const stats = await lstatSourcePath(options.sourcePath, options.entry); + if (stats.isSymbolicLink()) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} changed while its source path was resolved`, + ); + } + if (!stats.isFile() && !stats.isDirectory()) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} must match a regular file or directory`, + ); + } + + return stats.isDirectory() ? "directory" : "file"; +} + +async function realpathSourcePath( + sourcePath: string, + entry: WorktreeIncludeEntry, +): Promise { + try { + return await realpath(sourcePath); + } catch (error) { + const code = getErrorCode(error); + if (code === "ENOENT" || code === "ENOTDIR") { + throw noMatchError(entry); + } + if (code === "ELOOP") { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${entry.raw}' on line ${entry.lineNumber} contains a symbolic-link loop`, + ); + } + throw error; + } +} + +async function lstatSourcePath(sourcePath: string, entry: WorktreeIncludeEntry): Promise { + try { + return await lstat(sourcePath); + } catch (error) { + if (getErrorCode(error) === "ENOENT" || getErrorCode(error) === "ENOTDIR") { + throw noMatchError(entry); + } + throw error; + } +} + +async function assertCopyDirectorySafe(options: { + entry: WorktreeIncludeEntry; + sourcePath: string; +}): Promise { + for (const name of await readdir(options.sourcePath)) { + if (name.toLowerCase() === ".git") { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} contains git metadata`, + ); + } + + const sourcePath = join(options.sourcePath, name); + const stats = await lstatSourcePath(sourcePath, options.entry); + if (stats.isSymbolicLink() || (!stats.isFile() && !stats.isDirectory())) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} contains an unsupported file type or symbolic link`, + ); + } + if (stats.isDirectory()) { + await assertCopyDirectorySafe({ sourcePath, entry: options.entry }); + } + } +} + +async function assertSymlinkDirectorySafe(options: { + entry: WorktreeIncludeEntry; + excludedSourceRoots: string[]; + sourcePath: string; + sourceRoot: string; +}): Promise { + const visitedDirectories = new Set(); + + async function visit(directoryPath: string): Promise { + if (visitedDirectories.has(directoryPath)) { + return; + } + visitedDirectories.add(directoryPath); + + for (const name of await readdir(directoryPath)) { + if (name.toLowerCase() === ".git") { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} contains git metadata`, + ); + } + + const childPath = join(directoryPath, name); + const childStats = await lstatSourcePath(childPath, options.entry); + if (childStats.isSymbolicLink()) { + const linkedPath = await realpathSourcePath(childPath, options.entry); + assertSourcePathIsSafe({ + entry: options.entry, + excludedSourceRoots: options.excludedSourceRoots, + sourcePath: linkedPath, + sourceRoot: options.sourceRoot, + }); + const linkedKind = await getCanonicalSourceKind({ + entry: options.entry, + sourcePath: linkedPath, + }); + if (linkedKind === "directory") { + await visit(linkedPath); + } + continue; + } + + if (childStats.isDirectory()) { + await visit(childPath); + continue; + } + if (!childStats.isFile()) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} contains an unsupported file type`, + ); + } + } + } + + await visit(options.sourcePath); +} + +function normalizeMaterializations( + materializations: WorktreeIncludeMaterialization[], +): NormalizedWorktreeIncludeMaterializations { + const byPath = new Map(); + for (const materialization of materializations) { + const matches = byPath.get(materialization.relativePath) ?? []; + matches.push(materialization); + byPath.set(materialization.relativePath, matches); + } + + const skipped: WorktreeIncludeSkippedEntry[] = []; + const candidates: WorktreeIncludeMaterialization[] = []; + for (const [relativePath, matches] of byPath) { + if (new Set(matches.map((materialization) => materialization.mode)).size === 1) { + candidates.push(matches[0]!); + continue; + } + + const message = `.worktreeinclude entries for '${relativePath}' use both copy and symlink modes`; + skipped.push( + ...matches.map((materialization) => + toSkippedEntry(materialization, new WorktreeIncludeError("conflict", message)), + ), + ); + } + + const conflicted = new Set(); + const skippedConflictEntries = new Set(); + const skipConflict = (materialization: WorktreeIncludeMaterialization, message: string): void => { + conflicted.add(materialization); + if (skippedConflictEntries.has(materialization)) { + return; + } + skippedConflictEntries.add(materialization); + skipped.push(toSkippedEntry(materialization, new WorktreeIncludeError("conflict", message))); + }; + + for (let leftIndex = 0; leftIndex < candidates.length; leftIndex++) { + const left = candidates[leftIndex]!; + for (let rightIndex = leftIndex + 1; rightIndex < candidates.length; rightIndex++) { + const right = candidates[rightIndex]!; + let ancestor: WorktreeIncludeMaterialization | null = null; + if (isRelativePathAncestor(left.relativePath, right.relativePath)) { + ancestor = left; + } else if (isRelativePathAncestor(right.relativePath, left.relativePath)) { + ancestor = right; + } + if (ancestor === null || (left.mode === "copy" && right.mode === "copy")) { + continue; + } + + const descendant = ancestor === left ? right : left; + const message = `.worktreeinclude entries for '${ancestor.relativePath}' and '${descendant.relativePath}' overlap with a symlink`; + skipConflict(left, message); + skipConflict(right, message); + } + } + + const sorted = candidates + .filter((materialization) => !conflicted.has(materialization)) + .sort((left, right) => { + const depthDifference = + left.relativePath.split("/").length - right.relativePath.split("/").length; + return depthDifference === 0 + ? left.relativePath.localeCompare(right.relativePath) + : depthDifference; + }); + const normalized: WorktreeIncludeMaterialization[] = []; + + for (const materialization of sorted) { + const ancestor = normalized.find((candidate) => + isRelativePathAncestor(candidate.relativePath, materialization.relativePath), + ); + if (ancestor === undefined) { + normalized.push(materialization); + continue; + } + if (ancestor.mode === "copy" && materialization.mode === "copy") { + continue; + } + } + + return { materializations: normalized, skipped }; +} + +function isRelativePathAncestor(ancestor: string, candidate: string): boolean { + return ancestor !== candidate && candidate.startsWith(`${ancestor}/`); +} + +async function preflightDestination(options: { + resolved: ResolvedWorktreeIncludeMaterialization; + worktreeRoot: string; +}): Promise { + const { materialization } = options.resolved; + const destinationPath = getDestinationPath({ + worktreeRoot: options.worktreeRoot, + relativePath: materialization.relativePath, + }); + const destinationStats = await lstatIfExists(destinationPath); + if (destinationStats === null) { + return true; + } + + if (destinationStats.isSymbolicLink()) { + if ( + materialization.mode === "symlink" && + (await isExpectedSymlink({ + destinationPath, + sourcePath: options.resolved.sourcePath, + })) + ) { + return false; + } + throw destinationConflict(materialization); + } + + if (materialization.mode === "symlink") { + throw destinationConflict(materialization); + } + + if ( + (materialization.sourceKind === "file" && !destinationStats.isFile()) || + (materialization.sourceKind === "directory" && !destinationStats.isDirectory()) + ) { + throw destinationConflict(materialization); + } + + if (materialization.sourceKind === "directory") { + await assertCopyDestinationTreeSafe({ + sourcePath: options.resolved.sourcePath, + destinationPath, + materialization, + }); + } + return true; +} + +function getDestinationPath(options: { relativePath: string; worktreeRoot: string }): string { + const destinationPath = join(options.worktreeRoot, ...options.relativePath.split("/")); + if (!isPathInsideRoot(options.worktreeRoot, destinationPath)) { + throw new WorktreeIncludeError( + "invalid_entry", + `.worktreeinclude entry '${options.relativePath}' resolves outside the worktree`, + ); + } + return destinationPath; +} + +async function ensureDestinationParent(options: { + relativePath: string; + worktreeRoot: string; +}): Promise { + const parentSegments = options.relativePath.split("/").slice(0, -1); + let currentPath = options.worktreeRoot; + const createdPaths: string[] = []; + try { + for (const segment of parentSegments) { + currentPath = join(currentPath, segment); + const stats = await lstatIfExists(currentPath); + if (stats === null) { + await mkdir(currentPath); + createdPaths.push(currentPath); + continue; + } + if (stats.isSymbolicLink() || !stats.isDirectory()) { + throw new WorktreeIncludeError( + "conflict", + `Refusing to materialize .worktreeinclude entry '${options.relativePath}' through '${currentPath}'`, + ); + } + } + } catch (error) { + await cleanupCreatedDestinationParents(createdPaths); + throw error; + } + return createdPaths; +} + +async function assertCopyDestinationTreeSafe(options: { + destinationPath: string; + materialization: WorktreeIncludeMaterialization; + sourcePath: string; +}): Promise { + for (const name of await readdir(options.sourcePath)) { + const sourceChildPath = join(options.sourcePath, name); + const sourceStats = await lstat(sourceChildPath); + if (sourceStats.isSymbolicLink()) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.materialization.relativePath}' contains a symbolic link`, + ); + } + + const destinationChildPath = join(options.destinationPath, name); + const destinationStats = await lstatIfExists(destinationChildPath); + if (destinationStats === null) { + continue; + } + if (destinationStats.isSymbolicLink()) { + throw destinationConflict(options.materialization); + } + if ( + (sourceStats.isFile() && !destinationStats.isFile()) || + (sourceStats.isDirectory() && !destinationStats.isDirectory()) + ) { + throw destinationConflict(options.materialization); + } + if (sourceStats.isDirectory()) { + await assertCopyDestinationTreeSafe({ + sourcePath: sourceChildPath, + destinationPath: destinationChildPath, + materialization: options.materialization, + }); + } + } +} + +async function createMaterializationSymlink(options: { + destinationPath: string; + linkDestinationPath?: string; + resolved: ResolvedWorktreeIncludeMaterialization; +}): Promise { + if (process.platform !== "win32") { + const target = relative( + dirname(options.linkDestinationPath ?? options.destinationPath), + options.resolved.sourcePath, + ); + await symlink(target, options.destinationPath); + return; + } + + let type: "dir" | "file" | "junction" = "file"; + if (options.resolved.materialization.sourceKind === "directory") { + type = isWindowsNetworkPath(options.resolved.sourcePath) ? "dir" : "junction"; + } + try { + await symlink(options.resolved.sourcePath, options.destinationPath, type); + } catch (error) { + throw toWindowsSymlinkError({ error, entry: options.resolved.materialization }); + } +} + +function isWindowsSymlinkPrivilegeError(error: unknown): boolean { + const code = getErrorCode(error); + return code === "EACCES" || code === "EPERM" || code === "ENOTSUP"; +} + +function isWindowsNetworkPath(path: string): boolean { + return path.startsWith("\\\\"); +} + +function toWindowsSymlinkError(options: { + entry: WorktreeIncludeMaterialization; + error: unknown; +}): Error { + if (!isWindowsSymlinkPrivilegeError(options.error)) { + return options.error instanceof Error ? options.error : new Error(String(options.error)); + } + return new WorktreeIncludeError( + "windows_symlink_unavailable", + `Unable to create a Windows symlink for .worktreeinclude entry '${options.entry.relativePath}'. Enable Developer Mode or use copy ${options.entry.relativePath}.`, + ); +} + +async function isExpectedSymlink(options: { + destinationPath: string; + sourcePath: string; +}): Promise { + try { + const [destinationTarget, sourceTarget] = await Promise.all([ + realpath(options.destinationPath), + realpath(options.sourcePath), + ]); + return areEquivalentPaths(destinationTarget, sourceTarget); + } catch { + return false; + } +} + +async function lstatIfExists(path: string): Promise { + try { + return await lstat(path); + } catch (error) { + if (getErrorCode(error) === "ENOENT") { + return null; + } + throw error; + } +} + +function destinationConflict( + materialization: WorktreeIncludeMaterialization, +): WorktreeIncludeError { + return new WorktreeIncludeError( + "conflict", + `.worktreeinclude entry '${materialization.relativePath}' on line ${materialization.lineNumber} conflicts with the new worktree`, + ); +} + +function noMatchError(entry: WorktreeIncludeEntry): WorktreeIncludeError { + return new WorktreeIncludeError( + "missing_source", + `No paths matched .worktreeinclude entry '${entry.raw}' on line ${entry.lineNumber}`, + ); +} + +function toSkippedEntry( + entry: Pick, + error: unknown, +): WorktreeIncludeSkippedEntry { + return { + lineNumber: entry.lineNumber, + message: error instanceof Error ? error.message : String(error), + raw: entry.raw, + reason: getSkipReason(error), + }; +} + +function getSkipReason(error: unknown): WorktreeIncludeSkipReason { + if (!(error instanceof WorktreeIncludeError)) { + return "materialization"; + } + switch (error.code) { + case "conflict": + return "conflict"; + case "invalid_entry": + return "invalid"; + case "windows_symlink_unavailable": + return "materialization"; + case "missing_source": + return "missing"; + case "source_changed": + return "source_changed"; + case "unsupported_source": + return "unsafe"; + } +} + +function isSkippableWorktreeIncludeError(error: unknown): error is WorktreeIncludeError { + return error instanceof WorktreeIncludeError && error.code !== "windows_symlink_unavailable"; +} + +function isWorktreeIncludeMaterializationError(error: unknown): boolean { + return error instanceof WorktreeIncludeError || getErrorCode(error) !== null; +} + +function getErrorCode(error: unknown): string | null { + if (typeof error !== "object" || error === null || !("code" in error)) { + return null; + } + const { code } = error; + return typeof code === "string" ? code : null; +} diff --git a/packages/server/src/utils/worktree.posix.test.ts b/packages/server/src/utils/worktree.posix.test.ts index d895df5395..53a555cf4d 100644 --- a/packages/server/src/utils/worktree.posix.test.ts +++ b/packages/server/src/utils/worktree.posix.test.ts @@ -34,6 +34,7 @@ import { writeFileSync, readFileSync, chmodSync, + lstatSync, } from "fs"; import { delimiter, dirname, join } from "path"; import { tmpdir } from "os"; @@ -362,6 +363,68 @@ describe.skipIf(isPlatform("win32"))("worktree POSIX-only", () => { expect(metadata).toMatchObject({ baseRefName: "main" }); }); + it("removes fetched branches when include planning fails", async () => { + const remoteDir = join(tempDir, "remote.git"); + const remoteCloneDir = join(tempDir, "remote-clone"); + execFileSync("git", ["clone", "--bare", repoDir, remoteDir]); + execFileSync("git", ["remote", "add", "origin", remoteDir], { cwd: repoDir }); + + execFileSync("git", ["clone", remoteDir, remoteCloneDir]); + execFileSync("git", ["config", "user.email", "test@test.com"], { cwd: remoteCloneDir }); + execFileSync("git", ["config", "user.name", "Test"], { cwd: remoteCloneDir }); + execFileSync("git", ["checkout", "-b", "contributor/cleanup"], { cwd: remoteCloneDir }); + writeFileSync(join(remoteCloneDir, "file.txt"), "from-pr\n"); + execFileSync("git", ["add", "file.txt"], { cwd: remoteCloneDir }); + execFileSync("git", ["-c", "commit.gpgsign=false", "commit", "-m", "pr branch"], { + cwd: remoteCloneDir, + }); + const prHead = execFileSync("git", ["rev-parse", "HEAD"], { cwd: remoteCloneDir }) + .toString() + .trim(); + execFileSync("git", ["push", "origin", "contributor/cleanup"], { cwd: remoteCloneDir }); + execFileSync("git", [`--git-dir=${remoteDir}`, "update-ref", "refs/pull/44/head", prHead]); + mkdirSync(join(repoDir, ".worktreeinclude")); + + await expect( + createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "pr-44-cleanup", + source: { + kind: "checkout-github-pr", + githubPrNumber: 44, + headRef: "contributor/cleanup", + baseRefName: "main", + }, + runSetup: false, + paseoHome, + }), + ).rejects.toMatchObject({ code: "EISDIR" }); + + expect(() => + execFileSync("git", ["show-ref", "--verify", "--quiet", "refs/heads/contributor/cleanup"], { + cwd: repoDir, + stdio: "pipe", + }), + ).toThrow(); + + await expect( + createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "branch-cleanup", + source: { kind: "checkout-branch", branchName: "contributor/cleanup" }, + runSetup: false, + paseoHome, + }), + ).rejects.toMatchObject({ code: "EISDIR" }); + + expect(() => + execFileSync("git", ["show-ref", "--verify", "--quiet", "refs/heads/contributor/cleanup"], { + cwd: repoDir, + stdio: "pipe", + }), + ).toThrow(); + }); + it("fetches a GitHub PR branch when the head ref contains uppercase letters and dots", async () => { const remoteDir = join(tempDir, "remote.git"); const remoteCloneDir = join(tempDir, "remote-clone"); @@ -1062,6 +1125,241 @@ describe.skipIf(isPlatform("win32"))("worktree POSIX-only", () => { }); }); + it("materializes copies and symlinks before setup, then removes only the new worktree links", async () => { + writeFileSync( + join(repoDir, ".gitignore"), + [".copy.env", "copy-cache/", "linked-file.txt", "linked-state", "setup.log", ""].join("\n"), + ); + writeFileSync( + join(repoDir, "paseo.json"), + JSON.stringify({ + worktree: { + setup: [ + "test -f .copy.env", + "test -L linked-file.txt", + "test -L linked-state", + "cat linked-state/state.txt > setup.log", + ], + }, + }), + ); + execFileSync("git", ["add", ".gitignore", "paseo.json"], { cwd: repoDir }); + execFileSync("git", ["-c", "commit.gpgsign=false", "commit", "-m", "add include fixture"], { + cwd: repoDir, + }); + + writeFileSync( + join(repoDir, ".worktreeinclude"), + [".copy.env", "copy-cache/**", "symlink linked-file.txt", "symlink linked-state", ""].join( + "\n", + ), + ); + writeFileSync(join(repoDir, ".copy.env"), "copy-v1\n"); + mkdirSync(join(repoDir, "copy-cache"), { recursive: true }); + writeFileSync(join(repoDir, "copy-cache", "state.txt"), "copy-cache-v1\n"); + writeFileSync(join(repoDir, "linked-file.txt"), "linked-file-v1\n"); + mkdirSync(join(repoDir, "linked-state"), { recursive: true }); + writeFileSync(join(repoDir, "linked-state", "state.txt"), "linked-state-v1\n"); + + const result = await createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "include-links", + source: { kind: "branch-off", baseBranch: "main", branchName: "feature/include-links" }, + runSetup: true, + paseoHome, + }); + + expect(readFileSync(join(result.worktreePath, "setup.log"), "utf8")).toBe( + "linked-state-v1\n", + ); + expect(lstatSync(join(result.worktreePath, ".copy.env")).isSymbolicLink()).toBe(false); + expect(lstatSync(join(result.worktreePath, "copy-cache")).isSymbolicLink()).toBe(false); + expect(lstatSync(join(result.worktreePath, "linked-file.txt")).isSymbolicLink()).toBe(true); + expect(lstatSync(join(result.worktreePath, "linked-state")).isSymbolicLink()).toBe(true); + expect( + execFileSync("git", ["status", "--porcelain"], { + cwd: result.worktreePath, + encoding: "utf8", + }), + ).toBe(""); + + writeFileSync(join(repoDir, ".copy.env"), "copy-v2\n"); + writeFileSync(join(repoDir, "copy-cache", "state.txt"), "copy-cache-v2\n"); + writeFileSync(join(repoDir, "linked-file.txt"), "linked-file-v2\n"); + writeFileSync(join(repoDir, "linked-state", "state.txt"), "linked-state-v2\n"); + + expect(readFileSync(join(result.worktreePath, ".copy.env"), "utf8")).toBe("copy-v1\n"); + expect(readFileSync(join(result.worktreePath, "copy-cache", "state.txt"), "utf8")).toBe( + "copy-cache-v1\n", + ); + expect(readFileSync(join(result.worktreePath, "linked-file.txt"), "utf8")).toBe( + "linked-file-v2\n", + ); + expect(readFileSync(join(result.worktreePath, "linked-state", "state.txt"), "utf8")).toBe( + "linked-state-v2\n", + ); + + await deletePaseoWorktree({ + cwd: repoDir, + worktreePath: result.worktreePath, + paseoHome, + }); + + expect(existsSync(result.worktreePath)).toBe(false); + expect(readFileSync(join(repoDir, "linked-file.txt"), "utf8")).toBe("linked-file-v2\n"); + expect(readFileSync(join(repoDir, "linked-state", "state.txt"), "utf8")).toBe( + "linked-state-v2\n", + ); + }); + + it("skips missing includes and materializes paths that exist", async () => { + const projectHash = await deriveWorktreeProjectHash(repoDir); + const expectedWorktreePath = join(paseoHome, "worktrees", projectHash, "missing-include"); + writeFileSync( + join(repoDir, ".worktreeinclude"), + [".env", ".env.local", ".sops.yaml", ""].join("\n"), + ); + writeFileSync(join(repoDir, ".env"), "present\n"); + + const result = await createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "missing-include", + source: { kind: "branch-off", baseBranch: "main", branchName: "feature/missing-include" }, + runSetup: false, + paseoHome, + }); + + expect(result.worktreePath).toBe(expectedWorktreePath); + expect(readFileSync(join(result.worktreePath, ".env"), "utf8")).toBe("present\n"); + expect(existsSync(join(result.worktreePath, ".env.local"))).toBe(false); + expect(existsSync(join(result.worktreePath, ".sops.yaml"))).toBe(false); + expect( + execFileSync("git", ["worktree", "list", "--porcelain"], { + cwd: repoDir, + encoding: "utf8", + }), + ).toContain(expectedWorktreePath); + }); + + it("skips checkout-local worktree storage and creates the remaining includes", async () => { + const checkoutLocalPaseoHome = join(repoDir, ".dev", "paseo-home"); + const projectHash = await deriveWorktreeProjectHash(repoDir); + const expectedWorktreePath = join( + checkoutLocalPaseoHome, + "worktrees", + projectHash, + "protected-include", + ); + mkdirSync(join(checkoutLocalPaseoHome, "worktrees", projectHash), { recursive: true }); + writeFileSync(join(repoDir, ".worktreeinclude"), [".dev/**", ".env", ""].join("\n")); + writeFileSync(join(repoDir, ".env"), "present\n"); + + const result = await createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "protected-include", + source: { + kind: "branch-off", + baseBranch: "main", + branchName: "feature/protected-include", + }, + runSetup: false, + paseoHome: checkoutLocalPaseoHome, + }); + + expect(result.worktreePath).toBe(expectedWorktreePath); + expect(readFileSync(join(result.worktreePath, ".env"), "utf8")).toBe("present\n"); + expect(result.worktreeIncludeSummary?.skipped).toEqual([ + expect.objectContaining({ raw: ".dev/**", reason: "unsafe" }), + ]); + expect( + execFileSync("git", ["worktree", "list", "--porcelain"], { + cwd: repoDir, + encoding: "utf8", + }), + ).toContain(expectedWorktreePath); + expect( + execFileSync("git", ["branch", "--list", "feature/protected-include"], { + cwd: repoDir, + encoding: "utf8", + }).trim(), + ).toContain("feature/protected-include"); + }); + + it("keeps includes when branching from a Paseo-managed worktree", async () => { + writeFileSync(join(repoDir, ".gitignore"), ".env\n"); + writeFileSync(join(repoDir, ".worktreeinclude"), ".env\n"); + execFileSync("git", ["add", ".gitignore", ".worktreeinclude"], { cwd: repoDir }); + execFileSync("git", ["-c", "commit.gpgsign=false", "commit", "-m", "add include config"], { + cwd: repoDir, + }); + writeFileSync(join(repoDir, ".env"), "source\n"); + + const sourceWorktree = await createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "include-source", + source: { + kind: "branch-off", + baseBranch: "main", + branchName: "feature/include-source", + }, + runSetup: false, + paseoHome, + }); + const nestedWorktree = await createLegacyWorktreeForTest({ + cwd: sourceWorktree.worktreePath, + worktreeSlug: "include-nested", + source: { + kind: "branch-off", + baseBranch: "feature/include-source", + branchName: "feature/include-nested", + }, + runSetup: false, + paseoHome, + }); + + expect(readFileSync(join(sourceWorktree.worktreePath, ".env"), "utf8")).toBe("source\n"); + expect(readFileSync(join(nestedWorktree.worktreePath, ".env"), "utf8")).toBe("source\n"); + expect(nestedWorktree.worktreeIncludeSummary?.skipped).toEqual([]); + }); + + it("skips a symlink include conflict and keeps the new worktree", async () => { + const projectHash = await deriveWorktreeProjectHash(repoDir); + const expectedWorktreePath = join(paseoHome, "worktrees", projectHash, "include-conflict"); + writeFileSync(join(repoDir, "paseo.json"), JSON.stringify({ scripts: {} })); + writeFileSync(join(repoDir, ".worktreeinclude"), "symlink paseo.json\n"); + + const result = await createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "include-conflict", + source: { + kind: "branch-off", + baseBranch: "main", + branchName: "feature/include-conflict", + }, + runSetup: false, + paseoHome, + }); + + expect(result.worktreePath).toBe(expectedWorktreePath); + expect(existsSync(expectedWorktreePath)).toBe(true); + expect(lstatSync(join(expectedWorktreePath, "paseo.json")).isSymbolicLink()).toBe(false); + expect(result.worktreeIncludeSummary?.skipped).toEqual([ + expect.objectContaining({ raw: "symlink paseo.json", reason: "conflict" }), + ]); + expect( + execFileSync("git", ["worktree", "list", "--porcelain"], { + cwd: repoDir, + encoding: "utf8", + }), + ).toContain(expectedWorktreePath); + expect( + execFileSync("git", ["branch", "--list", "feature/include-conflict"], { + cwd: repoDir, + encoding: "utf8", + }).trim(), + ).toContain("feature/include-conflict"); + }); + it("creates a worktree without error when no paseo.json exists in the main repo", async () => { const result = await createLegacyWorktreeForTest({ cwd: repoDir, diff --git a/packages/server/src/utils/worktree.ts b/packages/server/src/utils/worktree.ts index 4ab641806b..4592e28bf8 100644 --- a/packages/server/src/utils/worktree.ts +++ b/packages/server/src/utils/worktree.ts @@ -36,6 +36,11 @@ import { createExternalProcessEnv } from "../server/paseo-env.js"; import { parseGitRevParsePath, resolveGitRevParsePath } from "./git-rev-parse-path.js"; import { validateBranchSlug } from "@getpaseo/protocol/branch-slug"; import { expandTilde, getRealpathAwareRelativePath, isPathInsideRoot } from "./path.js"; +import { + materializeWorktreeIncludePlan, + readWorktreeIncludePlan, + type WorktreeIncludeSummary, +} from "./worktree-include.js"; export { slugify, validateBranchSlug } from "@getpaseo/protocol/branch-slug"; @@ -46,6 +51,7 @@ const READ_ONLY_GIT_ENV = { export interface WorktreeConfig { branchName: string; + worktreeIncludeSummary?: WorktreeIncludeSummary; worktreePath: string; } @@ -1161,13 +1167,57 @@ export async function deletePaseoWorktree({ } } +export interface RollbackCreatedPaseoWorktreeOptions extends DeletePaseoWorktreeOptions { + createdBranchName?: string; +} + +async function removeCreatedWorktreeBranch(options: { + createdBranchName?: string; + cwd?: string | null; +}): Promise { + if (!options.createdBranchName || !options.cwd) { + return; + } + if (!(await localBranchExists(options.cwd, options.createdBranchName))) { + return; + } + await runGitCommand(["branch", "--delete", "--force", options.createdBranchName], { + cwd: options.cwd, + }); +} + +async function rollbackCreatedWorktreeBranch( + options: { + createdBranchName?: string; + cwd: string; + }, + cause: unknown, +): Promise { + let cleanupError: unknown; + try { + await removeCreatedWorktreeBranch(options); + } catch (error) { + cleanupError = error; + } + if (cleanupError) { + const failure = new Error( + `${cause instanceof Error ? cause.message : "Worktree workflow failed"}; rollback also failed: ${cleanupError instanceof Error ? cleanupError.message : String(cleanupError)}`, + { cause }, + ); + Object.assign(failure, { cleanupError }); + throw failure; + } + throw cause; +} + export async function rollbackCreatedPaseoWorktree( - options: DeletePaseoWorktreeOptions, + options: RollbackCreatedPaseoWorktreeOptions, cause: unknown, ): Promise { let cleanupError: unknown; try { await deletePaseoWorktree(options); + await removeCreatedWorktreeBranch(options); } catch (error) { cleanupError = error; } @@ -1235,48 +1285,92 @@ export const createWorktree = async ({ worktreesRoot, }: CreateWorktreeOptions): Promise => { const sourcePlan = await resolveWorktreeSourcePlan({ cwd, source, desiredSlug: worktreeSlug }); - let worktreePath = join(await getPaseoWorktreesRoot(cwd, paseoHome, worktreesRoot), worktreeSlug); - mkdirSync(dirname(worktreePath), { recursive: true }); + const { worktreeIncludePlan, worktreePath } = await (async () => { + try { + const paseoWorktreesRoot = await getPaseoWorktreesRoot(cwd, paseoHome, worktreesRoot); + const includePlan = await readWorktreeIncludePlan({ + sourceRoot: cwd, + excludedSourceRoots: [paseoWorktreesRoot], + }); + const requestedWorktreePath = join(paseoWorktreesRoot, worktreeSlug); + mkdirSync(dirname(requestedWorktreePath), { recursive: true }); + + // Also handle worktree path collision + let finalWorktreePath = requestedWorktreePath; + let pathSuffix = 1; + while (existsSync(finalWorktreePath)) { + finalWorktreePath = `${requestedWorktreePath}-${pathSuffix}`; + pathSuffix++; + } - // Also handle worktree path collision - let finalWorktreePath = worktreePath; - let pathSuffix = 1; - while (existsSync(finalWorktreePath)) { - finalWorktreePath = `${worktreePath}-${pathSuffix}`; - pathSuffix++; - } + // Primitive owner for `git worktree add`; callers route through createWorktreeCore. + await runGitCommand(["worktree", "add", finalWorktreePath, ...sourcePlan.addArguments], { + cwd, + timeout: 120_000, + }); - // Primitive owner for `git worktree add`; callers route through createWorktreeCore. - await runGitCommand(["worktree", "add", finalWorktreePath, ...sourcePlan.addArguments], { - cwd, - timeout: 120_000, - }); - worktreePath = normalizePathForOwnership(finalWorktreePath); + return { + worktreeIncludePlan: includePlan, + worktreePath: normalizePathForOwnership(finalWorktreePath), + }; + } catch (error) { + return rollbackCreatedWorktreeBranch( + { cwd, createdBranchName: sourcePlan.createdBranchName }, + error, + ); + } + })(); - if (sourcePlan.pushRemote) { - await configureWorktreePushRemote({ - cwd, - branchName: sourcePlan.branchName, - remote: sourcePlan.pushRemote, + let worktreeIncludeSummary: WorktreeIncludeSummary = { + materialized: 0, + skipped: [...worktreeIncludePlan.skipped], + }; + try { + if (sourcePlan.pushRemote) { + await configureWorktreePushRemote({ + cwd, + branchName: sourcePlan.branchName, + remote: sourcePlan.pushRemote, + }); + } + if (sourcePlan.trackingRemote) { + await configureWorktreeTrackingRemote({ + cwd, + branchName: sourcePlan.branchName, + remote: sourcePlan.trackingRemote, + }); + } + + writePaseoWorktreeMetadata(worktreePath, { + baseRefName: sourcePlan.metadataBaseRefName, + ...(sourcePlan.changeRequestLookupTarget + ? { changeRequestLookupTarget: sourcePlan.changeRequestLookupTarget } + : {}), }); - } - if (sourcePlan.trackingRemote) { - await configureWorktreeTrackingRemote({ - cwd, - branchName: sourcePlan.branchName, - remote: sourcePlan.trackingRemote, + + await seedPaseoConfigFile({ sourceCwd: cwd, targetCwd: worktreePath }); + const materialization = await materializeWorktreeIncludePlan({ + plan: worktreeIncludePlan, + worktreeRoot: worktreePath, }); + worktreeIncludeSummary = { + materialized: materialization.materialized, + skipped: [...worktreeIncludePlan.skipped, ...materialization.skipped], + }; + } catch (error) { + await rollbackCreatedPaseoWorktree( + { + cwd, + worktreePath, + teardownCwds: [], + paseoHome, + worktreesBaseRoot: worktreesRoot, + createdBranchName: sourcePlan.createdBranchName, + }, + error, + ); } - writePaseoWorktreeMetadata(worktreePath, { - baseRefName: sourcePlan.metadataBaseRefName, - ...(sourcePlan.changeRequestLookupTarget - ? { changeRequestLookupTarget: sourcePlan.changeRequestLookupTarget } - : {}), - }); - - await seedPaseoConfigFile({ sourceCwd: cwd, targetCwd: worktreePath }); - if (runSetup) { await runWorktreeSetupCommands({ worktreePath, @@ -1287,6 +1381,7 @@ export const createWorktree = async ({ return { branchName: sourcePlan.branchName, + worktreeIncludeSummary, worktreePath, }; }; @@ -1299,6 +1394,7 @@ interface ResolveWorktreeSourcePlanOptions { interface WorktreeSourcePlan { branchName: string; + createdBranchName?: string; metadataBaseRefName: string; changeRequestLookupTarget?: PaseoWorktreeChangeRequestLookupTarget; addArguments: string[]; @@ -1314,6 +1410,11 @@ interface WorktreeSourcePlan { }; } +type ChangeRequestWorktreeSource = Extract< + WorktreeSource, + { kind: "checkout-change-request" | "checkout-github-pr" } +>; + async function resolveWorktreeSourcePlan({ cwd, source, @@ -1332,93 +1433,127 @@ async function resolveWorktreeSourcePlan({ return { branchName: newBranchName, + createdBranchName: newBranchName, metadataBaseRefName: normalizedBaseBranch, addArguments: ["-b", newBranchName, "--no-track", base], }; } - case "checkout-branch": { - await validateExistingWorktreeBranchName(cwd, source.branchName); - if (!(await localBranchExists(cwd, source.branchName))) { - try { - await runGitCommand(["fetch", "origin", `${source.branchName}:${source.branchName}`], { - cwd, - timeout: 120_000, - }); - } catch { - throw new UnknownBranchError({ branchName: source.branchName, cwd }); - } - } - if (await isBranchCheckedOut(cwd, source.branchName)) { - throw new BranchAlreadyCheckedOutError(source.branchName); - } - - return { - branchName: source.branchName, - metadataBaseRefName: source.branchName, - addArguments: [source.branchName], - }; - } + case "checkout-branch": + return resolveCheckoutBranchWorktreeSourcePlan({ cwd, branchName: source.branchName }); case "checkout-change-request": - case "checkout-github-pr": { - const localBranchCandidate = source.localBranchName ?? source.headRef; - await validateExistingWorktreeBranchName(cwd, localBranchCandidate); - const localBranchName = await resolveUniqueLocalBranchName(cwd, localBranchCandidate); - const normalizedBaseRefName = normalizeRequiredBaseBranch(source.baseRefName); - const changeRequestNumber = - source.kind === "checkout-github-pr" ? source.githubPrNumber : source.changeRequestNumber; - await fetchWorktreeCheckoutRefs({ - cwd, - localBranchName, - checkoutRefs: source.checkoutRefs ?? [ - { remoteName: "origin", remoteRef: `refs/pull/${changeRequestNumber}/head` }, - ], + case "checkout-github-pr": + return resolveChangeRequestWorktreeSourcePlan({ cwd, source }); + } +} + +async function resolveCheckoutBranchWorktreeSourcePlan(options: { + branchName: string; + cwd: string; +}): Promise { + await validateExistingWorktreeBranchName(options.cwd, options.branchName); + const needsFetch = !(await localBranchExists(options.cwd, options.branchName)); + if (needsFetch) { + try { + await runGitCommand(["fetch", "origin", `${options.branchName}:${options.branchName}`], { + cwd: options.cwd, + timeout: 120_000, }); - const shouldTrackOriginHead = source.trackOriginHead === true; - const trackingRemote = shouldTrackOriginHead - ? await tryFetchWorktreeTrackingRemote({ - cwd, - remoteName: "origin", - headRef: source.headRef, - }) - : undefined; - const remotePlan: Pick = {}; - if (source.pushRemoteUrl) { - const remoteName = `paseo-pr-${changeRequestNumber}`; + } catch { + return rollbackCreatedWorktreeBranch( + { cwd: options.cwd, createdBranchName: options.branchName }, + new UnknownBranchError({ branchName: options.branchName, cwd: options.cwd }), + ); + } + } + + try { + if (await isBranchCheckedOut(options.cwd, options.branchName)) { + throw new BranchAlreadyCheckedOutError(options.branchName); + } + } catch (error) { + if (!needsFetch) { + throw error; + } + return rollbackCreatedWorktreeBranch( + { cwd: options.cwd, createdBranchName: options.branchName }, + error, + ); + } + + return { + branchName: options.branchName, + ...(needsFetch ? { createdBranchName: options.branchName } : {}), + metadataBaseRefName: options.branchName, + addArguments: [options.branchName], + }; +} + +async function resolveChangeRequestWorktreeSourcePlan(options: { + cwd: string; + source: ChangeRequestWorktreeSource; +}): Promise { + const { cwd, source } = options; + const localBranchCandidate = source.localBranchName ?? source.headRef; + await validateExistingWorktreeBranchName(cwd, localBranchCandidate); + const localBranchName = await resolveUniqueLocalBranchName(cwd, localBranchCandidate); + const normalizedBaseRefName = normalizeRequiredBaseBranch(source.baseRefName); + const changeRequestNumber = + source.kind === "checkout-github-pr" ? source.githubPrNumber : source.changeRequestNumber; + + try { + await fetchWorktreeCheckoutRefs({ + cwd, + localBranchName, + checkoutRefs: source.checkoutRefs ?? [ + { remoteName: "origin", remoteRef: `refs/pull/${changeRequestNumber}/head` }, + ], + }); + const shouldTrackOriginHead = source.trackOriginHead === true; + const trackingRemote = shouldTrackOriginHead + ? await tryFetchWorktreeTrackingRemote({ + cwd, + remoteName: "origin", + headRef: source.headRef, + }) + : undefined; + const remotePlan: Pick = {}; + if (source.pushRemoteUrl) { + const remoteName = `paseo-pr-${changeRequestNumber}`; + remotePlan.pushRemote = { + name: remoteName, + url: source.pushRemoteUrl, + headRef: source.headRef, + track: true, + }; + } else if (shouldTrackOriginHead && localBranchName !== source.headRef) { + const originUrl = await getWorktreeRemotePushUrl(cwd, "origin"); + if (originUrl) { remotePlan.pushRemote = { - name: remoteName, - url: source.pushRemoteUrl, + name: `paseo-pr-${changeRequestNumber}`, + url: originUrl, headRef: source.headRef, - track: true, + track: false, }; - } else if (shouldTrackOriginHead && localBranchName !== source.headRef) { - const originUrl = await getWorktreeRemotePushUrl(cwd, "origin"); - if (originUrl) { - remotePlan.pushRemote = { - name: `paseo-pr-${changeRequestNumber}`, - url: originUrl, - headRef: source.headRef, - track: false, - }; - } } - if (trackingRemote) { - remotePlan.trackingRemote = trackingRemote; - } - - return { - branchName: localBranchName, - metadataBaseRefName: normalizedBaseRefName, - changeRequestLookupTarget: { - headRef: source.headRef, - ...(source.headRepositoryOwner - ? { headRepositoryOwner: source.headRepositoryOwner } - : {}), - changeRequestNumber, - }, - addArguments: [localBranchName], - ...remotePlan, - }; } + if (trackingRemote) { + remotePlan.trackingRemote = trackingRemote; + } + + return { + branchName: localBranchName, + createdBranchName: localBranchName, + metadataBaseRefName: normalizedBaseRefName, + changeRequestLookupTarget: { + headRef: source.headRef, + ...(source.headRepositoryOwner ? { headRepositoryOwner: source.headRepositoryOwner } : {}), + changeRequestNumber, + }, + addArguments: [localBranchName], + ...remotePlan, + }; + } catch (error) { + return rollbackCreatedWorktreeBranch({ cwd, createdBranchName: localBranchName }, error); } } diff --git a/public-docs/worktrees.md b/public-docs/worktrees.md index bdc8a24f23..8f94e1626f 100644 --- a/public-docs/worktrees.md +++ b/public-docs/worktrees.md @@ -110,6 +110,44 @@ Both fields accept a multiline shell script or an array of commands; commands ru Commands run with the worktree as `cwd`. Use `$PASEO_SOURCE_CHECKOUT_PATH` to reach files in the original checkout (untracked config, local caches, etc). +## .worktreeinclude + +Use a root-level .worktreeinclude to materialize local source-checkout files before +worktree.setup runs. Each path is relative to the source checkout. + + # Copy is the default. + .env.local + .cache/** + + # Modes can also be explicit. + symlink node_modules + copy .tool-state/** + +Each line is `[copy|symlink] `; the mode is optional and defaults to `copy`. Blank lines +and whole-line comments are ignored. A single star matches within a path segment and a double +star matches recursively; a directory ending in /\*\* materializes that directory as one +recursive entry. Absolute paths, parent-directory paths, and .git paths are rejected. + +Copy entries are independent snapshots: a copied file is replaced on a later materialization, +and a copied directory merges into an existing directory. Symlink entries point directly at +the live source file or directory, so changes through either path affect the same data. Paseo +does not replace an existing file, directory, or different link with a symlink. + +Entries must resolve to regular files or directories. Copy entries reject source symbolic links +and nested symbolic links so Paseo never writes through an unexpected path. A symlinked +directory intentionally exposes its live source contents. + +Prefer paths ignored by the target branch. For a symlinked directory, use an ignore rule without +a trailing slash (for example, node_modules, not node_modules/), because Git treats the link +itself as a file. Unignored materialized paths appear in git status. + +On Windows, Paseo uses junctions for local directories. File links and network-directory links +require Windows symbolic-link support. It never silently copies an explicit `symlink ` +entry; enable Developer Mode or switch that entry to `copy ` if link creation fails. + +Archiving removes only the worktree's links, not their source targets. If the source path is +later moved or deleted, a symlink becomes broken; Paseo does not repair it automatically. + ## Scripts and services `scripts` are named commands you can run inside a worktree on demand. Mark one as a _service_ and Paseo supervises it as a long-running process, assigns it a port, and routes HTTP traffic to it through the daemon's reverse proxy.