From 4793d8b514e88e872b0daaf1b9b5604ea9572026 Mon Sep 17 00:00:00 2001 From: Mendy Landa Date: Tue, 6 Oct 2026 23:20:12 +0300 Subject: [PATCH 1/2] Add an opt-in setting that lets the API Worker apply its own schema Self-hosters and one-click installers such as Appflare deploy new versions without a checkout or a direct database URL, so they can't run `pnpm migrate` before each update. With MAIL_AUTO_MIGRATE_ENABLED=true, each isolate checks a schema marker in mail.schema_migrations once, before the first event that uses the database. If the marker is missing, it runs migrate() through Hyperdrive in one transaction under pg_advisory_xact_lock, without changing role. A failure rolls back and fails the event, and the next event retries. /healthz skips the check. migrate() records the marker, a hash of the migration list, so `pnpm migrate` and the Worker agree. The setting is off by default, so the hosted deployment keeps applying reviewed SQL by hand. --- README.md | 6 +- apps/email/README.md | 22 ++- apps/email/src/database.ts | 33 ++++ apps/email/src/worker.ts | 35 +++- apps/email/test/auto-migrate.test.ts | 172 ++++++++++++++++++++ apps/email/test/fixtures/postgres-worker.ts | 2 +- apps/email/test/native-migration.test.ts | 6 +- apps/email/test/postgres-runtime.test.ts | 60 ++++++- docs/planetscale.md | 5 + 9 files changed, 330 insertions(+), 11 deletions(-) create mode 100644 apps/email/test/auto-migrate.test.ts diff --git a/README.md b/README.md index a4be474..b7ea46b 100644 --- a/README.md +++ b/README.md @@ -118,7 +118,11 @@ lists the API Worker's settings. In the dashboard's file: 2. Create the database and connect Hyperdrive with the [database guide](docs/planetscale.md). Put a direct connection string in `apps/email/.env` as `DATABASE_URL` and run `pnpm migrate`. It applies the - full schema and is safe to rerun on upgrades. + full schema and is safe to rerun on upgrades. To skip this step on install + and on every upgrade, set `MAIL_AUTO_MIGRATE_ENABLED` to `true` in the API + Worker's `vars`, and the Worker applies the schema itself. It is off by + default, and the Hyperdrive user then needs the right to create the schema. + The [Worker guide](apps/email/README.md#set-up-cloudflare) has the details. 3. Pick a dashboard. The Cloudflare Worker dashboard supports public passwordless accounts; follow the [account guide](docs/accounts.md). The Node dashboard below uses one shared owner password and can see every inbox, diff --git a/apps/email/README.md b/apps/email/README.md index 866806b..8f484f2 100644 --- a/apps/email/README.md +++ b/apps/email/README.md @@ -61,15 +61,29 @@ flowchart LR Create a dedicated PlanetScale Postgres database, connect its primary to Hyperdrive with query caching disabled, and replace the `HYPERDRIVE` ID in `wrangler.jsonc`. Run migrations with a direct PostgreSQL `DATABASE_URL` - from `apps/email/.env`. The Worker uses the binding and needs no database URL secret. + from `apps/email/.env`, or let the Worker apply them as described below. + The Worker uses the binding and needs no database URL secret. 9. Deploy with `pnpm --filter @bezalel/email run deploy`. In Cloudflare Email Routing, send the domain's catch-all to this Worker. Remove any specific address rules that would override this route for agent addresses. Run `migrate` before deploying an upgrade too. Applied schema changes are recorded -in `mail.schema_migrations`. The search-index upgrade replaces the old generated -column and its index in one transaction, preserves messages, and can be rerun. -It takes a table lock while rebuilding the index; schedule it during a quiet period. +in `mail.schema_migrations`, with a marker for the current schema. The +search-index upgrade replaces the old generated column and its index in one +transaction, preserves messages, and can be rerun. It takes a table lock while +rebuilding the index; schedule it during a quiet period. + +To skip the manual step, set `MAIL_AUTO_MIGRATE_ENABLED` to `true` in `vars`. +It is off by default. Each Worker isolate then checks the marker once, before +the first request, email, queue batch, or cron run that uses the database. If +the marker is missing, the isolate runs the migrations through Hyperdrive in one +transaction under an advisory lock, so isolates that start together migrate +once. A failed migration rolls back and fails the event, and the next event +tries again. `/healthz` skips the check. An upgrade's migrations run on the +first event after the deploy. The Hyperdrive user needs the right to create the +`mail` schema and its objects, for example as the database owner. The Worker +never switches to the `postgres` role the way `pnpm migrate` does. The hosted +service leaves this off and applies reviewed SQL by hand. The Worker checks sending and receiving before creating an inbox. The default mode requires a catch-all targeting this Worker. For a subdomain sharing a zone diff --git a/apps/email/src/database.ts b/apps/email/src/database.ts index 4ce2909..61d1e7e 100644 --- a/apps/email/src/database.ts +++ b/apps/email/src/database.ts @@ -1,3 +1,4 @@ +import { createHash } from "node:crypto" import { Client } from "pg" import { migrations } from "./schema.js" @@ -38,6 +39,38 @@ export const postgresDatabase = (connectionString: string): Database => ({ }), }) +// Any edit to the migrations changes this marker, so a database that has it is up to date. +const schemaVersion = `schema-${createHash("sha256").update(JSON.stringify(migrations)).digest("hex").slice(0, 16)}` + export const migrate = async (db: DatabaseQueries): Promise => { for (const statement of migrations) await db.query(statement) + await db.query("insert into mail.schema_migrations(id) values ($1) on conflict do nothing", [schemaVersion]) +} + +const schemaRecorded = async (db: DatabaseQueries): Promise => { + const [row] = await db.query<{ recorded: boolean }>( + "select exists(select 1 from mail.schema_migrations where id = $1) as recorded", [schemaVersion]) + return row?.recorded === true +} + +// Brings the schema up to date through the Worker's own connection. A current database costs one +// query. Otherwise one transaction holds a fixed advisory lock, so isolates that start together +// migrate once, and a failure rolls back everything. It runs as the connecting user and never +// changes role, unlike `pnpm migrate`. +export const ensureSchema = async (db: Database): Promise<"current" | "migrated"> => { + try { + if (await schemaRecorded(db)) return "current" + } catch (error) { + // 42P01: a fresh database has no mail.schema_migrations table yet. + if ((error as { code?: unknown }).code !== "42P01") throw error + } + return db.transaction(async tx => { + await tx.query("select pg_advisory_xact_lock(hashtext('bezalel-email-schema'))") + // Another isolate may have finished while this one waited for the lock. + const [table] = await tx.query<{ present: boolean }>( + "select to_regclass('mail.schema_migrations') is not null as present") + if (table?.present && await schemaRecorded(tx)) return "current" + await migrate(tx) + return "migrated" + }) } diff --git a/apps/email/src/worker.ts b/apps/email/src/worker.ts index 66aa716..5574d7e 100644 --- a/apps/email/src/worker.ts +++ b/apps/email/src/worker.ts @@ -19,7 +19,7 @@ import { type MailConfig, type Operation } from "./contracts.js" -import { postgresDatabase } from "./database.js" +import { ensureSchema, postgresDatabase, type Database } from "./database.js" import { MailService } from "./mail-service.js" import { MailboxStore } from "./mailbox-store.js" import { CustomDomains, type GatewayConfig } from "./custom-domains.js" @@ -55,6 +55,7 @@ export interface Env { MAIL_DOMAIN_ENCRYPTION_KEY?: string MAIL_INBOUND_SCAN_ENABLED?: string MAIL_FEEDBACK_SIGNERS?: string + MAIL_AUTO_MIGRATE_ENABLED?: string AUTH_PUBLIC_URL?: string AUTH_SECRET?: string AUTH_PROXY_SECRET?: string @@ -75,7 +76,8 @@ const configuration = z.object({ PUBLIC_EMAIL_URL: z.url(), MAIL_EVENTS_URL: z.url().optional(), BEZALEL_EVENTS_URL: z.url().optional(), - WORKER_NAME: z.string().min(1) + WORKER_NAME: z.string().min(1), + MAIL_AUTO_MIGRATE_ENABLED: z.enum(["true", "false"]).optional() }) export const serviceFor = (env: Env): MailService => { @@ -151,6 +153,23 @@ export const serviceFor = (env: Env): MailService => { }) } +let schema: Promise | undefined + +// With MAIL_AUTO_MIGRATE_ENABLED=true, each isolate brings the schema up to date once, before the +// first event that uses the database. A failure clears the promise so the next event retries. +const schemaReady = (env: Env, db: Database, ctx?: ExecutionContext): Promise => { + if (env.MAIL_AUTO_MIGRATE_ENABLED !== "true") return Promise.resolve() + if (!schema) { + schema = ensureSchema(db).catch((error: unknown) => { + schema = undefined + throw error + }) + // Finish the migration even if the event that started it ends first. + ctx?.waitUntil(schema.catch(() => {})) + } + return schema +} + const json = (data: unknown, status = 200): Response => Response.json(data, { status, headers: { "cache-control": "no-store" } }) @@ -296,10 +315,15 @@ export async function consumeDeliveryBatch( } export default { - async fetch(request: Request, env: Env): Promise { + async fetch(request: Request, env: Env, ctx?: ExecutionContext): Promise { try { const service = serviceFor(env) const path = new URL(request.url).pathname + // Health checks stay off the database. + if (path !== "/healthz") { + try { await schemaReady(env, service.store.db, ctx) } + catch { return json({ error: { message: "Email operation failed", code: "internal_error", transient: true } }, 500) } + } if (path.startsWith("/api/auth/") || path.startsWith("/account-rpc/")) { const config = accountConfig(env) const accounts = postgresAccountAuth(config, env.HYPERDRIVE.connectionString, service) @@ -327,6 +351,7 @@ export default { ): Promise { const service = serviceFor(env) try { + await schemaReady(env, service.store.db, ctx) const raw = await readBytes(message.raw, 25 * 1024 * 1024) await service.acceptIncoming(message.to, raw, message.from) ctx.waitUntil(service.processIncoming().then(() => service.processTriage())) @@ -342,6 +367,9 @@ export default { }, async queue(batch: MessageBatch, env: Env, ctx: ExecutionContext): Promise { const service = serviceFor(env) + // Like a database error, a schema that is not ready retries the batch after a minute. + try { await schemaReady(env, service.store.db, ctx) } + catch { batch.retryAll({ delaySeconds: 60 }); return } await consumeDeliveryBatch(batch, service.store, service.config) ctx.waitUntil(service.flushEvents()) }, @@ -351,6 +379,7 @@ export default { ctx: ExecutionContext ): Promise { const service = serviceFor(env) + await schemaReady(env, service.store.db, ctx) ctx.waitUntil(service.processIncoming().then(() => service.flushEvents())) ctx.waitUntil(service.processTriage()) if (env.AUTH_PUBLIC_URL && env.AUTH_FROM) ctx.waitUntil(sendNotifications(service.store.db, service.transport, env.AUTH_FROM, env.AUTH_PUBLIC_URL)) diff --git a/apps/email/test/auto-migrate.test.ts b/apps/email/test/auto-migrate.test.ts new file mode 100644 index 0000000..51d13c2 --- /dev/null +++ b/apps/email/test/auto-migrate.test.ts @@ -0,0 +1,172 @@ +import { randomUUID } from "node:crypto" +import { afterEach, describe, expect, it, vi } from "vitest" +import { ensureSchema, migrate, type Database, type DatabaseQueries } from "../src/database.js" +import type { Env } from "../src/worker.js" +import { testDatabase } from "./database.js" +import { config } from "./support.js" + +// The Worker reaches each test database through its Hyperdrive connection string. +const hyperdrive = vi.hoisted(() => new Map()) +vi.mock("../src/database.js", async (original) => { + const actual = await original() + return { ...actual, postgresDatabase: (url: string) => hyperdrive.get(url) ?? actual.postgresDatabase(url) } +}) + +const closers: Array<() => Promise> = [] +afterEach(async () => { + for (const close of closers.splice(0)) await close() +}) + +// Records every statement and can fail the ones a test picks. +const database = async () => { + const { db, pg } = await testDatabase() + closers.push(() => pg.close()) + const statements: string[] = [] + const state = { transactions: 0, fail: undefined as ((text: string) => boolean) | undefined } + const queries = (target: DatabaseQueries): DatabaseQueries => ({ + async query(text: string, parameters?: unknown[]) { + statements.push(text) + if (state.fail?.(text)) throw new Error("database unavailable") + return target.query(text, parameters) + }, + }) + const tracked: Database = { + ...queries(db), + transaction: task => { + state.transactions++ + return db.transaction(tx => task(queries(tx))) + }, + } + const url = `postgres://hyperdrive.test/${randomUUID()}` + hyperdrive.set(url, tracked) + return { db: tracked, statements, state, url } +} + +// A fresh module graph stands in for a new isolate with nothing memoized. +const isolate = async () => { + vi.resetModules() + return (await import("../src/worker.js")).default +} + +const environment = (url: string, setting?: string): Env => ({ + HYPERDRIVE: { connectionString: url } as Hyperdrive, + MAIL_API_TOKEN: config.apiToken, MAIL_WEBHOOK_SECRET: config.webhookSecret, + CLOUDFLARE_API_TOKEN: "test-token", CLOUDFLARE_ACCOUNT_ID: "a".repeat(32), + EMAIL_DOMAINS: JSON.stringify(config.domains), DEFAULT_EMAIL_DOMAIN: config.defaultDomain, + PUBLIC_EMAIL_URL: config.publicUrl, WORKER_NAME: "auto-migrate-test", MAIL_OBJECTS: {} as R2Bucket, + MAIL_AUTO_MIGRATE_ENABLED: setting, +}) +const context = () => ({ waitUntil: vi.fn(), passThroughOnException: vi.fn() }) as unknown as ExecutionContext +const listInboxes = (authorized = true) => new Request("https://mail.example.com/rpc/listInboxes", { + method: "POST", headers: authorized ? { authorization: `Bearer ${config.apiToken}` } : {}, body: "{}", +}) +const schemaObjects = (db: DatabaseQueries) => db.query(` + select c.relname as name, c.relkind::text as kind from pg_class c + join pg_namespace n on n.oid = c.relnamespace where n.nspname = 'mail' + union all select p.proname, 'function' from pg_proc p + join pg_namespace n on n.oid = p.pronamespace where n.nspname = 'mail' + order by 1, 2`) +const markers = (db: DatabaseQueries) => db.query("select id from mail.schema_migrations order by id") +const markerLookup = "select exists(select 1 from mail.schema_migrations where id = $1) as recorded" + +describe("MAIL_AUTO_MIGRATE_ENABLED", () => { + it("leaves the database alone unless it is true", async () => { + for (const setting of [undefined, "false"]) { + const { db, statements, state, url } = await database() + const worker = await isolate() + expect((await worker.fetch(listInboxes(), environment(url, setting), context())).status).toBe(500) + expect(statements.some((text) => text.includes("schema_migrations"))).toBe(false) + expect(state.transactions).toBe(0) + expect(await db.query("select to_regclass('mail.schema_migrations') as marker")).toEqual([{ marker: null }]) + } + const { url } = await database() + const response = await (await isolate()).fetch(listInboxes(), environment(url, "yes"), context()) + expect(response.status).toBe(503) + expect(await response.json()).toMatchObject({ error: { code: "not_configured" } }) + }) + + it("applies the full schema to an empty database before it serves the first request", async () => { + const { db, statements, url } = await database() + const reference = await testDatabase() + closers.push(() => reference.pg.close()) + await migrate(reference.db) + const worker = await isolate() + const env = environment(url, "true") + expect((await worker.fetch(new Request("https://mail.example.com/healthz"), env, context())).status).toBe(200) + expect(statements).toEqual([]) + const ctx = context() + const response = await worker.fetch(listInboxes(), env, ctx) + expect(response.status).toBe(200) + expect(await response.json()).toMatchObject({ result: { inboxes: [] } }) + expect(ctx.waitUntil).toHaveBeenCalledTimes(1) + expect(await schemaObjects(db)).toEqual(await schemaObjects(reference.db)) + expect(await markers(db)).toEqual(await markers(reference.db)) + expect(statements.some((text) => /set\s+(local\s+)?role/i.test(text))).toBe(false) + }) + + it("costs one marker lookup per isolate once the schema is current", async () => { + const { db, statements, state, url } = await database() + await migrate(db) + statements.length = 0 + const env = environment(url, "true") + const first = await isolate() + expect((await first.fetch(listInboxes(false), env, context())).status).toBe(401) + expect((await first.fetch(listInboxes(false), env, context())).status).toBe(401) + const second = await isolate() + expect((await second.fetch(listInboxes(false), env, context())).status).toBe(401) + expect(statements).toEqual([markerLookup, markerLookup]) + expect(state.transactions).toBe(0) + expect(await ensureSchema(db)).toBe("current") + }) + + it("migrates an empty database once when isolates and requests start together", async () => { + const isolates = await database() + const results = await Promise.all(Array.from({ length: 4 }, () => ensureSchema(isolates.db))) + expect(results.sort()).toEqual(["current", "current", "current", "migrated"]) + expect(await markers(isolates.db)).toHaveLength(2) + + const { state, url } = await database() + const worker = await isolate() + const env = environment(url, "true") + const responses = await Promise.all(Array.from({ length: 6 }, () => worker.fetch(listInboxes(), env, context()))) + expect(responses.map((response) => response.status)).toEqual(Array(6).fill(200)) + expect(state.transactions).toBe(1) + }) + + it("fails every event while the schema cannot be checked and retries on the next one", async () => { + const { statements, state, url } = await database() + state.fail = () => true + const worker = await isolate() + const env = environment(url, "true") + const response = await worker.fetch(listInboxes(), env, context()) + expect(response.status).toBe(500) + expect(await response.json()).toEqual({ error: { + message: "Email operation failed", code: "internal_error", transient: true, + } }) + const setReject = vi.fn() + await expect(worker.email({ + from: "customer@example.net", to: "agent@example.com", raw: new Response("mail").body!, rawSize: 4, setReject, + } as unknown as ForwardableEmailMessage, env, context())).rejects.toThrow("Email could not be stored") + expect(setReject).not.toHaveBeenCalled() + const message = { body: {}, ack: vi.fn(), retry: vi.fn() } + const batch = { messages: [message], retryAll: vi.fn(), ackAll: vi.fn() } + await worker.queue(batch as unknown as MessageBatch, env, context()) + expect(batch.retryAll).toHaveBeenCalledWith({ delaySeconds: 60 }) + expect(message.ack).not.toHaveBeenCalled() + expect(message.retry).not.toHaveBeenCalled() + await expect(worker.scheduled({} as ScheduledController, env, context())).rejects.toThrow("database unavailable") + expect(statements).toEqual(Array(4).fill(markerLookup)) + state.fail = undefined + expect((await worker.fetch(listInboxes(), env, context())).status).toBe(200) + }) + + it("rolls back a migration that fails partway and finishes it on the next attempt", async () => { + const { db, state } = await database() + state.fail = (text) => text.includes("create table if not exists mail.sends") + await expect(ensureSchema(db)).rejects.toThrow("database unavailable") + expect(await db.query("select to_regclass('mail.inboxes') as inboxes")).toEqual([{ inboxes: null }]) + state.fail = undefined + expect(await ensureSchema(db)).toBe("migrated") + expect(await ensureSchema(db)).toBe("current") + }) +}) diff --git a/apps/email/test/fixtures/postgres-worker.ts b/apps/email/test/fixtures/postgres-worker.ts index ac16f5c..2383650 100644 --- a/apps/email/test/fixtures/postgres-worker.ts +++ b/apps/email/test/fixtures/postgres-worker.ts @@ -40,6 +40,6 @@ export default { await worker.scheduled({} as ScheduledController, env, ctx) return Response.json({ scheduled: true }) } - return worker.fetch(request, env) + return worker.fetch(request, env, ctx) } } satisfies ExportedHandler diff --git a/apps/email/test/native-migration.test.ts b/apps/email/test/native-migration.test.ts index 9d16f75..c5d836f 100644 --- a/apps/email/test/native-migration.test.ts +++ b/apps/email/test/native-migration.test.ts @@ -75,7 +75,11 @@ it("upgrades the deployed native schema without changing saved mail, credentials await migrate(f.db) await migrate(f.db) - expect(await snapshot(f.db, columns)).toEqual(before) + const after = await snapshot(f.db, columns) + // The only new row is the schema marker that migrate() records. + expect(after.schema_migrations).toEqual([...before.schema_migrations!, + { row: { id: expect.stringMatching(/^schema-[a-f0-9]{16}$/), applied_at: expect.any(String) } }]) + expect({ ...after, schema_migrations: before.schema_migrations }).toEqual(before) expect(await f.db.query("select * from mail.customer_inboxes")).toEqual([]) expect(await f.db.query("select * from mail.messages where triage is not null")).toEqual([]) expect((await f.service.execute("listInboxes", {}) as any).inboxes).toHaveLength(2) diff --git a/apps/email/test/postgres-runtime.test.ts b/apps/email/test/postgres-runtime.test.ts index 21b3147..947641f 100644 --- a/apps/email/test/postgres-runtime.test.ts +++ b/apps/email/test/postgres-runtime.test.ts @@ -6,7 +6,7 @@ import { execFile } from "node:child_process" import { promisify } from "node:util" import { afterAll, beforeAll, describe, expect, it } from "vitest" import { unstable_dev, type Unstable_DevWorker } from "wrangler" -import { migrate, postgresDatabase } from "../src/database.js" +import { ensureSchema, migrate, postgresDatabase } from "../src/database.js" import { postgresFixture } from "./database.js" import { config, rawMail } from "./support.js" @@ -34,6 +34,7 @@ it("the migration command preserves postgres ownership and application grants fo env: { ...process.env, DATABASE_URL: connection.href }, timeout: 20_000 }) expect(result.stdout).toContain("Email schema is ready") + expect(await ensureSchema(database.db)).toBe("current") const tables = await database.db.query<{ owner: string; readable: boolean; writable: boolean }>( `select pg_get_userbyid(c.relowner) as owner, has_table_privilege($1,c.oid,'SELECT') as readable, @@ -173,3 +174,60 @@ describe("Hyperdrive in the Workers runtime with PostgreSQL", () => { expect((await worker.fetch("http://localhost/__test/scheduled")).status).toBe(200) }) }) + +describe("automatic migrations through Hyperdrive in the Workers runtime", () => { + let database: Awaited> + let worker: Unstable_DevWorker + let directory: string + const owner = `owner_${randomUUID().replaceAll("-", "")}` + beforeAll(async () => { + database = await postgresFixture() + // The Hyperdrive user owns the database but cannot become postgres, as on Neon or Supabase. + const password = randomUUID() + const connection = new URL(database.connectionString) + await database.db.query(`create role ${owner} login password '${password}'`) + await database.db.query(`alter database ${connection.pathname.slice(1)} owner to ${owner}`) + connection.username = owner + connection.password = password + directory = await mkdtemp(join(tmpdir(), "bezalel-auto-migrate-")) + const configuration = join(directory, "wrangler.json") + await writeFile(configuration, JSON.stringify({ + name: "auto-migrate-runtime-test", + main: resolve("test/fixtures/postgres-worker.ts"), + compatibility_date: "2026-09-06", compatibility_flags: ["nodejs_compat"], + hyperdrive: [{ binding: "HYPERDRIVE", id: "0".repeat(32), localConnectionString: connection.href }], + r2_buckets: [{ binding: "MAIL_OBJECTS", bucket_name: "auto-migrate-runtime-test" }], + vars: { + MAIL_AUTO_MIGRATE_ENABLED: "true", + MAIL_API_TOKEN: config.apiToken, MAIL_WEBHOOK_SECRET: config.webhookSecret, + CLOUDFLARE_API_TOKEN: "test-token", CLOUDFLARE_ACCOUNT_ID: config.accountId, + EMAIL_DOMAINS: JSON.stringify(config.domains), DEFAULT_EMAIL_DOMAIN: config.defaultDomain, + PUBLIC_EMAIL_URL: config.publicUrl, WORKER_NAME: "auto-migrate-runtime-test", + } + }), { mode: 0o600 }) + worker = await unstable_dev("test/fixtures/postgres-worker.ts", { + config: configuration, local: true, ip: "127.0.0.1", port: 0, inspectorPort: 0, + logLevel: "error", experimental: { disableExperimentalWarning: true, disableDevRegistry: true }, + }) + }) + afterAll(async () => { + await worker?.stop() + await database?.pg.close() + await postgresDatabase(process.env.TEST_DATABASE_URL!).query(`drop role if exists ${owner}`) + if (directory) await rm(directory, { recursive: true, force: true }) + }) + it("migrates an empty database once while concurrent cold requests wait for it", async () => { + expect((await worker.fetch("http://localhost/healthz")).status).toBe(200) + expect(await database.db.query("select to_regclass('mail.schema_migrations') as marker")).toEqual([{ marker: null }]) + const responses = await Promise.all(Array.from({ length: 8 }, () => worker.fetch("http://localhost/rpc/listInboxes", { + method: "POST", headers: { authorization: `Bearer ${config.apiToken}` }, body: "{}" + }))) + for (const response of responses) expect(response.status, await response.clone().text()).toBe(200) + const markers = await database.db.query<{ id: string }>("select id from mail.schema_migrations order by id") + expect(markers.map((row) => row.id)).toEqual(["bounded-search-v1", expect.stringMatching(/^schema-[a-f0-9]{16}$/)]) + const owners = await database.db.query<{ owner: string }>(`select distinct pg_get_userbyid(c.relowner) as owner + from pg_class c join pg_namespace n on n.oid = c.relnamespace where n.nspname = 'mail'`) + expect(owners).toEqual([{ owner }]) + expect(await ensureSchema(database.db)).toBe("current") + }) +}) diff --git a/docs/planetscale.md b/docs/planetscale.md index 268a8d1..6b03e08 100644 --- a/docs/planetscale.md +++ b/docs/planetscale.md @@ -63,6 +63,11 @@ gateway, mail domains, R2 bucket, and delivery queues still need deployment setu `pnpm migrate`. Keep development and production credentials separate. For an existing deployment, migrate its data and verify a staging copy before changing the Worker's database connection; running this command only creates the schema. + To have the Worker migrate instead, set `MAIL_AUTO_MIGRATE_ENABLED` to `true`. + It is off by default, and this deployment leaves it off. The Worker then + migrates as the Hyperdrive user without `SET LOCAL ROLE`, so that user must be + able to create the `mail` schema and its objects, for example as the database + owner. The limited role in step 4 can't. 4. Create an application role for Hyperdrive with `USAGE` on the `mail` schema, read/write access to its tables, and `EXECUTE` on its functions. Apply grants for both existing and future objects created by the migration role. From 4795167ace5dd1af07bae2cf42d6fcf7e854eb22 Mon Sep 17 00:00:00 2001 From: Mendy Landa Date: Wed, 7 Oct 2026 01:07:02 +0300 Subject: [PATCH 2/2] Note that automatic upgrades need the Hyperdrive user to own the mail objects --- apps/email/README.md | 12 +++++++++--- docs/planetscale.md | 5 ++++- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/apps/email/README.md b/apps/email/README.md index 8f484f2..1691229 100644 --- a/apps/email/README.md +++ b/apps/email/README.md @@ -81,9 +81,15 @@ transaction under an advisory lock, so isolates that start together migrate once. A failed migration rolls back and fails the event, and the next event tries again. `/healthz` skips the check. An upgrade's migrations run on the first event after the deploy. The Hyperdrive user needs the right to create the -`mail` schema and its objects, for example as the database owner. The Worker -never switches to the `postgres` role the way `pnpm migrate` does. The hosted -service leaves this off and applies reviewed SQL by hand. +`mail` schema and its objects, for example as the database owner, and must own +any `mail` objects that already exist, directly or as a member of their owning +role. The Worker never switches to the `postgres` role the way `pnpm migrate` +does, so objects that `pnpm migrate` created belong to `postgres`, and an +upgrade's `alter table` fails on them. The event returns an error until the +ownership is fixed or someone runs `pnpm migrate`. For an existing database, +make the Hyperdrive user the owner of the `mail` schema and every object in it +first, or keep the setting off and run `pnpm migrate`. The hosted service leaves +this off and applies reviewed SQL by hand. The Worker checks sending and receiving before creating an inbox. The default mode requires a catch-all targeting this Worker. For a subdomain sharing a zone diff --git a/docs/planetscale.md b/docs/planetscale.md index 6b03e08..e97beba 100644 --- a/docs/planetscale.md +++ b/docs/planetscale.md @@ -67,7 +67,10 @@ gateway, mail domains, R2 bucket, and delivery queues still need deployment setu It is off by default, and this deployment leaves it off. The Worker then migrates as the Hyperdrive user without `SET LOCAL ROLE`, so that user must be able to create the `mail` schema and its objects, for example as the database - owner. The limited role in step 4 can't. + owner. The limited role in step 4 can't. On upgrades it also alters existing + tables, so it must own every `mail` object or belong to the role that does. + Objects that `pnpm migrate` already created belong to `postgres`; leave the + setting off for such a database unless you move their ownership first. 4. Create an application role for Hyperdrive with `USAGE` on the `mail` schema, read/write access to its tables, and `EXECUTE` on its functions. Apply grants for both existing and future objects created by the migration role.