feat(phase-2): data model and Firestore security rules foundation

.claude/plans/exxeed-task13-rtdb-rules-report.md

@@ -0,0 +1,56 @@
+# Exxeed Implementation Report — task13-rtdb-rules
+
+Date: 2026-04-17
+
+## What was built
+
+Replaced the placeholder `database.rules.json` (deny-all) with the §5.8 production rule set covering responder telemetry writes (capturedAt bounds, role/accountStatus guards, field validation), granular read access by role and responder_index cross-references, client-deny on `responder_index`, and shared_projection read/write controls. Created `rtdb.rules.test.ts` with 17 test cases exercising every positive and negative path.
+
+## Requirements coverage
+
+| ID  | Requirement                                         | Status | Notes                   |
+| --- | --------------------------------------------------- | ------ | ----------------------- |
+| R01 | Responder writes own location with valid capturedAt | ✅     | Test passes             |
+| R02 | capturedAt > now + 60000 fails                      | ✅     | +70 000 ms case         |
+| R03 | capturedAt < now - 600000 fails                     | ✅     | -700 000 ms case        |
+| R04 | Non-responder role write fails                      | ✅     | citizen role tested     |
+| R05 | Responder writes to another uid fails               | ✅     | Cross-uid guard         |
+| R06 | Suspended responder write fails                     | ✅     | accountStatus=suspended |
+| R07 | Responder reads own location                        | ✅     | Self-read               |
+| R08 | Superadmin reads any responder                      | ✅     | provincial_superadmin   |
+| R09 | Muni admin matching municipalityId reads            | ✅     | responder_index seeded  |
+| R10 | Muni admin mismatch fails                           | ✅     | san-vicente vs daet     |
+| R11 | Agency admin matching agencyId reads                | ✅     | pdrrmo match            |
+| R12 | Agency admin mismatch fails                         | ✅     | bfp vs pdrrmo           |
+| R13 | responder_index client read always fails            | ✅     | Even superadmin         |
+| R14 | responder_index client write always fails           | ✅     | Even superadmin         |
+| R15 | shared_projection matching muni admin reads         | ✅     | daet match              |
+| R16 | Mismatched muni admin fails on shared_projection    | ✅     | san-vicente vs daet     |
+| R17 | Any client write to shared_projection fails         | ✅     | superadmin denied       |
+
+## Files changed
+
+| File                                       | Change type | Reason                                 |
+| ------------------------------------------ | ----------- | -------------------------------------- |
+| infra/firebase/database.rules.json         | modified    | Replace placeholder with §5.8 rule set |
+| functions/src/**tests**/rtdb.rules.test.ts | created     | 17 RTDB rules test cases               |
+
+## Baseline vs final test state
+
+- Baseline: No rtdb.rules.test.ts → vitest exits with code 1 (no files found)
+- Final: 17/17 tests passing
+- Delta: +17 new passing tests, zero regressions
+
+## Open items
+
+- None
+
+## Divergences encountered
+
+- **Storage emulator not running (port 9199):** The shared `createTestEnv` helper requires all three emulators (firestore, database, storage). Storage returned ECONNREFUSED. Resolution: initialized test environment directly in the test file with only firestore + database emulators, which are all that RTDB rule testing requires.
+
+## Notes on test design
+
+- `validPayload(capturedAt)` helper ensures all 7 `.validate` fields are present in every write test, so the only variable is `capturedAt`.
+- `responder_index/$uid` is seeded via `withSecurityRulesDisabled` so the muni/agency admin read path tests exercise the actual database cross-reference in the rule.
+- `FIREBASE_DATABASE_EMULATOR_HOST=127.0.0.1:9000` must be set in the environment when running these tests.

docs/learnings.md

@@ -252,3 +252,35 @@ Vitest auto-discovers `vitest.config.ts` but NOT `vitest.workspace.ts`. Use `vit
 ### Terraform: `google_project_iam_member` vs `google_service_account_iam_member`
 
 When a service account needs to impersonate another SA, always use `google_service_account_iam_member` scoped to the _specific_ target SA — not `google_project_iam_member` at project level. Project-level `roles/iam.serviceAccountUser` grants impersonation of _every_ SA in the project, violating least privilege. The `google_service_account_iam_member` resource requires `service_account_id = google_service_account.target.name` (not email) and grants impersonation rights on that specific SA only.
+
+---
+
+## Phase 2: Data Model and Security Rules
+
+### Firestore rules: `resource.data.__reportId` does not exist
+
+Firestore rules do not expose a synthetic `__reportId` on `resource.data`. Cross-document sharing checks (e.g., `canReadReportDoc` helper used for report subcollections) must be implemented per-collection using the document ID from the path, not a hypothetical field on `resource.data`. The helper `canReadReportDoc(data)` was kept narrow; sharing logic lives at each collection's `match` block.
+
+### Rule coverage checker regex must match at path segment boundaries
+
+The regex `['"\`]<collection>[/'\`"]` must only match `match /<collection>/` at the start of a path segment. If the collection name appears as a substring inside another path (e.g., `hazard_zones_history` containing `hazard_zones`), the checker produces false negatives. Use `match\s+/` prefix in the regex.
+
+### Subagent commit reports are unreliable — always verify with git log
+
+Subagent implementers sometimes report commits that don't exist in the actual git history (due to hook failures, revert operations, or self-review issues). Always run `git log --oneline -3` to confirm the actual commit state before proceeding to review. The file on disk is the only source of truth.
+
+### Firebase RTDB `.validate` rules require all children at once
+
+A `.validate` rule like `newData.hasChildren([...])` only checks that those keys exist — not their types or values. It does not cascade to nested validation. For required field validation, the rule checks presence only; type validation must be done in Cloud Functions or security rules with explicit field-by-field checks.
+
+### RTDB and Storage emulators need explicit initialization in test harness
+
+The `initializeTestEnvironment` from `@firebase/rules-unit-testing` accepts an options object with `firestore`, `database`, and `storage` entries. When testing only one emulator (e.g., RTDB), you can omit storage — but if `createTestEnv` is called with all three emulators configured and only one is running, tests hang. Use `initializeTestEnvironment` directly with only the emulators you need for the test file.
+
+### Every `strict()` Zod object rejects unknown keys — critical for rule alignment
+
+Firestore `diff(resource.data).affectedKeys().hasOnly([...])` at the rule layer rejects any unknown key the same way a strict Zod schema does. If the Zod schema allows extra keys but the rules don't, production writes will be denied. Always use `.strict()` on Zod schemas that map to Firestore documents.
+
+### `allow write: if false` at collection level overrides subcollection rules
+
+When a parent collection has `allow write: if false` and a nested subcollection is defined after it, the subcollection inherits the parent rule unless explicitly overridden. To give a subcollection write access while keeping the parent deny-all, define both explicitly. Note: this inheritance is per-Firestore-rule-file structure, not a general Firestore behavior.

docs/progress.md

@@ -167,3 +167,40 @@ See `docs/learnings.md` for detailed technical decisions and lessons learned.
 - `pnpm typecheck` PASS
 - `pnpm test` PASS
 - `pnpm format:check` PASS
+
+---
+
+## Phase 2 Data Model and Security Rules Foundation (Complete)
+
+**Branch:** `feature/phase-2-data-model-rules`
+**Plan:** See `docs/superpowers/plans/2026-04-17-phase-2-data-model-security-rules.md`
+**Status:** All implementation tasks complete.
+
+### Implementation Summary (Tasks 7-18)
+
+| Task | Description | Status |
+|------|-------------|--------|
+| Task 7 | Dispatches, Users, Responders Firestore rules | ✅ |
+| Task 8 | Public, Audit, Event Collections rules | ✅ |
+| Task 9 | SMS Layer rules | ✅ |
+| Task 10 | Coordination Collections rules | ✅ |
+| Task 11 | Hazard Zones rules | ✅ |
+| Task 12 | Final Rules Cleanup + Default-Deny Audit | ✅ |
+| Task 13 | RTDB Rules + Tests | ✅ |
+| Task 14 | Storage Rules + Tests | ✅ |
+| Task 15 | Composite Indexes deployed (30 indexes) | ✅ |
+| Task 16 | Idempotency Guard Cloud Function helper | ✅ |
+| Task 17 | Rule Coverage CI Gate | ✅ |
+| Task 18 | Schema Migration Runbook | ✅ |
+
+### What was built
+
+- Full Zod schema coverage for every collection in Arch Spec §5.5
+- Reconciled enum literals (ReportStatus 15 states, VisibilityClass `internal`/`public_alertable`, HazardType bare literals)
+- Firestore rules for inbox, triptych, dispatches, users, responders, public collections, SMS, coordination, hazards, events
+- RTDB rules for responder_locations, responder_index, shared_projection
+- Storage rules locked to callable-only uploads with admin-read paths
+- 30 composite indexes in `firestore.indexes.json` per §5.9
+- Idempotency guard Cloud Function helper (`withIdempotency`) with payload-hash deduplication
+- CI rule-coverage gate (`scripts/check-rule-coverage.ts`)
+- Schema migration protocol runbook (`docs/runbooks/schema-migration.md`)

docs/runbooks/schema-migration.md

@@ -0,0 +1,62 @@
+# Schema Migration Protocol
+
+Source of truth: Arch Spec §13.12.
+
+## When this runbook applies
+
+Any breaking change to a Firestore document shape that already has production data:
+
+- Adding a required field without a default
+- Renaming a field
+- Changing an enum value set (removing or renaming literals)
+- Changing field types (e.g., `number` → `string`)
+- Collapsing or splitting collections
+
+Purely additive optional fields do NOT trigger this protocol — they follow the normal PR workflow.
+
+## Stage 1 — Plan document
+
+Before any code change, write a short migration plan covering:
+
+1. **Old schema** (link to current Zod schema in `packages/shared-validators/src/<file>.ts`)
+2. **New schema** (PR-drafted definition)
+3. **Trigger compatibility matrix**: for each Cloud Function that reads or writes this collection, which branch handles old vs new
+4. **Backfill strategy**: batched scheduled function, size limits, throttle
+5. **Rollback plan**: exact `firebase deploy --only functions:<name>` command to revert
+6. **Monitoring signals**: what dashboards confirm progress; what alert fires if progress stalls
+
+The plan lives in `docs/runbooks/migrations/<YYYY-MM-DD>-<schema>.md`.
+
+## Stage 2 — `schemaVersion` guard
+
+Every document class carries `schemaVersion: number`. New writes must increment. Read paths must accept both old and new versions during the migration window.
+
+## Stage 3 — Migration window
+
+Default 30 days. Both versions accepted; triggers have branched code paths with explicit unit tests for each branch. The date is recorded in `system_config/migration_progress/<schemaKey>`.
+
+## Stage 4 — Backfill
+
+A scheduled function reads old-version documents in batches, rewrites them to the new shape inside a transaction, and updates `system_config/migration_progress/<schemaKey>.completed`. Runs during low-traffic hours (01:00–05:00 Asia/Manila). Respect Firestore write quotas — default 500 docs/sec.
+
+## Stage 5 — Cutover
+
+When backfill `completed == true` AND zero old-version writes for 7 consecutive days, remove the old-version branches in a follow-up PR. This PR must include:
+
+- A counting query proving zero old-version documents remain
+- A screenshot of the monitoring dashboard showing the steady-state
+- The `system_config/migration_progress/<schemaKey>` document marked `closed: true`
+
+## Stage 6 — Rollback
+
+During the migration window, rollback is a function-only redeploy from the prior tag. Post-window rollback requires a reverse migration plan — treat it as a new migration.
+
+## Definition of done
+
+The migration is not complete until:
+
+- [ ] Counting query confirms zero old-version documents
+- [ ] Monitoring signal shows zero old-version writes for 7 consecutive days
+- [ ] Old-version trigger branches removed
+- [ ] Runbook entry closed in `docs/runbooks/migrations/`
+- [ ] Post-migration review logged in `docs/learnings.md`

functions/package.json

@@ -11,6 +11,10 @@
     "test": "vitest run --passWithNoTests",
     "test:unit": "vitest run src/__tests__/phase1-auth.test.ts",
     "test:rules": "vitest run src/__tests__/firestore.rules.test.ts",
+    "test:rules:firestore": "vitest run 'src/__tests__/rules/**/*.rules.test.ts'",
+    "test:rules:rtdb": "vitest run 'src/__tests__/rtdb.rules.test.ts'",
+    "test:rules:storage": "vitest run 'src/__tests__/storage.rules.test.ts'",
+    "test:rules:coverage": "tsx ../scripts/check-rule-coverage.ts",
     "serve": "pnpm build && firebase emulators:start --only functions",
     "shell": "pnpm build && firebase functions:shell",
     "deploy": "echo 'Use deploy:dev, deploy:staging, or deploy:prod' && exit 1",

functions/src/__tests__/helpers/rules-harness.ts

@@ -0,0 +1,34 @@
+import { readFileSync } from 'node:fs'
+import { resolve } from 'node:path'
+import { initializeTestEnvironment, type RulesTestEnvironment } from '@firebase/rules-unit-testing'
+
+const FIRESTORE_RULES_PATH = resolve(process.cwd(), '../infra/firebase/firestore.rules')
+const RTDB_RULES_PATH = resolve(process.cwd(), '../infra/firebase/database.rules.json')
+const STORAGE_RULES_PATH = resolve(process.cwd(), '../infra/firebase/storage.rules')
+
+export async function createTestEnv(projectId: string): Promise<RulesTestEnvironment> {
+  return initializeTestEnvironment({
+    projectId,
+    firestore: {
+      rules: readFileSync(FIRESTORE_RULES_PATH, 'utf8'),
+    },
+    database: {
+      rules: readFileSync(RTDB_RULES_PATH, 'utf8'),
+    },
+    storage: {
+      rules: readFileSync(STORAGE_RULES_PATH, 'utf8'),
+    },
+  })
+}
+
+export function authed(env: RulesTestEnvironment, uid: string, claims: Record<string, unknown>) {
+  return env.authenticatedContext(uid, claims).firestore() as unknown as ReturnType<
+    RulesTestEnvironment['authenticatedContext']
+  >['firestore']
+}
+
+export function unauthed(env: RulesTestEnvironment) {
+  return env.unauthenticatedContext().firestore() as unknown as ReturnType<
+    RulesTestEnvironment['unauthenticatedContext']
+  >['firestore']
+}

functions/src/__tests__/helpers/seed-factories.ts

@@ -0,0 +1,95 @@
+import { type RulesTestEnvironment } from '@firebase/rules-unit-testing'
+import { setDoc, doc } from 'firebase/firestore'
+
+export const ts = 1713350400000
+
+export async function seedActiveAccount(
+  env: RulesTestEnvironment,
+  opts: {
+    uid: string
+    role: 'citizen' | 'responder' | 'municipal_admin' | 'agency_admin' | 'provincial_superadmin'
+    municipalityId?: string
+    agencyId?: string
+    permittedMunicipalityIds?: string[]
+    accountStatus?: 'active' | 'suspended' | 'disabled'
+  },
+): Promise<void> {
+  await env.withSecurityRulesDisabled(async (ctx) => {
+    const db = ctx.firestore()
+    await setDoc(doc(db, 'active_accounts', opts.uid), {
+      uid: opts.uid,
+      role: opts.role,
+      accountStatus: opts.accountStatus ?? 'active',
+      municipalityId: opts.municipalityId ?? null,
+      agencyId: opts.agencyId ?? null,
+      permittedMunicipalityIds: opts.permittedMunicipalityIds ?? [],
+      mfaEnrolled: true,
+      lastClaimIssuedAt: ts,
+      updatedAt: ts,
+    })
+  })
+}
+
+export function staffClaims(opts: {
+  role: 'municipal_admin' | 'agency_admin' | 'provincial_superadmin' | 'responder' | 'citizen'
+  municipalityId?: string
+  agencyId?: string
+  permittedMunicipalityIds?: string[]
+  accountStatus?: 'active' | 'suspended'
+}): Record<string, unknown> {
+  return {
+    role: opts.role,
+    accountStatus: opts.accountStatus ?? 'active',
+    municipalityId: opts.municipalityId ?? null,
+    agencyId: opts.agencyId ?? null,
+    permittedMunicipalityIds: opts.permittedMunicipalityIds ?? [],
+  }
+}
+
+export async function seedReport(
+  env: RulesTestEnvironment,
+  reportId: string,
+  overrides: Partial<Record<string, unknown>> = {},
+): Promise<void> {
+  await env.withSecurityRulesDisabled(async (ctx) => {
+    const db = ctx.firestore()
+    await setDoc(doc(db, 'reports', reportId), {
+      municipalityId: 'daet',
+      reporterRole: 'citizen',
+      reportType: 'flood',
+      severity: 'high',
+      status: 'verified',
+      mediaRefs: [],
+      description: 'seeded',
+      submittedAt: ts,
+      retentionExempt: false,
+      visibilityClass: 'internal',
+      visibility: { scope: 'municipality', sharedWith: [] },
+      source: 'web',
+      hasPhotoAndGPS: false,
+      schemaVersion: 1,
+      ...overrides,
+    })
+    await setDoc(doc(db, 'report_ops', reportId), {
+      municipalityId: 'daet',
+      status: 'verified',
+      severity: 'high',
+      createdAt: ts,
+      agencyIds: [],
+      activeResponderCount: 0,
+      requiresLocationFollowUp: false,
+      visibility: { scope: 'municipality', sharedWith: [] },
+      updatedAt: ts,
+      schemaVersion: 1,
+      ...(overrides.opsOverrides as Record<string, unknown> | undefined),
+    })
+    await setDoc(doc(db, 'report_private', reportId), {
+      municipalityId: 'daet',
+      reporterUid: 'citizen-1',
+      isPseudonymous: true,
+      publicTrackingRef: 'ref-12345',
+      createdAt: ts,
+      schemaVersion: 1,
+    })
+  })
+}