ci: nightly pg_dump artifacts and restore runbook - #122
Conversation
|
@namdamdoi68-oss Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
Signed-off-by: namdamdoi68-oss <namdamdoi68@gmail.com>
77f5b01 to
18b5969
Compare
Miracle656
left a comment
There was a problem hiding this comment.
The design here is right and the docs are genuinely good — the direct-vs-pooler distinction for Neon, skipping a network whose secret is absent, --format=custom for selective restore, and scheduling at 03:00 to stay clear of the 04:00 load test are all considered choices. Thank you for the workflow test too; that is rarer than it should be.
There is one problem, and unfortunately it is a blocking one.
This repository is public, so the artifacts are public
actions/upload-artifact inherits the repository's visibility. On a public repo the Actions tab is world-readable and artifacts can be downloaded by anyone with a GitHub account. This workflow would therefore publish a complete nightly dump of the production database, retained 14 days.
You were careful with the secrets going in — "Do not put these URIs in the repo, .env.example, or workflow YAML" is exactly right. The gap is the data coming out.
What is in the dump
Checking prisma/schema.prisma, it is not only market data:
model Webhook {
url String
secret String // plaintext HMAC signing key
}
model ApiKey {
hash String @unique
label String
ratePerMin Int
monthlyQuotaCents Int?
}webhooks.secretis plaintext. It is the signing key subscribers use to verify our deliveries. Anyone who downloads a dump can forge webhook payloads that pass signature verification, aimed at the endpoints inwebhooks.url— which are in the same dump.api_keysexposes hashes (credit for hashing them), plus customer labels and commercial quota terms.
Price history is re-derivable from chain and harmless. These two tables are not.
Options, roughly in order of preference
- Check whether this is needed at all. Neon has branching and point-in-time restore built in. If the retention window there already covers the recovery objective, the strongest version of this PR may be
docs/backup-restore.mddocumenting that, with no workflow. - Send dumps somewhere private — S3/R2/Backblaze with credentials in secrets. Keeps the schedule and the runbook; changes only the destination.
- Encrypt before upload. Smallest change that preserves the current shape: pipe through
ageorgpgwith the recipient key in a secret, so the artifact is public but inert. Then the runbook grows a decrypt step and a note on where the private key lives. - Excluding the two tables would technically stop the leak, but it also stops this being a restore path, so I would not go that way.
Whichever you pick, worth adding to docs/backup-restore.md: a dump of this database contains live credentials and must never land anywhere world-readable. That sentence is the thing that stops it regressing later.
Happy to merge once the destination changes — the rest of this is ready.
Signed-off-by: namdamdoi68-oss <namdamdoi68@gmail.com>
|
Public artifacts were the gap — dumps now go through age before upload, so the Actions tab only has ciphertext. AGE_RECIPIENT is the public key secret; the identity stays off GitHub. Decrypt + the webhook-secret warning are in docs/backup-restore.md. |
Nightly pg_dump of mainnet/testnet to Actions artifacts (14 days). Restore into a fresh DB, then swap DATABASE_URL — docs/backup-restore.md.
Closes #119