-
Notifications
You must be signed in to change notification settings - Fork 33
PER-14389: Migrate from Permit Cloud — self-service export + on-prem import guide #647
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
EliMoshkovich
wants to merge
8
commits into
master
Choose a base branch
from
PER-14389-migrate-from-cloud-import-guide
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+421
−1
Open
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
c5a226c
PER-14389: add "Migrate from Permit Cloud" on-prem import guide
EliMoshkovich dc345be
PER-14389: round-2 review fixes — disclosure + customer-friendliness
EliMoshkovich 702f53d
PER-14389: single-transaction import (FK-graph analysis fix)
EliMoshkovich 2c8ac07
PER-14389: final proofread fixes — version-safe atomicity + clarity
EliMoshkovich c3f3ff7
PER-14389: review fixes — drop /api prefix, portable kubectl scale form
EliMoshkovich 1bdbb2e
PER-14389: self-service export via Organization API key
EliMoshkovich 1a88974
PER-14389: remove audit-log/OpenSearch references from the migration …
EliMoshkovich 80288b6
PER-14389: async export flow in the migration guide
EliMoshkovich File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,385 @@ | ||
| --- | ||
| title: "Migrate from Permit Cloud" | ||
| description: "Import your Permit Cloud data into your on-premises deployment" | ||
| --- | ||
|
|
||
| :::info Enterprise Only | ||
| This section is only relevant to Enterprise customers who acquired an on-prem license. | ||
| ::: | ||
|
|
||
| # Migrate from Permit Cloud to On-Premises | ||
|
|
||
| This guide explains how to import your existing Permit Cloud (SaaS) data into your on-premises | ||
| Permit Platform deployment, so you can switch environments without rebuilding your authorization | ||
| model from scratch. | ||
|
|
||
| Migration is a coordinated process: the Permit team exports your organization's data from Permit | ||
| Cloud and delivers it to you as a **migration data package**. You then import that package into | ||
| your own deployment using the steps below. | ||
|
|
||
| :::info At a glance | ||
| The migration has five phases: **request the package** → **freeze & export** (done by Permit) → | ||
| **import** (steps 1-9 below) → **verify** → **point your applications at on-prem**. Your on-prem | ||
| platform is offline between steps 2 and 9 - for a typical organization the import itself takes | ||
| well under an hour, and your Permit account team is available to assist during the window. You | ||
| are done when the row counts match the manifest and the UI checks in | ||
| [Verify the Import](#verify-the-import) pass. | ||
| ::: | ||
|
|
||
| ## Requesting Your Migration Package | ||
|
|
||
| Contact your Permit account team to schedule the migration. You will agree on: | ||
|
|
||
| - **A migration window** - plan a short change freeze on your Permit Cloud workspace right before | ||
| the export, so the package captures your final state | ||
| - **Audit log history** (optional) - historical audit logs are not part of the standard package. | ||
| Permit Cloud retains them for a limited period, so history that is not exported around the | ||
| migration window cannot be recovered later - if you want it, request it when scheduling and | ||
| confirm the retention window with your account team | ||
| - **Version alignment** - Permit confirms that your on-prem installer version matches the data | ||
| package (the package's `manifest.json` records the source schema version) | ||
|
|
||
| Permit then performs the export and sends you a **secure, time-limited download link** to the | ||
| package. Your Permit contact will tell you when the link expires - download it promptly. | ||
|
|
||
| ## What Gets Migrated | ||
|
|
||
| The migration data package contains a complete snapshot of your organization's data: | ||
|
|
||
| - **Authorization model** - resources, actions, roles, permissions, relations, condition sets | ||
| - **Directory data** - users, tenants, role assignments, relationship tuples, resource instances | ||
| - **Configuration** - PDP configurations, webhooks, proxy configs, Permit Elements, email templates | ||
| - **Workspace members** - your team members and their access levels | ||
| - **API keys** - your existing `permit_key_*` tokens migrate as-is; your applications keep using | ||
| the same keys and only the endpoints change (see | ||
| [Point Your Applications at On-Prem](#point-your-applications-at-on-prem)) | ||
|
|
||
| **What is not migrated:** | ||
|
|
||
| - **Login credentials** - on-prem uses its own identity provider (Keycloak). Team members sign in | ||
| again and regain their workspace access on first login, provided they use the **same verified | ||
| email address** they used in Permit Cloud (see | ||
| [Team member access](#team-member-access-after-import)) | ||
| - **Audit log history** - optional, by request only (see | ||
| [Requesting Your Migration Package](#requesting-your-migration-package)). New audit logs start | ||
| flowing as soon as your on-prem deployment is running | ||
| - **Runtime state** - PDPs re-register automatically, and generated policy is rebuilt from the | ||
| imported data on startup | ||
|
|
||
| :::caution Handle the package securely | ||
| The migration data package contains your API keys. Store it securely, restrict access to it, and | ||
| delete all copies once the import is verified. | ||
| ::: | ||
|
|
||
| ## Package Contents | ||
|
|
||
| ``` | ||
| permit-migration-<org>-<date>/ | ||
| ├── data/ | ||
| │ ├── ... (one CSV per table) | ||
| │ └── v2_identity.csv # reference only - do NOT import (see below) | ||
| ├── manifest.json # row counts, checksums, and the import order | ||
| ├── policy-repo/ # only if you used the default Permit-managed policy repo | ||
| └── README.md | ||
| ``` | ||
|
|
||
| :::warning Do not import `v2_identity.csv` | ||
| This file is a reference snapshot of your previous cloud login identities. It is intentionally | ||
| excluded from the import steps below - importing it can prevent team members from regaining | ||
| access when they first sign in to your on-prem deployment. | ||
| ::: | ||
|
|
||
| ### Policy repository | ||
|
|
||
| Your policy-as-code (Rego) lives in a Git repository, not in the database: | ||
|
|
||
| - **If you connected your own Git repository in Permit Cloud (GitOps)** - keep using it. Point | ||
| your on-prem deployment at the same repository in `values.yaml` | ||
| - **If you used the default Permit-managed repository** - your package includes a `policy-repo/` | ||
| clone. Push it to a Git server you control and point your on-prem deployment at it, as | ||
| described in the package `README.md` | ||
|
|
||
| Any custom Rego you wrote exists **only** in the Git repository - it is not part of the database | ||
| import, so make sure the repository is connected before you rely on custom policies. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - A healthy on-premises Permit Platform deployment | ||
| (see the [Installation Guide](./installation.mdx)) - all pods `Running`, migrations job | ||
| `Completed` | ||
| - The installer version confirmed by your Permit contact to match your migration data package | ||
| - Access to the Kubernetes cluster with `kubectl` configured | ||
| - `jq`, `tar`, and `sha256sum` (macOS: `shasum`) available on the workstation where you extract | ||
| the package | ||
| - The migration data package downloaded from the secure link provided by Permit | ||
|
|
||
| ## Step-by-Step Import | ||
|
|
||
| ### 1. Verify the package integrity | ||
|
|
||
| Extract the package and verify every file against the manifest checksums: | ||
|
|
||
| ```bash | ||
| tar -xzf permit-migration-<org>-<date>.tar.gz | ||
| cd permit-migration-<org>-<date>/ | ||
|
|
||
| # Verify checksums (on macOS use: shasum -a 256 -c -) | ||
| jq -r '.tables[] | "\(.sha256) \(.file)"' manifest.json | sha256sum -c - | ||
| ``` | ||
|
|
||
| Every line should print `OK`. If any file fails verification, re-download the package before | ||
| continuing. | ||
|
|
||
| ### 2. Stop services that write to the database | ||
|
|
||
| First note the current replica counts, so you can restore them in step 9: | ||
|
|
||
| ```bash | ||
| kubectl get deployment -n permit-platform \ | ||
| permit-backend-v2 celery-general permit-dl-enricher-v2 | ||
| ``` | ||
|
|
||
| Then pause these services so nothing changes mid-import: | ||
|
|
||
| ```bash | ||
| kubectl scale deployment -n permit-platform \ | ||
| permit-backend-v2 celery-general permit-dl-enricher-v2 --replicas=0 | ||
| ``` | ||
|
|
||
| ### 3. Back up the database | ||
|
|
||
| Take a backup before importing, so you can roll back cleanly if anything goes wrong: | ||
|
|
||
| ```bash | ||
| PG_POD=$(kubectl get pods -n permit-platform -l app=postgres \ | ||
| --field-selector=status.phase=Running -o jsonpath='{.items[0].metadata.name}') | ||
|
|
||
| kubectl exec -n permit-platform $PG_POD -- \ | ||
| pg_dump -U permit -d permit -F c -f /tmp/pre-migration.dump | ||
| kubectl cp permit-platform/$PG_POD:/tmp/pre-migration.dump ./pre-migration.dump | ||
| kubectl exec -n permit-platform $PG_POD -- rm -f /tmp/pre-migration.dump | ||
| ``` | ||
|
|
||
| If your PostgreSQL runs outside the cluster, use your own snapshot mechanism instead. | ||
|
|
||
| **To roll back later** (with the services still scaled to 0): | ||
|
|
||
| ```bash | ||
| kubectl cp ./pre-migration.dump permit-platform/$PG_POD:/tmp/pre-migration.dump | ||
| kubectl exec -n permit-platform $PG_POD -- \ | ||
| pg_restore -U permit -d permit --clean --if-exists /tmp/pre-migration.dump | ||
| ``` | ||
|
|
||
| ### 4. Copy the data files to the PostgreSQL pod | ||
|
|
||
| ```bash | ||
| kubectl exec -n permit-platform $PG_POD -- mkdir -p /tmp/import | ||
| kubectl cp data permit-platform/$PG_POD:/tmp/import/data | ||
| ``` | ||
|
|
||
| ### 5. Import the data | ||
|
|
||
| The manifest lists every table to import (`v2_identity` is deliberately not in the list). The | ||
| whole import runs as **one database transaction**: if anything fails, nothing is imported - fix | ||
| the cause and simply run it again. psql prints `COPY <row-count>` for each table as it loads: | ||
|
|
||
| ```bash | ||
| { | ||
| echo '\set ON_ERROR_STOP on' | ||
| echo 'BEGIN;' | ||
| echo "SET session_replication_role = 'replica';" | ||
| jq -r '.import.tables_in_order[]' manifest.json | while read -r TABLE; do | ||
| echo "\\COPY v2.${TABLE} FROM '/tmp/import/data/${TABLE}.csv' WITH CSV HEADER" | ||
| done | ||
| echo 'COMMIT;' | ||
| } | kubectl exec -i -n permit-platform $PG_POD -- psql -U permit -d permit | ||
| ``` | ||
|
|
||
| :::info Why referential checks are turned off during the import | ||
| Some Permit tables reference each other in both directions, so no import order can satisfy | ||
| per-row referential checks - the import therefore turns them off for this session | ||
| (`session_replication_role`). This is safe: your package was exported as a single consistent | ||
| database snapshot. The one known exception - references to members who left your organization | ||
| before the export - is cleaned up in step 6. Use this technique only for this import - not for | ||
| ad-hoc database changes. | ||
| ::: | ||
|
|
||
| ### 6. Clear references to departed members | ||
|
|
||
| Clear any references to members that are not part of the imported data (these come from | ||
| members who left your organization before the export): | ||
|
|
||
| ```bash | ||
| kubectl exec -n permit-platform $PG_POD -- psql -U permit -d permit -c " | ||
| UPDATE v2.v2_api_key k SET created_by_member_id = NULL | ||
| WHERE created_by_member_id IS NOT NULL | ||
| AND NOT EXISTS (SELECT 1 FROM v2.v2_member m WHERE m.id = k.created_by_member_id); | ||
| UPDATE v2.v2_user_invite ui SET member_id = NULL | ||
| WHERE member_id IS NOT NULL | ||
| AND NOT EXISTS (SELECT 1 FROM v2.v2_member m WHERE m.id = ui.member_id);" | ||
| ``` | ||
|
|
||
| ### 7. Review cloud-specific configuration | ||
|
|
||
| Run the short cloud-reference cleanup included in your package (it removes settings that only | ||
| apply to Permit Cloud) - see the **Cleanup** section of the package `README.md`. | ||
|
|
||
| Afterwards, review these imported settings - they may carry over values that only made sense in | ||
| Permit Cloud: | ||
|
|
||
| - **SSO connections** - imported SSO settings reference the Permit Cloud login integration. | ||
| Reconfigure SSO against your on-prem Keycloak (or delete the stale entries) | ||
| - **Webhooks** - your on-prem deployment sends webhooks from a different source IP; update | ||
| firewall allowlists on the receiving side if needed | ||
| - **PDP configurations** - review any PDP settings that point at Permit Cloud endpoints | ||
|
|
||
| ### 8. Grant your admin access to the imported organization | ||
|
|
||
| The admin account you created during installation is not a member of the imported organization | ||
| yet. Make it a superuser so it can see and manage all organizations, including the imported one: | ||
|
|
||
| ```bash | ||
| kubectl exec -n permit-platform $PG_POD -- psql -U permit -d permit -c " | ||
| UPDATE v2.v2_member SET is_superuser = true WHERE email = '<your-admin-email>';" | ||
| ``` | ||
|
|
||
| After you restart the services in step 9, log out and log back in for the change to take | ||
| effect. | ||
|
|
||
| :::caution Superuser applies platform-wide | ||
| A superuser can see and manage **all** organizations on your deployment, not just the imported | ||
| one. Grant it only to your installation admin account. | ||
| ::: | ||
|
|
||
| ### 9. Clean up and restart | ||
|
|
||
| ```bash | ||
| # Remove the import files from the pod | ||
| kubectl exec -n permit-platform $PG_POD -- rm -rf /tmp/import | ||
|
|
||
| # Scale each service back to the replica count you noted in step 2 | ||
| kubectl scale deployment -n permit-platform permit-backend-v2 --replicas=<count-from-step-2> | ||
| kubectl scale deployment -n permit-platform celery-general --replicas=<count-from-step-2> | ||
| kubectl scale deployment -n permit-platform permit-dl-enricher-v2 --replicas=<count-from-step-2> | ||
| ``` | ||
|
|
||
| ## Verify the Import | ||
|
|
||
| Compare the imported row counts against the values in `manifest.json`. The counts must be | ||
| filtered to your imported organization - your deployment may already contain other organizations | ||
| (for example, the one created by your installation admin): | ||
|
|
||
| ```bash | ||
| # Your organization id is recorded in the manifest | ||
| ORG_ID=$(jq -r '.org_id' manifest.json) | ||
|
|
||
| kubectl exec -n permit-platform $PG_POD -- psql -U permit -d permit -c " | ||
| SELECT 'projects' AS table_name, count(*) FROM v2.v2_project WHERE org_id = '$ORG_ID' | ||
| UNION ALL SELECT 'environments', count(*) FROM v2.v2_environment WHERE org_id = '$ORG_ID' | ||
| UNION ALL SELECT 'users', count(*) FROM v2.v2_user WHERE org_id = '$ORG_ID' | ||
| UNION ALL SELECT 'tenants', count(*) FROM v2.v2_tenant WHERE org_id = '$ORG_ID' | ||
| UNION ALL SELECT 'roles', count(*) FROM v2.v2_role WHERE org_id = '$ORG_ID' | ||
| UNION ALL SELECT 'relationship_tuples', count(*) FROM v2.v2_relationship_tuple WHERE org_id = '$ORG_ID' | ||
| UNION ALL SELECT 'api_keys', count(*) FROM v2.v2_api_key WHERE org_id = '$ORG_ID';" | ||
| ``` | ||
|
|
||
| Then confirm in the UI: | ||
|
|
||
| 1. Log in to your on-prem dashboard as the admin - the imported organization should be visible | ||
| 2. Open the **Members** page - your team members appear with their original access levels | ||
| 3. Open the **Policy** page of a migrated environment - your resources, roles, and permissions | ||
| are present | ||
|
|
||
| ## Team Member Access After Import | ||
|
|
||
| Your team members' profiles and permission levels are fully migrated. To sign in, each member | ||
| authenticates against your on-prem identity provider (Keycloak) **using the same email address | ||
| they used in Permit Cloud** - the platform automatically links them to their migrated profile and | ||
| restores their workspace access on first login. | ||
|
|
||
| Depending on how you configured Keycloak, members either: | ||
|
|
||
| - **Sign in through your corporate SSO** (if you federated your IdP with Keycloak), or | ||
| - **Self-register** on the on-prem login page with their work email | ||
|
|
||
| :::caution Email verification is required | ||
| The automatic account linking only happens for verified email addresses. Make sure email | ||
| verification is enabled in your Keycloak realm (or that your federated IdP passes | ||
| `email_verified=true`). A member who registers with a different or unverified email will start | ||
| with a fresh account and no access. | ||
| ::: | ||
|
|
||
| ## Point Your Applications at On-Prem | ||
|
|
||
| Your API keys were migrated as-is, so your applications keep using the same tokens - they just | ||
| need to talk to your on-prem deployment instead of Permit Cloud. Two endpoints change: | ||
|
|
||
| 1. **The PDP address** - deploy a PDP against your on-prem platform (see | ||
| [PDP Deployment](./pdp-deployment.mdx)) and point your application at it | ||
| 2. **The API address** - point the SDK's API URL at your on-prem platform, so management calls | ||
| (like syncing users) reach your deployment and not Permit Cloud | ||
|
|
||
| ```python | ||
| # Before (Permit Cloud) | ||
| permit = Permit( | ||
| pdp="https://cloudpdp.api.permit.io", | ||
| token="permit_key_XXXX", | ||
| ) | ||
|
|
||
| # After (On-Premises) - same key, new endpoints | ||
| permit = Permit( | ||
| pdp="http://<your-pdp-address>:7766", # the PDP you deployed on-prem | ||
| api_url="https://<your-permit-domain>/api", # your on-prem Permit API | ||
|
EliMoshkovich marked this conversation as resolved.
Outdated
|
||
| token="permit_key_XXXX", | ||
| ) | ||
| ``` | ||
|
|
||
| :::caution Do not skip the API URL | ||
| If `api_url` still points at Permit Cloud, your migrated API keys remain valid there - your | ||
| application would silently keep writing users and permissions to your old cloud workspace | ||
| instead of your on-prem deployment. | ||
| ::: | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### The import fails partway through | ||
|
|
||
| The import runs in a single transaction, so a failure means **nothing was imported** - the | ||
| database is unchanged. Fix the cause shown in the error (usually a missing file or a wrong | ||
| path), then run step 5 again from the start. If you are unsure of the state, roll back using | ||
| the backup from step 3, or contact Permit support. | ||
|
|
||
| ### The import prints no `COPY` lines | ||
|
|
||
| Your package was produced before manifests included the import list | ||
| (`.import.tables_in_order`). Check the package `README.md` for the ready-to-run import script | ||
| matching your package version, or contact Permit support. | ||
|
|
||
| ### The imported organization is not visible in the UI | ||
|
|
||
| Complete step 8 (superuser grant), then log out and log back in. You can list all organizations | ||
| with: | ||
|
|
||
| ```bash | ||
| kubectl exec -n permit-platform $PG_POD -- psql -U permit -d permit -c \ | ||
| "SELECT id, name, key FROM v2.v2_organization;" | ||
| ``` | ||
|
|
||
| ### A team member logs in and sees an empty workspace | ||
|
|
||
| Their login email does not match their Permit Cloud email, or their email is not verified. Check | ||
| the email on the **Members** page, and verify your Keycloak realm has email verification enabled. | ||
|
|
||
| ## Support | ||
|
|
||
| Need help with your migration? | ||
|
|
||
| - 📧 **Email**: [support@permit.io](mailto:support@permit.io) | ||
| - 💬 **Slack**: [Join our community](https://io.permit.io/slack) | ||
|
|
||
| ## See Also | ||
|
|
||
| - [Installation Guide](./installation.mdx) | ||
| - [PDP Deployment](./pdp-deployment.mdx) | ||
| - [Management Guide](./management.mdx) | ||
| - [Troubleshooting](./troubleshooting.mdx) | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.