Add Repo Radius feature specification - #12078
Conversation
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
There was a problem hiding this comment.
Pull request overview
Adds a new design note / feature specification for “Repo Radius”, describing a proposed delivery model where Radius runs ephemerally inside GitHub Actions with state persisted in GitHub-native storage, to reduce the Kubernetes/platform-engineering setup burden for developers.
Changes:
- Introduces a comprehensive feature specification covering goals, non-goals, personas, scenarios, dependencies/risks, and open questions.
- Documents the intended end-to-end UX interaction model via GitHub APIs (environments, OIDC auth, workflow dispatch/execution, artifact-based outputs).
- Summarizes key technical investments required (external cluster deployment, externalized state store, standardized workflow contract, credential integration).
…up / rad shutdown) (#12214) # Repo Radius: externalize control-plane and Terraform state (`rad startup` / `rad shutdown`) ## Description Implements **Investment 2 of the Repo Radius feature spec — externalization of the Radius data store**, scoped to the state-storage aspects only (control-plane PostgreSQL state and Terraform recipe state). It adds two kind-agnostic commands, `rad startup` and `rad shutdown`, that back up and restore all durable Radius state across an ephemeral control plane, plus the Helm and Terraform-state plumbing they depend on. Design note: `eng/design-notes/2026-06-repo-radius-state-storage.md`. There is **no dedicated workspace kind**. The commands operate on the current workspace's Kubernetes context like any other command and do not create or delete clusters or install Radius — cluster lifecycle is the caller's responsibility. ## What's included The change is organized as three logical commits: 1. **PostgreSQL enablement (closes the chart gaps for `database.enabled=true`)** - UCP, Applications RP, and Dynamic RP configmaps/deployments now switch to the `postgresql` provider and inject `POSTGRES_PASSWORD` when `database.enabled=true` (previously hardcoded to `apiserver`). - Adds an init-db ConfigMap (option 3 from #8398) that creates the per-RP databases, users, and tables before the servers start. - Fixes the `POSTGRES_DB` secret value and pins the postgres image to `16-alpine`. - Fixes the `databaseProvider` URL env-var substitution in `factory.go`. - Adds helm-unittest coverage and a unit test for the env-var helper. 2. **Terraform recipe state backup/restore (`pkg/cli/tfstate`)** - Terraform recipes store state in Kubernetes Secrets (`tfstate=true`), not in PostgreSQL, so that state is lost on teardown. This package exports and restores those Secrets, including the chunked `tfstate-{workspace}-{suffix}-{index}` Secrets for large state. 3. **`rad startup` / `rad shutdown` + end-to-end lifecycle test** - `pkg/cli/pgbackup`: control-plane PostgreSQL backup/restore via `kubectl exec pg_dump/psql`. - `pkg/cli/gitstate`: persists the state directory to a `radius-state` git orphan branch in an isolated worktree; the backup push fails loudly when a remote is configured (a failed push would otherwise be silent data loss) and tolerates the no-remote local/test case. - `rad shutdown` backs up both stores then commits and pushes; `rad startup` waits for the database then restores both stores. ## Testing Unit tests cover the chart rendering, the Terraform-state round-trip (fake Kubernetes client), the git orphan-branch worktree behaviour (real temporary repos), and the command runners (hand-written fakes). All pass, along with the existing 85 Helm chart tests. ### End-to-end test dependency (please read) The end-to-end lifecycle test lives at `test/functional-portable/statestore` and exercises the full path that this work exists to protect: install → deploy a Terraform-backed resource → `rad shutdown` → tear down → reinstall → `rad startup` → **deploy an update to the same resource**. The update is the path that fails when Terraform state is lost. **This test depends on the separate Repo Radius workflow code (in flight) that creates the ephemeral cluster, installs Radius, and runs the deploy.** Because `rad startup` / `rad shutdown` are intentionally kind-agnostic and do not manage cluster lifecycle, the test needs that workflow to stand up the environment. Until it lands, the test drives the install/uninstall itself and is **gated behind the `RADIUS_STATE_E2E` environment variable**, so it does not run in the normal functional suite. Once the shared cluster-create + deploy workflow is merged, the test's install/uninstall helpers should be re-pointed at that code rather than duplicating it. ## Related - Feature spec: Repo Radius (#12078) - Relates to #8096 (External data store for Radius), #8398 (Postgres DB initialization) - Supersedes the unmerged prototype #11457 ## Type of change This pull request adds new features (state externalization commands) for Radius. --------- Signed-off-by: Sylvain Niles <sylvainniles@microsoft.com>
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
|
The consolidation to a single 1. Offer a canonical
|
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
…ands Align the Repo Radius workflow with the updated spec (PR #12078): adopt the two-action model naming and the renamed GitHub Actions variables. - Rename .github/extension/radius-deploy.yml -> radius-run-rad-commands.yml and set its name to "Radius - Run rad Commands" (this is the run-rad-commands engine). - Rename the cloud/cluster Actions variables to the spec contract: - AWS_IAM_ROLE_ARN -> AWS_ROLE_ARN - RADIUS_K8S_CLUSTER -> AZURE_AKS_CLUSTER_NAME (Azure) / AWS_EKS_CLUSTER_NAME (AWS) - RADIUS_K8S_NAMESPACE -> KUBERNETES_NAMESPACE - Update README.md, the radius-deploy skill, and the deploy-workflow design note to match the new file name, workflow name, and variables. No behavior change beyond the variable/name renames. Signed-off-by: Sylvain Niles <sylvainniles@microsoft.com>
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
…up / rad shutdown) (#12214) # Repo Radius: externalize control-plane and Terraform state (`rad startup` / `rad shutdown`) ## Description Implements **Investment 2 of the Repo Radius feature spec — externalization of the Radius data store**, scoped to the state-storage aspects only (control-plane PostgreSQL state and Terraform recipe state). It adds two kind-agnostic commands, `rad startup` and `rad shutdown`, that back up and restore all durable Radius state across an ephemeral control plane, plus the Helm and Terraform-state plumbing they depend on. Design note: `eng/design-notes/2026-06-repo-radius-state-storage.md`. There is **no dedicated workspace kind**. The commands operate on the current workspace's Kubernetes context like any other command and do not create or delete clusters or install Radius — cluster lifecycle is the caller's responsibility. ## What's included The change is organized as three logical commits: 1. **PostgreSQL enablement (closes the chart gaps for `database.enabled=true`)** - UCP, Applications RP, and Dynamic RP configmaps/deployments now switch to the `postgresql` provider and inject `POSTGRES_PASSWORD` when `database.enabled=true` (previously hardcoded to `apiserver`). - Adds an init-db ConfigMap (option 3 from #8398) that creates the per-RP databases, users, and tables before the servers start. - Fixes the `POSTGRES_DB` secret value and pins the postgres image to `16-alpine`. - Fixes the `databaseProvider` URL env-var substitution in `factory.go`. - Adds helm-unittest coverage and a unit test for the env-var helper. 2. **Terraform recipe state backup/restore (`pkg/cli/tfstate`)** - Terraform recipes store state in Kubernetes Secrets (`tfstate=true`), not in PostgreSQL, so that state is lost on teardown. This package exports and restores those Secrets, including the chunked `tfstate-{workspace}-{suffix}-{index}` Secrets for large state. 3. **`rad startup` / `rad shutdown` + end-to-end lifecycle test** - `pkg/cli/pgbackup`: control-plane PostgreSQL backup/restore via `kubectl exec pg_dump/psql`. - `pkg/cli/gitstate`: persists the state directory to a `radius-state` git orphan branch in an isolated worktree; the backup push fails loudly when a remote is configured (a failed push would otherwise be silent data loss) and tolerates the no-remote local/test case. - `rad shutdown` backs up both stores then commits and pushes; `rad startup` waits for the database then restores both stores. ## Testing Unit tests cover the chart rendering, the Terraform-state round-trip (fake Kubernetes client), the git orphan-branch worktree behaviour (real temporary repos), and the command runners (hand-written fakes). All pass, along with the existing 85 Helm chart tests. ### End-to-end test dependency (please read) The end-to-end lifecycle test lives at `test/functional-portable/statestore` and exercises the full path that this work exists to protect: install → deploy a Terraform-backed resource → `rad shutdown` → tear down → reinstall → `rad startup` → **deploy an update to the same resource**. The update is the path that fails when Terraform state is lost. **This test depends on the separate Repo Radius workflow code (in flight) that creates the ephemeral cluster, installs Radius, and runs the deploy.** Because `rad startup` / `rad shutdown` are intentionally kind-agnostic and do not manage cluster lifecycle, the test needs that workflow to stand up the environment. Until it lands, the test drives the install/uninstall itself and is **gated behind the `RADIUS_STATE_E2E` environment variable**, so it does not run in the normal functional suite. Once the shared cluster-create + deploy workflow is merged, the test's install/uninstall helpers should be re-pointed at that code rather than duplicating it. ## Related - Feature spec: Repo Radius (#12078) - Relates to #8096 (External data store for Radius), #8398 (Postgres DB initialization) - Supersedes the unmerged prototype #11457 ## Type of change This pull request adds new features (state externalization commands) for Radius. --------- Signed-off-by: Sylvain Niles <sylvainniles@microsoft.com> Signed-off-by: sk593 <shruthikumar@microsoft.com>
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
| | --- | --- | | ||
| | `environment` | The GitHub Environment name | | ||
| | `ref` | The git commit or tag to deploy (defaults to the latest commit on the default branch). Not used in this user story but is used in User Story 3.1. | | ||
| | `rad_commands` | `['deploy .radius/app.bicep', 'app graph storefront -o json']` | |
|
|
||
| ## Appendix 4: Example Result Artifact | ||
|
|
||
| Below is an example run-rad-commands-result artifact from the prototype. This example shows the workflow being run with three commands: rad resource list -o json, rad app show todo-list -o json, and rad app graph todo-list -o json. Note that the second commands failed and the third did not execute. |
| "startedAt": "2026-06-29T18:42:05Z", | ||
| "completedAt": "2026-06-29T18:42:11Z", | ||
| "durationSeconds": 6, | ||
| "stdout": "[{\"id\":\"/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/frontend\",\"name\":\"frontend\",\"type\":\"Applications.Core/containers\"}]", |
| "index": 0, | ||
| "command": "resource list -o json", | ||
| "artifact": "rad-command-0", | ||
| "outcome": "succeeded", |
| "index": 1, | ||
| "command": "app show todo-list -o json", | ||
| "artifact": "rad-command-1", | ||
| "outcome": "failed", | ||
| "exitCode": 1, |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (2)
eng/design-notes/github/2026-06-repo-radius.md:488
- Grammar: “the second commands failed” should be singular (“the second command failed”).
Below is an example run-rad-commands-result artifact from the prototype. This example shows the workflow being run with three commands: rad resource list -o json, rad app show todo-list -o json, and rad app graph todo-list -o json. Note that the second commands failed and the third did not execute.
eng/design-notes/github/2026-06-repo-radius.md:512
- The example result artifact uses legacy
Applications.Core/containersresource types/IDs. This doc otherwise describes the modern Radius.Core/Radius.Compute model; updating the example avoids confusion for readers and keeps it consistent with current resource type names.
"stdout": "[{\"id\":\"/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/frontend\",\"name\":\"frontend\",\"type\":\"Applications.Core/containers\"}]",
|
|
||
| Repo Radius is a rethinking of how to deliver Radius to developers. It transforms Radius from a heavyweight persistent control plane into a lightweight on-demand engine. Concretely, Repo Radius enables developers to run Radius within a GitHub Actions runner, with state stored externally in the same GitHub account. | ||
|
|
||
| ### Top level goals |
|
|
||
| **Self-hosted Radius** – The existing deployment model in which Radius runs as a persistent control plane installed on a Kubernetes cluster. | ||
|
|
||
| **Frontend** – The separate user-facing component (such as the Copilot app) that drives Repo Radius by dispatching GitHub Action workflows and presenting their results. |
|
|
||
| The user experience is organized into five journeys. The first four follow the developer's lifecycle of an application: getting it running, iterating on it, operating it across environments, and evolving it over time. The fifth covers the platform engineer who tailors Repo Radius for a team. This fifth user journey is largely deferred beyond the initial release. Each journey is made up of user stories that describe a single thing the user wants to accomplish. The journeys and their stories are detailed in the sections that follow. | ||
|
|
||
| Repo Radius is the backend for these journeys, not a user interface. A separate frontend (such as the Copilot app) turns the user's intent into GitHub Action workflow runs and shows the results. The user stories describe what the user wants to accomplish and what Repo Radius does on each operation; what the user sees and clicks is defined by the frontend. |
|
|
||
| The developer works entirely through the frontend, which walks them through three steps: | ||
|
|
||
| 1. **Enable Repo Radius for the repository.** The frontend asks the developer for permission to enable Radius for the repository. With the developer's acknowledgement, it commits the required GitHub Action workflows to the repository so that GitHub Actions has a dispatchable entry point for each Radius operation. |
Signed-off-by: Zach Casper <zachcasper@microsoft.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (3)
eng/design-notes/github/2026-06-repo-radius.md:480
- The command examples are inconsistent with the
rad_commandsinput described earlier (which omits theradprefix), and there is a grammar issue (“second commands”). Consider listing subcommands consistently as they would appear in therad_commandsarray.
Below is an example run-rad-commands-result artifact from the prototype. This example shows the workflow being run with three commands: rad resource list -o json, rad app show todo-list -o json, and rad app graph todo-list -o json. Note that the second commands failed and the third did not execute.
eng/design-notes/github/2026-06-repo-radius.md:504
- The example artifact output uses legacy
Applications.Core/containersidentifiers. Elsewhere in the repo (e.g., built-in providers manifest) the container resource type isRadius.Compute/containers; updating the example avoids confusion about which type Repo Radius is expected to use going forward.
"stdout": "[{\"id\":\"/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/frontend\",\"name\":\"frontend\",\"type\":\"Applications.Core/containers\"}]",
eng/design-notes/github/2026-06-repo-radius.md:196
- This section says the app is modeled in
app.bicep, but therad_commandsexample usesdeploy .radius/app.bicep. Please make the spec consistent about the expected location/name of the app definition file, since that affects how the action checks out and runs deployments.
With the application already modeled in `app.bicep`, the developer asks the frontend to deploy it. Repo Radius runs on demand inside a GitHub Actions runner, provisions the resources defined in `app.bicep` and shuts down. The developer never installs or configures Radius or learns each provider's resource types and arguments.
This is the same standard deployment Radius performs today; Repo Radius does not change how Radius deploys applications. The only potential change to core Radius is adjusting the output of the `rad deploy` command so the frontend can collect incremental deployment progress updates.
#### User Experience
The developer asks the frontend to deploy an application:
1. **Request the deployment.** The developer asks for the application to be deployed, selecting the environment by name. For example, the developer requests the storefront application to be deployed to the `dev` environment.
2. **Preview what will be created.** Before anything is provisioned, the frontend shows the cloud resources that will be created and asks the developer for confirmation. For example, since the `dev` environment is configured to deploy to Azure, the frontend shows the developer:
| Application resource | Cloud resource |
| -------------------------- | ----------------------------------------------- |
| `frontend` container | Kubernetes service, deployment, service account |
| `api` container | Kubernetes service, deployment, service account |
| `orders-db` database | Azure Database for PostgreSQL Flexible Server |
| `cache` Redis cache | Azure Cache for Redis |
| `orders-queue` Kafka queue | Azure Event Hubs |
It is the frontend's responsibility to examine the application's resources and their recipes to produce this initial deployment plan; previewing changes to an already-deployed application is covered in User Story 2.2.
3. **Watch the deployment progress.** As the deployment runs, the frontend shows the status of each individual cloud resource. The developer sees the Azure Database for PostgreSQL Flexible Server move from pending to provisioning to succeeded (or failed), so progress is visible resource by resource rather than as a single opaque step.
4. **See each resource's identity.** When the deployment completes, the frontend shows the cloud provider resource ID for each provisioned resource. For the storefront's database, this is the Azure resource ID of the Flexible Server, for example `/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/storefront-dev/providers/Microsoft.DBforPostgreSQL/flexibleServers/storefront-orders-db`.
#### Result
The frontend deploys the application by dispatching the `radius-run-rad-commands.yml` workflow passing in the target environment name and the `rad deploy` command.
| Input | Value |
| --- | --- |
| `environment` | The GitHub Environment name |
| `ref` | The git commit or tag to deploy (defaults to the latest commit on the default branch). Not used in this user story but is used in User Story 3.1. |
| `rad_commands` | `['deploy .radius/app.bicep', 'app graph storefront -o json']` |
This pull request introduces a comprehensive feature specification document for "Repo Radius," detailing a new approach to delivering Radius as a lightweight, on-demand engine that runs within GitHub Actions, removing the need for a pre-configured Kubernetes cluster. The document outlines goals, user personas, scenarios, dependencies, risks, and required investments for implementing Repo Radius, making Radius more accessible to developers without platform engineering resources.
Repo Radius Feature Specification:
Overview and Motivation:
Key Scenarios and User Experience:
Technical Implementation and Dependencies:
Type of change
Contributor checklist
Please verify that the PR meets the following requirements, where applicable:
eng/design-notes/in this repository, if new APIs are being introduced.