From 5ae7037c550fa64617c7578c4dcd85e41fe89cd9 Mon Sep 17 00:00:00 2001 From: John Garbutt Date: Tue, 19 Mar 2024 18:27:04 +0000 Subject: [PATCH 1/3] Add in start of Azimuth architecture docs --- docs/architecture.md | 156 ++++++++++++++++++++++++++++++++++++++++++ docs/best-practice.md | 11 +-- docs/index.md | 58 ++++++---------- mkdocs.yml | 1 + 4 files changed, 185 insertions(+), 41 deletions(-) create mode 100644 docs/architecture.md diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 00000000..684f41e4 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,156 @@ +# Azimuth Architecture + +These docs look at the Azimuth Architecture from a deployer point of view. + +For more details on Azimuth Architecture from a developer point of view, please see the +[Azimuth Architecture Developer Docs](https://github.com/stackhpc/azimuth/blob/master/docs/architecture.md). + +## Azimuth Components + +Lets look at the key parts of an Azimuth deployment, +working through them as they are setup by the +Ansible automation. + +### Config git repository + +The desired state is recorded in git. +Updating involves merging in the latest changes from the +[azimuth-config reference configuration](https://github.com/stackhpc/azimuth-config). + +Good practice is to use a single git repository +for all your [Azimuth environments](./environments.md), +e.g. for both staging and production. +Similarly, `git-crypt` is recommended +for [encrypting secrets](./repository/secrets.md). + +### Control host + +This is where Ansible is run. + +Ideally Ansible is run from within an ephemeral runner +created by [GitHub or GitLab automation](./deployment/automation.md). + +### Ansible collection + +TODO + +### Deployment OpenStack Project + +Azimuth is typically run within the OpenStack cloud it is targeting. + +To isolate your Azimuth deployment from other workloads, +it is good practice to run Azimuth within a separate OpenStack project +dedicated to running Azimuth. +Moreover, production and staging deployments are typically +given their own separate OpenStack project. + +### Seed VM with K3S + +This VM is created using OpenTofu, within the deployment OpenStack project +used to deploy Azimuth. +We run [K3S](https://k3s.io/) in this VM. + +For single node deployments, all services run here. + +For HA deployments, K3S is used to run a +[Cluster API management cluster](https://cluster-api.sigs.k8s.io/user/concepts#management-cluster). +We then use the +[CAPI helm charts](https://github.com/stackhpc/capi-helm-charts) +to create a highly available kubernetes cluster. + +!!! warning + + HA deployments require that [Octavia](https://docs.openstack.org/octavia/latest/index.html) + is available on the target cloud to provide load-balancers for Azimuth components. + + For single node deployment, the seed VM is given a floating IP. + +### Seed VM OpenTofu state + +An [OpenTofu remote state store](./repository/terraform.md) must be configured +in order to persist the OpenTofu state used to create the Seed VM. + +### Azimuth Management Kubernetes Cluster + +For ha clusters, we run all Azimuth services in the HA K8s cluster. +For the single node deployment, we run everything in K3s on the seed VM. +For simplicity we call this the Azimuth Management Cluster. + +For more details about the architecture of the Azimuth services, please see the +[Azimuth Architecture Developer Docs](https://github.com/stackhpc/azimuth/blob/master/docs/architecture.md). + + +### CaaS images, templates and workloads + +TODO + +### Kubernetes images, templates and workloads + +TODO + +### Kubernetes application templates + +TODO + +### Zenith + +TODO + +### Azimuth Ingress and SSL + +Most Azimuth traffic goes through a single wildcard DNS entry. +This includes: ... + +TODO + +### Azimuth Monitoring and Alerting + +TODO + +### Workload OpenStack Projects + +When a user logs into Azimuth, they select a tenancy. +This maps to a specific OpenStack Project. + +### Azimuth CRDs + +Azimuth doesn't have a database, as such. + +TODO + + +### Backup and Disaster recovery + +TODO + +## Authentication and Authorization + +Azimuth currently delegates all authentication and authorization to OpenStack. + +In many ways, you can consider the Azimuth to be +a sophisticated OpenStack client, +similar to Horizon and the OpenStack CLI. + +Any user with access to an OpenStack project, +will see that project listed as a tenancy within Azimuth. +In addition, people with access to that OpenStack project +get full access to all platforms created within that tenancy. + +We make an assumption that all users with access to the OpenStack +API can elevate themselves to have "root access" within any VM +in that OpenStack project. This is a feature of OpenStack's APIs. + +There are plans for "managed" tenancies within Azimuth, +where users do not get "root access". +This will depend on OpenStack efforts around adding the +project-reader role into 2024.1 release. +For more details, please see +[OpenStack consistent RBAC](https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html). + +### Upstairs and Downstairs + +TODO + +### Keycloak + +TODO \ No newline at end of file diff --git a/docs/best-practice.md b/docs/best-practice.md index 0ab30e2e..5223ff47 100644 --- a/docs/best-practice.md +++ b/docs/best-practice.md @@ -1,12 +1,15 @@ -# Best practice for deployments +# Getting started with production deployment This document guides you through the process of setting up a production-ready Azimuth deployment following recommended best practice. -## Prerequisites +To quickly try out Azimuth on your OpenStack cloud, you can follow [these instructions](./try.md) +to get a simple deployment running within a single VM in your OpenStack cloud. -Before proceeding with an Azimuth deployment, you should ensure that the target cloud -meets the [prerequisites](./configuration/01-prerequisites.md). +## 1. Prerequisites + +Before proceeding with an Azimuth deployment, you should ensure that the target +OpenStack cloud meets the [prerequisites](./configuration/01-prerequisites.md). ## OpenStack projects diff --git a/docs/index.md b/docs/index.md index 5460a453..4fa3fbf2 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,7 +1,18 @@ # Azimuth Operator Documentation This documentation describes how to manage deployments of -[Azimuth](https://github.com/stackhpc/azimuth), including all the required dependencies. +[Azimuth](https://github.com/stackhpc/azimuth), +including all the required dependencies. + +## Try Azimuth on your OpenStack cloud + +If you have access to a project on an OpenStack cloud, you can try Azimuth! +You don't need admin level access. + +To try out Azimuth on your OpenStack cloud, you can follow [these instructions](./try.md) +to get a simple deployment running within a single VM in your OpenStack cloud. + +## Getting started with a Production Azimuth Azimuth is deployed using [Ansible](https://www.ansible.com/) with playbooks from the [azimuth-ops Ansible collection](https://github.com/stackhpc/ansible-collection-azimuth-ops), @@ -12,43 +23,16 @@ The `azimuth-config` repository is designed to be forked for a specific site and into multiple [environments](#environments). This structure allows common configuration to be shared but overridden where required using composition of environments. -## Structure of an Azimuth deployment - -A fully-featured Azimuth deployment consists of many components, such as -[Zenith](https://github.com/stackhpc/zenith), [Cluster API](https://cluster-api.sigs.k8s.io/) -and the [CaaS operator](https://github.com/stackhpc/azimuth-caas-operator), which -require a [Kubernetes](https://kubernetes.io/) cluster to run. - -However when you consider an Azimuth deployment as a whole, the only _real_ dependency is -an [OpenStack](https://www.openstack.org/) cloud to target - we can create a Kubernetes -cluster within an OpenStack project on the target cloud to host our Azimuth deployment. -This is exactly what the playbooks in the `azimuth-ops` collection will do, when driven by -a configuration derived from `azimuth-config`. - -There are two methods that `azimuth-ops` can use to deploy Azimuth and all of its -dependencies: - - 1. Onto a managed single-node [K3S](https://k3s.io/) cluster in an OpenStack project. - 2. Onto a managed highly-available Kubernetes cluster in an OpenStack project. - -Option 1 is useful for development or demo deployments, but is not suitable for a production -deployment. - -Option 2 is the recommended deployment mechanism for most deployments. In this mode, -[Terraform](https://www.terraform.io/) is used to provision a single-node K3S cluster -that is configured as a [Cluster API](https://cluster-api.sigs.k8s.io/) management -cluster. Cluster API is then used to provision a highly-available Kubernetes cluster in -the same OpenStack project onto which Azimuth is deployed. - -!!! warning +For a production-ready deployment, you should follow the steps in the +[getting started documentation](./best-practice.md). - Option 2 requires that [Octavia](https://docs.openstack.org/octavia/latest/index.html) - is available on the target cloud to provide load-balancers for Azimuth components. +## Understating Azimuth Architecture -## Deploying Azimuth +If you want to know more about how Azimuth is architected, +and better understand its security model, +please read +[Azimuth Architecture](./architecture.md) -To try out Azimuth on your OpenStack cloud, you can follow [these instructions](./try.md) -to get a simple single-node deployment. +## Developing Azimuth -For a production-ready deployment, you should follow the steps in the -[best practice document](./best-practice.md). +For a developer setup, please see [developing azimuth](./developing/index.md). \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index d00e23ff..2f01ae48 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -7,6 +7,7 @@ nav: - Home: index.md - try.md - best-practice.md + - architecture.md - environments.md - Repository: - repository/index.md From a3f2c07913e0d8de886a09a79c587f0d4124a9ce Mon Sep 17 00:00:00 2001 From: John Garbutt Date: Tue, 19 Mar 2024 18:51:21 +0000 Subject: [PATCH 2/3] Updates to getting starting guide --- docs/best-practice.md | 82 ++++++++++++++++++++++++++++++------------- 1 file changed, 58 insertions(+), 24 deletions(-) diff --git a/docs/best-practice.md b/docs/best-practice.md index b404626f..c7cf8f23 100644 --- a/docs/best-practice.md +++ b/docs/best-practice.md @@ -6,12 +6,13 @@ deployment following recommended best practice. To quickly try out Azimuth on your OpenStack cloud, you can follow [these instructions](./try.md) to get a simple deployment running within a single VM in your OpenStack cloud. -## 1. Prerequisites +## 1. Check Prerequisites -Before proceeding with an Azimuth deployment, you should ensure that the target -OpenStack cloud meets the [prerequisites](./configuration/01-prerequisites.md). +Before proceeding with an Azimuth deployment, +ensure you meet the required +[prerequisites](./configuration/01-prerequisites.md). -## OpenStack projects +### Create OpenStack projects Azimuth is usually deployed on the cloud that is being targeted for workloads. It is recommended to have three OpenStack projects for a production Azimuth deployment, to contain: @@ -25,7 +26,7 @@ The production and staging projects must have deployment. The required quota in the CI/CD project will depend on the number of proposed changes that are open concurrently. -## Repository +## 2. Create your Azimuth Configuration Git Repository Before building your Azimuth configuration, you must first [set up your configuration repository](./repository/index.md), including initialising @@ -35,20 +36,9 @@ It is recommended to use a [feature branch workflow](./repository/index.md#making-changes-to-your-configuration) to make changes to your Azimuth configuration in a controlled way. -## OpenTofu state +## 3. Define your Environments -Azimuth deployments use [OpenTofu](https://opentofu.org/), an open-source fork of -[Terraform](https://www.terraform.io/), to manage some parts of the infrastructure. - -A [remote state store](./repository/opentofu.md#remote-state) must be configured in -order to persist the OpenTofu state across playbook executions. If GitLab is being -used for the Azimuth configuration repository, it is recommended to use -[GitLab-managed Terraform state](./repository/opentofu.md#gitlab) for this. If not, -[S3](./repository/opentofu.md#s3) is the preferred approach. - -## Environments - -An Azimuth configuration repository contains multiple [environments](./environments.md), +Your Azimuth configuration git repository contains multiple [environments](./environments.md), some of which contain common configuration ("mixin" environments), and some of which represent a deployment ("concrete" environments). @@ -86,7 +76,49 @@ base --> ha --> site --> site-ha --> production with only necessary differences configured in each environment, e.g. the ingress base domain, between `staging` and `production`. -## Continuous delivery +## 4. Populate required group_vars + +Before looking to customize your local deployment, +we recommend you first get a minimal configuration +up and running. + +Above you created the following environments: + +* site +* site-ha +* aio, staging and production + +You now need to populate the required group_vars +such aio, staging and production environments can be deployed. + +When building an environment for the first time, +it is recommended to follow each configuration documentation page in order, +beginning with the +[Deployment method](./configuration/02-deployment-method.md). +A quick overview of the minimal required parameters can be seen in the +(example environment)[https://github.com/stackhpc/azimuth-config/tree/devel/environments/example/inventory/group_vars/all]. + +## 5. Review Production Good Practice + +### 5.1. OpenTofu state + +Azimuth deployments use [OpenTofu](https://opentofu.org/), an open-source fork of +[Terraform](https://www.terraform.io/), to manage some parts of the infrastructure. + +A [remote state store](./repository/opentofu.md#remote-state) must be configured in +order to persist the OpenTofu state across playbook executions. If GitLab is being +used for the Azimuth configuration repository, it is recommended to use +[GitLab-managed Terraform state](./repository/opentofu.md#gitlab) for this. If not, +[S3](./repository/opentofu.md#s3) is the preferred approach. + +### 5.2. Setup Monitoring Alerts + +Ensure you can access your monitoring and alerts are flowing somewhere +you will notice them. + +TODO... + +### 5.3. Setup Continuous delivery A production Azimuth deployment should use [continuous delivery](./deployment/automation.md), where changes to the configuration are automatically deployed to the `aio` and `staging` @@ -106,7 +138,7 @@ A [sample GitLab CI/CD configuration](https://github.com/stackhpc/azimuth-config/tree/stable/.gitlab-ci.yml.sample) is provided that implements this workflow for GitLab-hosted repositories. -## Disaster recovery +### 5.4. Backups and Disaster recovery Azimuth uses [Velero](https://velero.io/) to backup the data that is required to restore an Azimuth instance in the event of a catastrophic failure. This functionality is not enabled by @@ -115,8 +147,10 @@ default, as it requires credentials for an S3 bucket in which the backups will b It is recommended that [disaster recovery is enabled](./configuration/15-disaster-recovery.md) for a production deployment. -## Configuration +### 5.5. Do you first upgrade + +TODO + +## 6. Join the community! -You are now ready to begin adding configuration to your environments. When building an environment -for the first time, it is recommended to follow each documentation page in order, beginning with -the [Deployment method](./configuration/02-deployment-method.md). +TODO From 151b7554035b878eac9715c15b6aa79014d24238 Mon Sep 17 00:00:00 2001 From: sd109 Date: Mon, 29 Apr 2024 16:28:56 +0100 Subject: [PATCH 3/3] Add more documentation content --- docs/architecture.md | 55 ++++++++++++++++++++++---------------------- 1 file changed, 28 insertions(+), 27 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 684f41e4..ed8b3458 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,15 +1,14 @@ # Azimuth Architecture -These docs look at the Azimuth Architecture from a deployer point of view. +These docs look at the Azimuth Architecture from an operator's point of view. For more details on Azimuth Architecture from a developer point of view, please see the [Azimuth Architecture Developer Docs](https://github.com/stackhpc/azimuth/blob/master/docs/architecture.md). ## Azimuth Components -Lets look at the key parts of an Azimuth deployment, -working through them as they are setup by the -Ansible automation. +The following sections highlight the key parts of an Azimuth deployment, +working through them in the order that they are setup by the Ansible automation. ### Config git repository @@ -17,46 +16,48 @@ The desired state is recorded in git. Updating involves merging in the latest changes from the [azimuth-config reference configuration](https://github.com/stackhpc/azimuth-config). -Good practice is to use a single git repository +It is recommmended to use a single git repository for all your [Azimuth environments](./environments.md), e.g. for both staging and production. -Similarly, `git-crypt` is recommended +Similarly, `git-crypt` should be used for [encrypting secrets](./repository/secrets.md). ### Control host -This is where Ansible is run. - -Ideally Ansible is run from within an ephemeral runner -created by [GitHub or GitLab automation](./deployment/automation.md). +This is where Ansible is run. Ideally, this should be from within an ephemeral runner +created by [GitHub or GitLab automation](./deployment/automation.md). The control host must +be able to reach the OpenStack public API endpoints for the target cloud. ### Ansible collection -TODO +The [azimuth-ops](https://github.com/stackhpc/ansible-collection-azimuth-ops/) Ansible collection +contains all of the roles and playbooks required to deploy and update Azimuth. The +[provision](https://github.com/stackhpc/ansible-collection-azimuth-ops/blob/main/playbooks/provision.yml) +playbook provides the main entry point into the Ansible collection. ### Deployment OpenStack Project -Azimuth is typically run within the OpenStack cloud it is targeting. +Azimuth management infrastructure is typically run within the OpenStack cloud it is targeting. -To isolate your Azimuth deployment from other workloads, -it is good practice to run Azimuth within a separate OpenStack project -dedicated to running Azimuth. +To isolate your Azimuth manangement infrastructure from other workloads, +it is good practice to run Azimuth within a separate dedicated OpenStack project. Moreover, production and staging deployments are typically given their own separate OpenStack project. ### Seed VM with K3S -This VM is created using OpenTofu, within the deployment OpenStack project -used to deploy Azimuth. -We run [K3S](https://k3s.io/) in this VM. +This VM is created in the deployment OpenStack project using OpenTofu. +A light-weight [K3S](https://k3s.io/) cluster is run on the seed node, with all +K3S data stored in a dedicated Cinder volume mounted at `/var/lib/rancher/`, +allowing the seed VM to be upgraded or recreated without data loss (as long as the +Cinder volume is preserved). -For single node deployments, all services run here. +For single node deployments, all Azimuth components run on the seed's K3S cluster. -For HA deployments, K3S is used to run a +For highly-available (HA) deployments, K3S is instead used to run a [Cluster API management cluster](https://cluster-api.sigs.k8s.io/user/concepts#management-cluster). -We then use the -[CAPI helm charts](https://github.com/stackhpc/capi-helm-charts) -to create a highly available kubernetes cluster. +which in turn uses [CAPI Helm charts](https://github.com/stackhpc/capi-helm-charts) +to create a highly-available Kubernetes cluster for hosting the Azimuth management components. !!! warning @@ -68,18 +69,18 @@ to create a highly available kubernetes cluster. ### Seed VM OpenTofu state An [OpenTofu remote state store](./repository/terraform.md) must be configured -in order to persist the OpenTofu state used to create the Seed VM. +in order to persist the OpenTofu state used to create the Seed VM. This state store is accessed, +for example, when [accessing the seed VM](https://stackhpc.github.io/azimuth-config/debugging/access-k3s/). ### Azimuth Management Kubernetes Cluster -For ha clusters, we run all Azimuth services in the HA K8s cluster. +For HA deployments, we run all Azimuth services in the HA K8s cluster. For the single node deployment, we run everything in K3s on the seed VM. For simplicity we call this the Azimuth Management Cluster. For more details about the architecture of the Azimuth services, please see the [Azimuth Architecture Developer Docs](https://github.com/stackhpc/azimuth/blob/master/docs/architecture.md). - ### CaaS images, templates and workloads TODO @@ -153,4 +154,4 @@ TODO ### Keycloak -TODO \ No newline at end of file +TODO