Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/commands/pr.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
description: Format code, stage changes, and create a pull request
allowed-tools:
- Bash
argument-hint: [base-branch]
argument-hint: "[base-branch]"
---

# Prepare and create pull request
Expand Down
11 changes: 8 additions & 3 deletions docs/book/component-guide/orchestrators/kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ description: Orchestrating your pipelines to run on Kubernetes clusters.

Using the ZenML `kubernetes` integration, you can orchestrate and scale your ML pipelines on a [Kubernetes](https://kubernetes.io/) cluster without writing a single line of Kubernetes code.

The orchestrator uses standard Kubernetes APIs and works on managed cloud clusters (EKS, GKE, AKS), enterprise platforms like Red Hat OpenShift and SUSE Rancher, and self-managed clusters on-premises or in your own cloud.

This Kubernetes-native orchestrator is a minimalist, lightweight alternative to other distributed orchestrators like Airflow or Kubeflow.

Overall, the Kubernetes orchestrator is quite similar to the Kubeflow orchestrator in that it runs each pipeline step in a separate Kubernetes pod. However, the orchestration of the different pods is not done by Kubeflow but by a separate master pod that orchestrates the step execution via topological sort.
Expand All @@ -20,15 +22,18 @@ This component is only meant to be used within the context of a [remote ZenML de

You should use the Kubernetes orchestrator if:

* you already have a Kubernetes cluster—managed cloud, Red Hat OpenShift, SUSE Rancher, or self-managed on-premises.
* you're looking for a lightweight way of running your pipelines on Kubernetes.
* you're not willing to maintain [Kubeflow Pipelines](kubeflow.md) on your Kubernetes cluster.
* you're not interested in paying for managed solutions like [Vertex](vertex.md).

## How to deploy it

The Kubernetes orchestrator requires a Kubernetes cluster in order to run. There are many ways to deploy a Kubernetes cluster using different cloud providers or on your custom infrastructure, and we can't possibly cover all of them, but you can check out our [our production guide](https://docs.zenml.io/user-guides/production-guide).
The Kubernetes orchestrator requires a Kubernetes cluster (version 1.21 or higher recommended).

There are many ways to deploy a Kubernetes cluster using different cloud providers or on your custom infrastructure, and we can't possibly cover all of them, but you can check out our [production guide](https://docs.zenml.io/user-guides/production-guide).

If the above Kubernetes cluster is deployed remotely on the cloud, then another pre-requisite to use this orchestrator would be to deploy and connect to a [remote ZenML server](https://docs.zenml.io/getting-started/deploying-zenml/).
If the Kubernetes cluster is deployed remotely, another prerequisite is to deploy and connect to a [remote ZenML server](https://docs.zenml.io/getting-started/deploying-zenml/).

## How to use it

Expand All @@ -42,7 +47,7 @@ To use the Kubernetes orchestrator, we need:
* [Docker](https://www.docker.com) installed and running.
* A [remote artifact store](../artifact-stores/README.md) as part of your stack.
* A [remote container registry](../container-registries/README.md) as part of your stack.
* A Kubernetes cluster [deployed](kubernetes.md#how-to-deploy-it)
* A Kubernetes cluster [deployed](kubernetes.md#how-to-deploy-it) (version 1.21 or higher recommended)
* [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) installed and the name of the Kubernetes configuration context which points to the target cluster (i.e. run`kubectl config get-contexts` to see a list of available contexts) . This is optional (see below).

{% hint style="info" %}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,15 @@ description: Deploying ZenML in a Kubernetes cluster with Helm.

If you wish to manually deploy and manage ZenML in a Kubernetes cluster of your choice, ZenML also includes a Helm chart among its available deployment options.

The chart uses standard Kubernetes APIs and works on managed cloud clusters (EKS, GKE, AKS), enterprise platforms like Red Hat OpenShift and SUSE Rancher, and self-managed clusters on-premises or in your own cloud.

You can find the chart on this [ArtifactHub repository](https://artifacthub.io/packages/helm/zenml/zenml), along with the templates, default values and instructions on how to install it. Read on to find detailed explanations on prerequisites, configuration, and deployment scenarios.

## Prerequisites

You'll need the following:

* A Kubernetes cluster
* A Kubernetes cluster (version 1.21 or higher recommended)
* Optional, but recommended: a MySQL-compatible database reachable from the Kubernetes cluster (e.g. one of the managed databases offered by Google Cloud, AWS, or Azure). A MySQL server version of 8.0 or higher is required
* the [Kubernetes client](https://kubernetes.io/docs/tasks/tools/#kubectl) already installed on your machine and configured to access your cluster
* [Helm](https://helm.sh/docs/intro/install/) installed on your machine
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ layout:

This guide provides step-by-step instructions for deploying ZenML Pro on Kubernetes using Helm charts. All components run within your infrastructure with zero external dependencies.

ZenML Pro deployments use standard Kubernetes APIs and work on managed cloud clusters (EKS, GKE, AKS), enterprise platforms like Red Hat OpenShift and SUSE Rancher, and self-managed clusters on-premises or in your own cloud.

## Deployment Architecture

All components run entirely within your Kubernetes cluster and infrastructure:
Expand Down
4 changes: 3 additions & 1 deletion docs/book/getting-started/zenml-pro/deploy-prerequisites.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,12 +238,14 @@ The Control Plane and Workspace Server components are completely independent fro

| Infrastructure Type | Support Level |
|---------------------|---------------|
| Kubernetes (managed or self-hosted, 1.24+) | ✅ Officially supported and documented |
| Kubernetes (1.24+) | ✅ Officially supported and documented |
| Container-as-a-Service (ECS, Cloud Run, etc.) | ⚠️ Possible but not documented |
| Serverless containers (Fargate, Lambda, etc.) | ⚠️ Possible but not documented |
| Virtual machines with Docker (VMware, VirtualBox, etc.) | ⚠️ Possible but not documented |
| Docker-compose | ✅ Can be provided upon request for local testing purposes |

These Helm-based deployments use standard Kubernetes APIs and work on managed cloud clusters (EKS, GKE, AKS), enterprise platforms like Red Hat OpenShift and SUSE Rancher, and self-managed clusters on-premises or in your own cloud.

{% hint style="warning" %}
While ZenML Pro can theoretically run on any container orchestration platform, only Kubernetes deployments are officially fully supported and documented. If you require a different deployment target, please contact [cloud@zenml.io](mailto:cloud@zenml.io) to discuss your requirements.
{% endhint %}
Expand Down
2 changes: 2 additions & 0 deletions docs/book/getting-started/zenml-pro/deploy-workspace-k8s.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ layout:

This guide provides step-by-step instructions for deploying ZenML Pro workspaces on Kubernetes using Helm and enrolling them in the ZenML Pro control plane.

ZenML Pro workspace deployments use standard Kubernetes APIs and work on managed cloud clusters (EKS, GKE, AKS), enterprise platforms like Red Hat OpenShift and SUSE Rancher, and self-managed clusters on-premises or in your own cloud.


## Prerequisites

Expand Down
2 changes: 1 addition & 1 deletion docs/book/reference/legacy-docs.md

Large diffs are not rendered by default.