Add vllm-kubernetes model deployer flavor

docs/book/component-guide/model-deployers/README.md

@@ -63,6 +63,7 @@ integrations:
 | [Hugging Face](huggingface.md) | `huggingface` | `huggingface` | Deploys ML model on Hugging Face Inference Endpoints |
 | [Databricks](databricks.md) | `databricks` | `databricks` | Deploying models to Databricks Inference Endpoints with Databricks |
 | [vLLM](vllm.md)                | `vllm`  | `vllm`      | Deploys LLM using vLLM locally |
+| [vLLM](vllm.md)                | `vllm-kubernetes` | `vllm`      | Deploys LLM using vLLM on Kubernetes |
 | [Custom Implementation](custom.md) | _custom_  |               | Extend the Artifact Store abstraction and provide your own implementation    |
 
 {% hint style="info" %}

docs/book/component-guide/model-deployers/vllm.md

@@ -73,4 +73,133 @@ Within the `VLLMDeploymentService` you can configure:
 * `dtype`: Data type for model weights and activations. Allowed choices: ['auto', 'half', 'float16', 'bfloat16', 'float', 'float32']
 * `revision`: The specific model version to use. It can be a branch name, a tag name, or a commit id. If unspecified, will use the default version.
 
+## Deploying on Kubernetes
+
+The `vllm-kubernetes` flavor deploys the vLLM OpenAI-compatible server as a long-lived Deployment and Service on a Kubernetes cluster, instead of running it as a local daemon process. It builds on the same [model deployer abstraction](README.md) as [Seldon Core](seldon.md), with the Kubernetes cluster reached through a `kubectl` context, an in-cluster configuration, or a [Service Connector](https://docs.zenml.io/how-to/infrastructure-deployment/auth-management/).
+
+Only `deploy` and `delete` are supported for this flavor. Stopping and starting a deployed server raise `NotImplementedError`, delete it and deploy again instead.
+
+### Installation
+
+The Kubernetes flavor talks to the cluster through the `kubernetes` package and to the deployed server through the `openai` client. It does not need the `vllm` package itself. If you only need the Kubernetes flavor, you can skip the `vllm` install:
+
+```bash
+pip install kubernetes openai
+```
+
+`zenml integration install vllm -y` also works and additionally installs `vllm`, which is only needed for the local flavor.
+
+### Registering the model deployer
+
+```bash
+zenml model-deployer register <MODEL_DEPLOYER_NAME> --flavor=vllm-kubernetes \
+  --kubernetes_namespace=<KUBERNETES-NAMESPACE> \
+  --hf_token=<HF_TOKEN>
+```
+
+You can configure:
+
+* `kubernetes_context`: name of a `kubectl` context to deploy to. Ignored if the component is linked to a Kubernetes service connector.
+* `kubernetes_namespace`: namespace vLLM deployments are provisioned into, defaults to `zenml-vllm`. Can be overridden per deployment.
+* `incluster`: if `True`, connects using the configuration of the cluster the client itself runs in. Requires the client to run inside a Kubernetes pod, and ignores `kubernetes_context` when set.
+* `default_image`: image used when a deployment doesn't specify its own, defaults to `vllm/vllm-openai:v0.25.1`.
+* `default_service_type`: Kubernetes Service type used when a deployment doesn't specify its own, defaults to `LoadBalancer`.
+* `hf_token`: Hugging Face access token used to download gated models, used as a fallback for deployments that configure neither their own token nor an existing secret.
+
+### Using a service connector
+
+As with Seldon Core, the recommended way to authenticate to the target cluster is [a Service Connector](https://docs.zenml.io/how-to/infrastructure-deployment/auth-management/) instead of `kubernetes_context`. Pick the connector for your cloud provider (AWS, GCP, Azure) or the generic [Kubernetes Service Connector](https://docs.zenml.io/how-to/infrastructure-deployment/auth-management/kubernetes-service-connector), then connect it to the model deployer:
+
+```sh
+zenml service-connector register <CONNECTOR_NAME> --type kubernetes --auto-configure
+
+zenml model-deployer register <MODEL_DEPLOYER_NAME> --flavor=vllm-kubernetes
+zenml model-deployer connect <MODEL_DEPLOYER_NAME> --connector <CONNECTOR_NAME>
+```
+
+We can now use the model deployer in our stack:
+
+```bash
+zenml stack update <CUSTOM_STACK_NAME> --model-deployer=<MODEL_DEPLOYER_NAME>
+```
+
+### Deploy an LLM
+
+The deploy entry point is the same `deploy_model` method used by every model deployer. Here is an example step that deploys a gated Hugging Face model with a GPU:
+
+```python
+from typing import Annotated
+from zenml import pipeline, step
+from zenml.integrations.vllm.model_deployers import KubernetesVLLMModelDeployer
+from zenml.integrations.vllm.services import (
+    VLLMKubernetesDeploymentService,
+    VLLMKubernetesServiceConfig,
+)
+
+
+@step
+def deploy_vllm_kubernetes(
+    model: str,
+    timeout: int = 1200,
+) -> Annotated[VLLMKubernetesDeploymentService, "vllm_deployment"]:
+    model_deployer = KubernetesVLLMModelDeployer.get_active_model_deployer()
+
+    service_config = VLLMKubernetesServiceConfig(
+        model=model,
+        resources={"limits": {"nvidia.com/gpu": "1"}},
+        hf_token="<HF_TOKEN>",
+    )
+
+    service = model_deployer.deploy_model(
+        config=service_config,
+        service_type=VLLMKubernetesDeploymentService.SERVICE_TYPE,
+        timeout=timeout,
+    )
+    return service
+
+
+@pipeline
+def deploy_vllm_kubernetes_pipeline(model: str, timeout: int = 1200):
+    deploy_vllm_kubernetes(model=model, timeout=timeout)
+```
+
+`VLLMKubernetesServiceConfig` accepts the same engine arguments as the local `VLLMDeploymentService` (`model`, `tokenizer`, `served_model_name`, `trust_remote_code`, `tokenizer_mode`, `dtype`, `revision`), plus:
+
+* `namespace`: Kubernetes namespace to deploy into. Falls back to the model deployer's `kubernetes_namespace` when unset.
+* `image`: container image running the vLLM OpenAI server. Falls back to the model deployer's `default_image` when unset.
+* `port`: port the vLLM OpenAI server listens on, defaults to `8000`.
+* `service_type`: Kubernetes Service type used to expose the server. Falls back to the model deployer's `default_service_type` when unset.
+* `replicas`: number of pod replicas for the Deployment, defaults to `1`.
+* `resources`: native Kubernetes resource requests and limits for the container, e.g. `{"limits": {"nvidia.com/gpu": "1"}}`. A GPU limit is automatically mirrored into requests, since Kubernetes requires the two to match for extended resources.
+* `pod_settings`: a `KubernetesPodSettings` object for node selectors, tolerations, affinity, and other pod configuration not modeled as a dedicated field.
+* `hf_token`: Hugging Face access token injected into the container through a managed secret.
+* `existing_hf_secret`: name of an existing Kubernetes secret to use instead of `hf_token`. Must contain the token under a `token` key.
+* `env`: additional environment variables set on the container.
+* `extra_serve_args`: additional CLI args appended to the vLLM OpenAI server args, for engine options not modeled as dedicated fields.
+* `shm_size`: size limit of the `/dev/shm` volume mounted into the container, defaults to `2Gi`.
+
+### Reaching the deployed server
+
+A deployment that doesn't set its own `service_type` is exposed through the model deployer's `default_service_type`, which is `LoadBalancer` unless the deployer was registered with a different one, for example `--default_service_type=ClusterIP`.
+
+With a `LoadBalancer` Service, point an OpenAI client at the external address assigned to the Service. With a `ClusterIP` Service, which only has an address inside the cluster, forward the Service port instead to reach it from your local machine:
+
+```bash
+kubectl port-forward -n <KUBERNETES-NAMESPACE> svc/<SERVICE_NAME> 8000:8000
+```
+
+and then point an OpenAI client at `http://localhost:8000/v1`.
+
+### Custom images
+
+`image` (per deployment) and `default_image` (on the model deployer) both accept any vLLM OpenAI-compatible image, and override the pinned default `vllm/vllm-openai:v0.25.1`. For clusters without a GPU, use a CPU-capable image such as `vllm/vllm-openai-cpu`.
+
+### Teardown
+
+```bash
+zenml model-deployer models delete <SERVICE_UUID>
+```
+
+This deletes the Deployment, Service, and the managed secret if one was created for `hf_token`. The namespace itself is never deleted.
+
 <figure><img src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" alt="ZenML Scarf"><figcaption></figcaption></figure>

src/zenml/integrations/kubernetes/manifest_utils.py

@@ -225,7 +225,8 @@ def add_pod_settings(
 
     for container in pod_spec.containers:
         assert isinstance(container, k8s_client.V1Container)
-        container._resources = settings.resources
+        if settings.resources:
+            container._resources = settings.resources
         if settings.volume_mounts:
             if container.volume_mounts:
                 container.volume_mounts.extend(settings.volume_mounts)

src/zenml/integrations/vllm/__init__.py

@@ -19,6 +19,7 @@
 from zenml.integrations.constants import VLLM
 
 VLLM_MODEL_DEPLOYER = "vllm"
+VLLM_KUBERNETES_MODEL_DEPLOYER_FLAVOR = "vllm-kubernetes"
 
 logger = get_logger(__name__)
 
@@ -28,22 +29,40 @@ class VLLMIntegration(Integration):
 
     NAME = VLLM
 
-    REQUIREMENTS = ["vllm>=0.6.0,<0.7.0", "openai>=1.0.0"]
+    REQUIREMENTS = [
+        "vllm>=0.6.0,<0.7.0",
+        "openai>=1.0.0",
+        "kubernetes>=21.7,<26",
+    ]
 
     @classmethod
     def activate(cls) -> None:
         """Activates the integration."""
         from zenml.integrations.vllm import services
 
+        try:
+            from zenml.integrations.vllm.services import (  # noqa: F401
+                vllm_kubernetes_deployment,
+            )
+        except ImportError:
+            logger.debug(
+                "Could not import the vLLM Kubernetes deployment service. "
+                "The `vllm-kubernetes` model deployer flavor will not be "
+                "usable until the `kubernetes` package is installed."
+            )
+
     @classmethod
     def flavors(cls) -> List[Type[Flavor]]:
         """Declare the stack component flavors for the vLLM integration.
 
         Returns:
             List of stack component flavors for this integration.
         """
-        from zenml.integrations.vllm.flavors import VLLMModelDeployerFlavor
+        from zenml.integrations.vllm.flavors import (
+            KubernetesVLLMModelDeployerFlavor,
+            VLLMModelDeployerFlavor,
+        )
 
-        return [VLLMModelDeployerFlavor]
+        return [VLLMModelDeployerFlavor, KubernetesVLLMModelDeployerFlavor]
 
 

src/zenml/integrations/vllm/flavors/__init__.py

@@ -13,9 +13,18 @@
 #  permissions and limitations under the License.
 """vLLM integration flavors."""
 
+from zenml.integrations.vllm.flavors.vllm_kubernetes_model_deployer_flavor import (  # noqa
+    KubernetesVLLMModelDeployerConfig,
+    KubernetesVLLMModelDeployerFlavor,
+)
 from zenml.integrations.vllm.flavors.vllm_model_deployer_flavor import (  # noqa
     VLLMModelDeployerConfig,
     VLLMModelDeployerFlavor,
 )
 
-__all__ = ["VLLMModelDeployerConfig", "VLLMModelDeployerFlavor"]
+__all__ = [
+    "KubernetesVLLMModelDeployerConfig",
+    "KubernetesVLLMModelDeployerFlavor",
+    "VLLMModelDeployerConfig",
+    "VLLMModelDeployerFlavor",
+]

src/zenml/integrations/vllm/flavors/vllm_kubernetes_model_deployer_flavor.py

@@ -0,0 +1,164 @@
+#  Copyright (c) ZenML GmbH 2026. All Rights Reserved.
+#
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at:
+#
+#       https://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+#  or implied. See the License for the specific language governing
+#  permissions and limitations under the License.
+"""vLLM Kubernetes model deployer flavor."""
+
+from typing import TYPE_CHECKING, Optional, Type
+
+from pydantic import Field
+
+from zenml.constants import KUBERNETES_CLUSTER_RESOURCE_TYPE
+from zenml.enums import KubernetesServiceType
+from zenml.integrations.vllm import VLLM_KUBERNETES_MODEL_DEPLOYER_FLAVOR
+from zenml.model_deployers.base_model_deployer import (
+    BaseModelDeployerConfig,
+    BaseModelDeployerFlavor,
+)
+from zenml.models import ServiceConnectorRequirements
+from zenml.utils.secret_utils import SecretField
+
+if TYPE_CHECKING:
+    from zenml.integrations.vllm.model_deployers import (
+        KubernetesVLLMModelDeployer,
+    )
+
+DEFAULT_VLLM_IMAGE = "vllm/vllm-openai:v0.25.1"
+
+
+class KubernetesVLLMModelDeployerConfig(BaseModelDeployerConfig):
+    """Configuration for the Kubernetes vLLM model deployer.
+
+    Attributes:
+        kubernetes_context: name of a Kubernetes context to deploy vLLM servers to. If the stack component is linked to a Kubernetes service connector, this field is ignored.
+        kubernetes_namespace: default Kubernetes namespace for vLLM deployments. Can be overridden per-deployment on the service config.
+        incluster: if `True`, connect using the Kubernetes configuration of the cluster the client itself is running in. Requires the client to run in a Kubernetes pod. If set, `kubernetes_context` is ignored.
+        default_image: default vLLM server image used when a deployment doesn't specify one.
+        default_service_type: default Kubernetes Service type used when a deployment doesn't specify one.
+        hf_token: Hugging Face access token used to download gated models when a deployment doesn't configure its own token or existing secret.
+    """
+
+    kubernetes_context: Optional[str] = Field(
+        default=None,
+        description="Name of a Kubernetes context to deploy vLLM servers "
+        "to. Ignored when the stack component is linked to a Kubernetes "
+        "service connector",
+    )
+    kubernetes_namespace: str = Field(
+        default="zenml-vllm",
+        description="Kubernetes namespace in which vLLM deployment "
+        "servers are provisioned and managed by ZenML",
+    )
+    incluster: bool = Field(
+        default=False,
+        description="Connect using the Kubernetes configuration of the "
+        "cluster the client itself is running in. Requires the client to "
+        "run inside a Kubernetes pod. When set, `kubernetes_context` is "
+        "ignored",
+    )
+    default_image: str = Field(
+        default=DEFAULT_VLLM_IMAGE,
+        description="Container image used to run the vLLM OpenAI-"
+        "compatible server when a deployment doesn't specify its own "
+        "image",
+    )
+    default_service_type: KubernetesServiceType = Field(
+        default=KubernetesServiceType.LOAD_BALANCER,
+        description="Kubernetes Service type used to expose a vLLM "
+        "deployment when a deployment doesn't specify its own service "
+        "type",
+    )
+    hf_token: Optional[str] = SecretField(
+        default=None,
+        description="Hugging Face access token used to download gated "
+        "models. Used as a fallback for deployments that configure "
+        "neither their own token nor an existing Kubernetes secret",
+    )
+
+
+class KubernetesVLLMModelDeployerFlavor(BaseModelDeployerFlavor):
+    """Kubernetes vLLM model deployer flavor."""
+
+    @property
+    def name(self) -> str:
+        """Name of the flavor.
+
+        Returns:
+            The name of the flavor.
+        """
+        return VLLM_KUBERNETES_MODEL_DEPLOYER_FLAVOR
+
+    @property
+    def service_connector_requirements(
+        self,
+    ) -> Optional[ServiceConnectorRequirements]:
+        """Service connector resource requirements for service connectors.
+
+        Specifies resource requirements that are used to filter the available
+        service connector types that are compatible with this flavor.
+
+        Returns:
+            Requirements for compatible service connectors, if a service
+            connector is required for this flavor.
+        """
+        return ServiceConnectorRequirements(
+            resource_type=KUBERNETES_CLUSTER_RESOURCE_TYPE,
+        )
+
+    @property
+    def docs_url(self) -> Optional[str]:
+        """A url to point at docs explaining this flavor.
+
+        Returns:
+            A flavor docs url.
+        """
+        return self.generate_default_docs_url()
+
+    @property
+    def sdk_docs_url(self) -> Optional[str]:
+        """A url to point at SDK docs explaining this flavor.
+
+        Returns:
+            A flavor SDK docs url.
+        """
+        return self.generate_default_sdk_docs_url()
+
+    @property
+    def logo_url(self) -> str:
+        """A url to represent the flavor in the dashboard.
+
+        Returns:
+            The flavor logo.
+        """
+        return "https://public-flavor-logos.s3.eu-central-1.amazonaws.com/model_deployer/vllm.png"
+
+    @property
+    def config_class(self) -> Type[KubernetesVLLMModelDeployerConfig]:
+        """Returns `KubernetesVLLMModelDeployerConfig` config class.
+
+        Returns:
+            The config class.
+        """
+        return KubernetesVLLMModelDeployerConfig
+
+    @property
+    def implementation_class(self) -> Type["KubernetesVLLMModelDeployer"]:
+        """Implementation class for this flavor.
+
+        Returns:
+            The implementation class.
+        """
+        from zenml.integrations.vllm.model_deployers import (
+            KubernetesVLLMModelDeployer,
+        )
+
+        return KubernetesVLLMModelDeployer