Skip to content

docker: upgrade dev vLLM image to 0.26.0 #604

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
gc-fu wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

docker: upgrade dev vLLM image to 0.26.0 #604

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
763a7dc
docker: upgrade dev vLLM image to 0.26.0
gc-fu Aug 7, 2026
4548a64
docker: remove fixed library path
gc-fu Aug 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
282 changes: 103 additions & 179 deletions vllm/docker/Dockerfile.dev
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,44 +1,53 @@
# Copyright (C) 2025 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
# DEV / DEBUG build of llm-scaler-vllm (Intel XPU). Single stage.
# Development image for upstream vLLM v0.26.0 on Intel XPU.
#
# base : intel/omix:0.1.0-devel-ubuntu24.04
# - oneAPI 2025.3 DPC++ compiler (icpx), oneCCL 2021.17, oneDNN/oneMKL
# - AND the SAME Intel GPU runtime stack as the prod runtime base
# (intel/pytorch:xpu-2.11.0): intel-opencl-icd/ocloc/libze-intel-gpu1
# 26.14.37833.4, libze_loader 1.28.2, libigc2 2.32.7, libigdgmm 22.10.0.
# Verified byte-identical -> this image can BOTH compile and run on GPU.
# This image intentionally uses only the upstream vLLM source tree and the
# upstream vllm-xpu-kernels wheel. It does not include llm-scaler multi-arc
# patches, custom ESIMD kernels, private kernel builds, or BigDL extensions.
#
# Relationship to prod (docker/Dockerfile):
# prod is a 2-stage build that ships a stripped, cleaned runtime WITHOUT a compiler.
# Great for size, painful to debug. This dev image keeps EVERYTHING:
# * the oneAPI compiler + GPU runtime (base already has both)
# * the /llm-scaler/vllm/{vllm,vllm-xpu-kernels,custom-esimd-kernels-vllm} source
# trees AND their build/ dirs, so any component can be rebuilt in place
# * debug symbols (NO `strip`, NO plugin dedup, NO cleanup)
# * gdb / debug tooling
# Package versions are kept in lockstep with prod: torch 2.11.0+xpu, the same
# vLLM v0.21.0 + multi_arc patch, the same esimd kernels, the same
# vllm-xpu-kernels commit + patch, transformers 5.8.0, triton-xpu 3.7.0,
# bigdl-core 2.4.0b2.
#
# The ONE intentional divergence: vLLM is installed EDITABLE (`pip install -e`)
# from the persisted source tree, so Python-side edits take effect without a
# rebuild. The two kernel packages are installed as wheels (native .so), same
# as prod, but their sources + build/ dirs are kept for recompilation.
#
# Build context root MUST be the `vllm/` dir (so ./patches, ./custom-esimd-kernels-vllm
# and ./examples resolve):
# docker build -f docker/Dockerfile.dev -t <name>:v0.21.0-omix-pr508-dev .
# Build context root must be this `vllm/` directory:
# docker build -f docker/Dockerfile.dev -t <name>:v0.26.0-xpu-dev .

######################### RUST BUILD IMAGE #########################
# vLLM 0.26.0 ships Rust frontend artifacts. Build them separately so the
# development image only contains the artifacts, not Rust or protoc.
FROM ubuntu:22.04 AS rust-build

ARG VLLM_VERSION=v0.26.0
ENV DEBIAN_FRONTEND=noninteractive \
CARGO_BUILD_JOBS=4

RUN apt-get update -y \
&& apt-get install -y --no-install-recommends \
ca-certificates curl git build-essential unzip python3 python3-pip \
&& rm -rf /var/lib/apt/lists/*

RUN git clone --depth 1 --branch "${VLLM_VERSION}" \
https://github.com/vllm-project/vllm.git /workspace/vllm

WORKDIR /workspace/vllm

RUN ./tools/install_protoc.sh \
&& python3 -m pip install --no-cache-dir -r requirements/build/rust.txt

# Keep the Cargo cache mounts on the Rust build itself. The separate RUN above
# installs the lightweight Python build dependencies first for better caching.
RUN --mount=type=cache,target=/root/.cargo/registry,sharing=locked \
--mount=type=cache,target=/root/.cargo/git,sharing=locked \
bash build_rust.sh

######################### VLLM XPU DEVELOPMENT IMAGE #########################
# omix 0.1.0 provides oneAPI 2025.3, matching Torch 2.12's
# intel-sycl-rt==2025.3.2 dependency. Do not replace it with omix 0.2.0
# (oneAPI 2026.0) without also changing the Torch XPU ABI.
FROM intel/omix:0.1.0-devel-ubuntu24.04

ARG http_proxy
ARG https_proxy
ARG no_proxy
# vllm-xpu-kernels pinned base commit (identical to prod build).
ARG VLLM_XPU_KERNELS_COMMIT=3cab97a
ARG VLLM_VERSION=v0.26.0
ARG ONECCL_INSTALLER="intel-oneccl-2021.15.9.14_offline.sh"
ARG ONECCL_INSTALLER_SHA256="f7ab81b6ed1b10dd35fadec366a78046d8af214888dfd625047ce8953d5aa4ef"

Expand All @@ -50,169 +59,84 @@ ENV VIRTUAL_ENV=/opt/venv \
UV_INDEX_STRATEGY="unsafe-best-match" \
UV_LINK_MODE="copy" \
VLLM_TARGET_DEVICE=xpu \
WHEELS=/wheels \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
VLLM_QUANTIZE_Q40_LIB="/opt/venv/lib/python3.12/site-packages/vllm_int4_for_multi_arc.so" \
VLLM_OFFLOAD_WEIGHTS_BEFORE_QUANT=1 \
VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn

# Build RUN steps use `source .../setvars.sh` and `set -eo pipefail` (bash-only).
VLLM_WORKER_MULTIPROC_METHOD=spawn \
PIP_DISABLE_PIP_VERSION_CHECK=1

SHELL ["bash", "-c"]
WORKDIR /llm-scaler/vllm

# --- build tooling + runtime OS deps + debug tooling (the oneAPI DPC++ compiler
# and the Intel GPU runtime are already in the OMIX devel base) ---
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt/lists,sharing=locked \
apt-get update -y && \
apt-get install -y --no-install-recommends --fix-missing \
apt-get update -y \
&& apt-get install -y --no-install-recommends --fix-missing \
git curl ca-certificates \
python3-dev python3-venv python3-pip \
ninja-build cmake pkg-config numactl \
ffmpeg libsndfile1 libsm6 libxext6 libgl1 \
gdb vim less htop procps

# --- isolated venv at /opt/venv (same path prod runtime uses, so the env-var
# paths above resolve; OMIX system python is PEP-668 blocked) ---
RUN python3 -m venv ${VIRTUAL_ENV}

# --- uv for fast, deterministic installs (matches prod) ---
RUN curl -LsSf https://astral.sh/uv/install.sh | sh

# --- torch pinned to the prod runtime ABI (2.11.0+xpu) so all native .so link
# against the libtorch the image ships ---
RUN --mount=type=cache,target=/root/.cache/uv \
pip install --upgrade pip wheel setuptools && \
uv pip install torch==2.11.0+xpu torchaudio torchvision \
--index-url https://download.pytorch.org/whl/xpu

RUN mkdir -p ${WHEELS}

COPY ./patches/vllm_for_multi_arc.patch ./patches/vllm_xpu_kernels.patch /tmp/
COPY ./custom-esimd-kernels-vllm /llm-scaler/vllm/custom-esimd-kernels-vllm

WORKDIR /llm-scaler/vllm

# This Intel(R) oneAPI Collective Communications Library (oneCCL) contains several
# enhancements for Intel(R) Arc(TM) Pro graphics.
# For details, please refer to:
# https://github.com/uxlfoundation/oneCCL/releases/tag/2021.15.9
RUN curl -fL --retry 3 -o "${ONECCL_INSTALLER}" "https://github.com/uxlfoundation/oneCCL/releases/download/2021.15.9/${ONECCL_INSTALLER}" && \
printf "%s %s\n" "${ONECCL_INSTALLER_SHA256}" "${ONECCL_INSTALLER}" > /tmp/oneccl.sha256 && \
sha256sum -c /tmp/oneccl.sha256 && \
rm -f /tmp/oneccl.sha256 && \
bash "${ONECCL_INSTALLER}" -s -x -f /tmp && \
ONECCL_CUP="/tmp/${ONECCL_INSTALLER%.sh}/packages/intel.oneapi.lin.ccl.runtime,v=2021.15.9+13/cupPayload.cup" && \
ONECCL_DST="/tmp/oneccl-runtime" && \
python3 -m zipfile -e "${ONECCL_CUP}" "${ONECCL_DST}" && \
ONECCL_LIB="${ONECCL_DST}/_installdir/ccl/2021.15/lib" && \
rm -f "${ONECCL_LIB}/libccl.so" "${ONECCL_LIB}/libccl.so.1" "${ONECCL_LIB}/libccl_openmp.so" "${ONECCL_LIB}/libccl_openmp.so.0" && \
ln -sf libccl.so.1 "${ONECCL_LIB}/libccl.so" && \
ln -sf libccl.so.1.0 "${ONECCL_LIB}/libccl.so.1" && \
ln -sf libccl_openmp.so.0 "${ONECCL_LIB}/libccl_openmp.so" && \
ln -sf libccl_openmp.so.0.1 "${ONECCL_LIB}/libccl_openmp.so.0" && \
ONECCL_CPU_LIB="${ONECCL_DST}/_installdir/ccl/2021.15/lib/ccl/cpu/lib" && \
rm -f "${ONECCL_CPU_LIB}/libccl.so" "${ONECCL_CPU_LIB}/libccl.so.1" "${ONECCL_CPU_LIB}/libccl_openmp.so" "${ONECCL_CPU_LIB}/libccl_openmp.so.0" && \
ln -sf libccl.so.1 "${ONECCL_CPU_LIB}/libccl.so" && \
ln -sf libccl.so.1.0 "${ONECCL_CPU_LIB}/libccl.so.1" && \
ln -sf libccl_openmp.so.0 "${ONECCL_CPU_LIB}/libccl_openmp.so" && \
ln -sf libccl_openmp.so.0.1 "${ONECCL_CPU_LIB}/libccl_openmp.so.0" && \
mkdir -p /opt/intel/oneapi/ccl && \
rm -rf /opt/intel/oneapi/ccl/2021.15 && \
cp -a /tmp/oneccl-runtime/_installdir/ccl/2021.15 /opt/intel/oneapi/ccl/ && \
rm "${ONECCL_INSTALLER}" && \
echo "source /opt/intel/oneapi/setvars.sh --force" >> /root/.bashrc && \
echo "source /opt/intel/oneapi/ccl/2021.15/env/vars.sh --force" >> /root/.bashrc && \
rm -f /opt/intel/oneapi/ccl/latest && \
ln -s /opt/intel/oneapi/ccl/2021.15 /opt/intel/oneapi/ccl/latest && \
find /opt/intel/oneapi/ccl -mindepth 1 -maxdepth 1 -type d -not -name 2021.15 -exec rm -rf {} + && \
rm -rf "/tmp/${ONECCL_INSTALLER%.sh}" /tmp/oneccl-runtime

# ---------------------------------------------------------------------------
# vLLM v0.21.0 + multi-arc patch -> EDITABLE install (source kept at
# /llm-scaler/vllm/vllm, native ext + build/ compiled in place).
# Deps resolve here (mirrors prod's `pip install vllm-*.whl`); the triton
# clobber they pull is repaired at the end, same as prod.
# ---------------------------------------------------------------------------
RUN --mount=type=cache,target=/root/.cache/uv \
set -eo pipefail && \
source /opt/intel/oneapi/setvars.sh --force && \
git clone --depth 1 -b v0.21.0 https://github.com/vllm-project/vllm.git ./vllm && \
cd ./vllm && \
git apply /tmp/vllm_for_multi_arc.patch && \
uv pip install -r requirements/xpu.txt && \
uv pip install grpcio-tools protobuf nanobind && \
export CMAKE_PREFIX_PATH="$(python3 -c 'import site; print(site.getsitepackages()[0])'):${CMAKE_PREFIX_PATH:-}" && \
pip install --no-build-isolation -e . \
--extra-index-url https://download.pytorch.org/whl/xpu

# ---------------------------------------------------------------------------
# custom ESIMD kernels -> wheel + install (TORCH_XPU_ARCH_LIST=bmg-g21).
# Source + build/ + dist/ kept for recompilation.
# ---------------------------------------------------------------------------
RUN --mount=type=cache,target=/root/.cache/uv \
set -eo pipefail && \
source /opt/intel/oneapi/setvars.sh --force && \
cd ./custom-esimd-kernels-vllm && \
TORCH_XPU_ARCH_LIST=bmg-g21 MAX_JOBS=1 python3 setup.py bdist_wheel && \
cp dist/*.whl ${WHEELS}/ && \
pip install --no-deps --force-reinstall dist/*.whl

# ---------------------------------------------------------------------------
# vllm-xpu-kernels (patch + pinned commit) -> wheel + install.
# Source + build/ kept for recompilation.
# ---------------------------------------------------------------------------
RUN --mount=type=cache,target=/root/.cache/uv \
set -eo pipefail && \
source /opt/intel/oneapi/setvars.sh --force && \
git clone https://github.com/vllm-project/vllm-xpu-kernels.git ./vllm-xpu-kernels && \
cd ./vllm-xpu-kernels && \
git checkout ${VLLM_XPU_KERNELS_COMMIT} && \
git apply /tmp/vllm_xpu_kernels.patch && \
sed -i 's|^--extra-index-url=https://download.pytorch.org/whl/xpu|# --extra-index-url=https://download.pytorch.org/whl/xpu|' requirements.txt && \
sed -i '/^torch==/s/^/# /' requirements.txt && \
sed -i 's|^triton-xpu|# triton-xpu|' requirements.txt && \
sed -i 's|^transformers|# transformers|' requirements.txt && \
uv pip install -r requirements.txt && \
export CMAKE_PREFIX_PATH="$(python3 -c 'import site; print(site.getsitepackages()[0])'):${CMAKE_PREFIX_PATH:-}" && \
pip wheel --no-build-isolation --no-deps . -w ${WHEELS} && \
pip install --no-deps --force-reinstall ${WHEELS}/vllm_xpu_kernels-*.whl

# ---------------------------------------------------------------------------
# lean-core serving deps (same versions as prod) + bigdl-core (ships
# vllm_int4_for_multi_arc.so used by the sym_int4 path).
# ---------------------------------------------------------------------------
ENV LD_LIBRARY_PATH=/opt/intel/oneapi/ccl/2021.15/lib:/opt/intel/oneapi/mpi/2021.15/lib:/opt/intel/oneapi/compiler/2025.3/lib:/usr/local/lib

RUN curl -fL --retry 3 -o "${ONECCL_INSTALLER}" \
"https://github.com/uxlfoundation/oneCCL/releases/download/2021.15.9/${ONECCL_INSTALLER}" \
&& printf "%s %s\n" "${ONECCL_INSTALLER_SHA256}" "${ONECCL_INSTALLER}" > /tmp/oneccl.sha256 \
&& sha256sum -c /tmp/oneccl.sha256 \
&& rm -f /tmp/oneccl.sha256 \
&& bash "${ONECCL_INSTALLER}" -s -x -f /tmp \
&& ONECCL_CUP="/tmp/${ONECCL_INSTALLER%.sh}/packages/intel.oneapi.lin.ccl.runtime,v=2021.15.9+13/cupPayload.cup" \
&& ONECCL_DST="/tmp/oneccl-runtime" \
&& python3 -m zipfile -e "${ONECCL_CUP}" "${ONECCL_DST}" \
&& ONECCL_LIB="${ONECCL_DST}/_installdir/ccl/2021.15/lib" \
&& rm -f "${ONECCL_LIB}/libccl.so" "${ONECCL_LIB}/libccl.so.1" "${ONECCL_LIB}/libccl_openmp.so" "${ONECCL_LIB}/libccl_openmp.so.0" \
&& ln -sf libccl.so.1 "${ONECCL_LIB}/libccl.so" \
&& ln -sf libccl.so.1.0 "${ONECCL_LIB}/libccl.so.1" \
&& ln -sf libccl_openmp.so.0 "${ONECCL_LIB}/libccl_openmp.so" \
&& ln -sf libccl_openmp.so.0.1 "${ONECCL_LIB}/libccl_openmp.so.0" \
&& ONECCL_CPU_LIB="${ONECCL_DST}/_installdir/ccl/2021.15/lib/ccl/cpu/lib" \
&& rm -f "${ONECCL_CPU_LIB}/libccl.so" "${ONECCL_CPU_LIB}/libccl.so.1" "${ONECCL_CPU_LIB}/libccl_openmp.so" "${ONECCL_CPU_LIB}/libccl_openmp.so.0" \
&& ln -sf libccl.so.1 "${ONECCL_CPU_LIB}/libccl.so" \
&& ln -sf libccl.so.1.0 "${ONECCL_CPU_LIB}/libccl.so.1" \
&& ln -sf libccl_openmp.so.0 "${ONECCL_CPU_LIB}/libccl_openmp.so" \
&& ln -sf libccl_openmp.so.0.1 "${ONECCL_CPU_LIB}/libccl_openmp.so.0" \
&& mkdir -p /opt/intel/oneapi/ccl \
&& rm -rf /opt/intel/oneapi/ccl/2021.15 \
&& cp -a /tmp/oneccl-runtime/_installdir/ccl/2021.15 /opt/intel/oneapi/ccl/ \
&& rm "${ONECCL_INSTALLER}" \
&& echo "source /opt/intel/oneapi/setvars.sh --force" >> /root/.bashrc \
&& echo "source /opt/intel/oneapi/ccl/2021.15/env/vars.sh --force" >> /root/.bashrc \
&& rm -f /opt/intel/oneapi/ccl/latest \
&& ln -s /opt/intel/oneapi/ccl/2021.15 /opt/intel/oneapi/ccl/latest \
&& find /opt/intel/oneapi/ccl -mindepth 1 -maxdepth 1 -type d -not -name 2021.15 -exec rm -rf {} + \
&& rm -rf "/tmp/${ONECCL_INSTALLER%.sh}" /tmp/oneccl-runtime

# Clone and build only the upstream vLLM release. requirements/xpu.txt pins
# Torch 2.12, vllm-xpu-kernels 0.1.11.1, AutoRound 0.14.1, and TorchCodec.
RUN --mount=type=cache,target=/root/.cache/uv \
set -eo pipefail && \
pip install transformers==5.8.0 accelerate hf_transfer 'modelscope!=1.15.0' && \
pip install librosa soundfile decord ijson && \
pip install bigdl-core==2.4.0b2

# ---------------------------------------------------------------------------
# triton-xpu fix (same as prod): installing vllm deps pulls upstream CUDA
# triton==3.7.1 (via xgrammar), which drops the `intel` backend. Reinstall
# triton-xpu 3.7.0. DEV DIFF: do NOT strip libtriton.so / dedup plugins /
# run any cleanup -- debug symbols are kept.
# ---------------------------------------------------------------------------
set -eo pipefail \
&& source /opt/intel/oneapi/setvars.sh --force \
&& git clone --depth 1 --branch "${VLLM_VERSION}" \
https://github.com/vllm-project/vllm.git ./vllm \
&& cd ./vllm \
&& uv pip install -r requirements/xpu.txt \
&& uv pip install grpcio-tools protobuf nanobind

COPY --from=rust-build /workspace/vllm/vllm/vllm-rs /llm-scaler/vllm/vllm/vllm-rs
COPY --from=rust-build /workspace/vllm/vllm/_rust_*.so /llm-scaler/vllm/vllm/

# xgrammar may install CUDA Triton. XPU must use Triton-XPU 3.7.1.
# The offline oneCCL installation above replaces the Python wheel runtime.
RUN --mount=type=cache,target=/root/.cache/uv \
set -eo pipefail && \
pip uninstall -y triton triton-xpu || true && \
pip install --no-deps triton-xpu==3.7.0 \
--extra-index-url https://download.pytorch.org/whl/xpu

# Replace torch-bundled oneCCL stubs in /opt/venv/lib with links to oneCCL 2021.15.
RUN set -eo pipefail && \
pip uninstall -y oneccl oneccl-devel || true && \
rm -f /opt/venv/lib/libccl.so /opt/venv/lib/libccl.so.1 /opt/venv/lib/libccl.so.1.0 \
/opt/venv/lib/libccl_openmp.so /opt/venv/lib/libccl_openmp.so.0 /opt/venv/lib/libccl_openmp.so.0.1 && \
ln -sf /opt/intel/oneapi/ccl/2021.15/lib/libccl.so.1.0 /opt/venv/lib/libccl.so.1 && \
ln -sf /opt/venv/lib/libccl.so.1 /opt/venv/lib/libccl.so && \
ln -sf /opt/intel/oneapi/ccl/2021.15/lib/libccl_openmp.so.0.1 /opt/venv/lib/libccl_openmp.so.0 && \
ln -sf /opt/venv/lib/libccl_openmp.so.0 /opt/venv/lib/libccl_openmp.so

# Component inventory in the build log for sanity.
RUN pip list 2>/dev/null | grep -iE "^(vllm|torch|triton|transformers|bigdl|accelerate)" || true

# --- dev default: interactive login shell (oneAPI env auto-sourced by the base).
# Override with `docker run ... vllm serve <model> ...` to serve. ---
uv pip uninstall triton triton-xpu \
&& uv pip install triton-xpu==3.7.1 \
&& uv pip uninstall oneccl oneccl-devel \
Comment on lines +132 to +134
&& source /opt/intel/oneapi/setvars.sh --force \
&& cd ./vllm \
&& uv pip install --no-build-isolation --no-deps -e .

# The dev image defaults to a login shell. To serve, pass:
# docker run ... <image> vllm serve <model> ...
CMD ["/bin/bash", "-l"]