diff --git a/docs/configuration/README.md b/docs/configuration/README.md index fc12c2663..b8640ecad 100644 --- a/docs/configuration/README.md +++ b/docs/configuration/README.md @@ -2,7 +2,7 @@ This section lists the most common options for running vLLM-Omni. -For options within a vLLM Engine, please refer to the [vLLM 0.30 configuration guide](https://docs.vllm.ai/en/v0.30.0/configuration/index.html). +For options within a vLLM Engine, please refer to the [vLLM 0.31 configuration guide](https://docs.vllm.ai/en/v0.31.0/configuration/index.html). Each model defines fixed topology in a registered `PipelineConfig` and runtime overrides in a deploy YAML. diff --git a/docs/configuration/environment_variables.md b/docs/configuration/environment_variables.md index a872eaf3c..0e614b789 100644 --- a/docs/configuration/environment_variables.md +++ b/docs/configuration/environment_variables.md @@ -186,7 +186,7 @@ their keys only. ## Inherited vLLM variables vLLM-Omni also reads variables through its aligned vLLM dependency. Refer to -the [vLLM 0.30 environment-variable reference](https://docs.vllm.ai/en/v0.30.0/configuration/env_vars.html) +the [vLLM 0.31 environment-variable reference](https://docs.vllm.ai/en/v0.31.0/configuration/env_vars.html) for their definitions. This includes vLLM launch, cache, logging, plugin, ROCm, XPU, ModelScope, and FlashInfer workspace settings. diff --git a/docs/getting_started/installation/README.md b/docs/getting_started/installation/README.md index 507a0bd96..e82075aba 100644 --- a/docs/getting_started/installation/README.md +++ b/docs/getting_started/installation/README.md @@ -1,7 +1,7 @@ # Installation !!! important - vLLM-Omni is released against the matching upstream vLLM major/minor version. The vLLM-Omni 0.30.x release line uses vLLM 0.30.x; the source-install instructions pin vLLM 0.30.0. + vLLM-Omni is released against the matching upstream vLLM major/minor version. This source checkout targets vLLM 0.31.0. The published vLLM-Omni 0.30.0 wheel instructions use vLLM 0.30.0; choose the installation path that matches your Omni revision. vLLM-Omni supports the following hardware platforms: diff --git a/docs/getting_started/installation/gpu/cuda.inc.md b/docs/getting_started/installation/gpu/cuda.inc.md index d3704b0b1..fb7db8717 100644 --- a/docs/getting_started/installation/gpu/cuda.inc.md +++ b/docs/getting_started/installation/gpu/cuda.inc.md @@ -1,11 +1,13 @@ # --8<-- [start:requirements] -- GPU: compute capability 7.0 or higher (e.g., V100, T4, RTX20xx, A100, L4, H100, etc.) +- GPU: compute capability 7.5 or higher (e.g., T4, RTX20xx, A100, L4, H100, etc.) # --8<-- [end:requirements] # --8<-- [start:set-up-using-python] -vLLM-Omni depends on the matching major/minor release of vLLM. The vLLM-Omni 0.30.x release line uses vLLM 0.30.x. +vLLM-Omni depends on the matching major/minor release of vLLM. This source +checkout targets vLLM 0.31.0. The pre-built Omni wheel instructions below install +the published 0.30.0 release with its matching vLLM dependency. !!! note PyTorch installed via `conda` will statically link `NCCL` library, which can cause issues when vLLM tries to use `NCCL`. See for more details. @@ -41,13 +43,17 @@ uv pip install 'vllm-omni[demo]==0.30.0' # --8<-- [start:build-wheel-from-source] #### Installation of vLLM -If you do not need to modify source code of vLLM, you can directly install the stable 0.30.0 release version of the library +If you do not need to modify source code of vLLM, install the final 0.31.0 +release wheel from its commit-pinned index: ```bash -uv pip install vllm==0.30.0 --torch-backend=auto +uv pip install vllm==0.31.0 --torch-backend=auto \ + --extra-index-url https://wheels.vllm.ai/db9527a46873454610df6dbedf79a36d6bf1a7f6 ``` -The 0.30.0 release of vLLM ships CUDA 13.0-compatible binaries by default. If you need a different CUDA variant or want to reuse an existing PyTorch installation, build vLLM from source instead. +The default 0.31.0 wheel contains CUDA 12.9 binaries. If you need a different +CUDA variant or want to reuse an existing PyTorch installation, build vLLM +from source instead. #### Installation of vLLM-Omni Since vllm-omni is rapidly evolving, it's recommended to install it from source @@ -68,16 +74,16 @@ If you want to check, modify or debug with source code of vLLM, install the libr ```bash git clone https://github.com/vllm-project/vllm.git cd vllm -git checkout v0.30.0 +git checkout v0.31.0 ``` Set up environment variables to get pre-built wheels. If there are internet problems, just download the whl file manually. And set `VLLM_PRECOMPILED_WHEEL_LOCATION` as your local absolute path of whl file. ```bash -#For CUDA 13.0 (the default for v0.30.0; the wheel filename has no `+cu130` suffix) -export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.30.0/vllm-0.30.0-cp38-abi3-manylinux_2_28_x86_64.whl +# Default CUDA 12.9 wheel for v0.31.0 on x86_64 +export VLLM_PRECOMPILED_WHEEL_LOCATION=https://wheels.vllm.ai/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm-0.31.0-cp38-abi3-manylinux_2_28_x86_64.whl ``` Install vllm with command below (If you have no existing PyTorch). ```bash -uv pip install --editable . +VLLM_USE_PRECOMPILED=1 uv pip install --editable . --torch-backend=auto ``` Install vllm with command below (If you already have PyTorch). ```bash @@ -128,12 +134,13 @@ docker run --runtime nvidia --gpus 2 \ DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.cuda -t vllm-omni-cuda . ``` -If you want to specify the base vLLM version: +The source build defaults to `vllm/vllm-openai:v0.31.0`. To select that +upstream release image explicitly: ```bash DOCKER_BUILDKIT=1 docker build \ -f docker/Dockerfile.cuda \ - --build-arg BASE_IMAGE=vllm/vllm-openai:v0.30.0 \ + --build-arg BASE_IMAGE=vllm/vllm-openai:v0.31.0 \ -t vllm-omni-cuda . ``` diff --git a/docs/getting_started/installation/gpu/rocm.inc.md b/docs/getting_started/installation/gpu/rocm.inc.md index 63675ea94..db63f9488 100644 --- a/docs/getting_started/installation/gpu/rocm.inc.md +++ b/docs/getting_started/installation/gpu/rocm.inc.md @@ -10,8 +10,9 @@ For ROCm, vLLM-Omni currently recommends the setup steps through Docker Images. -vLLM-Omni depends on the matching major/minor release of vLLM. The vLLM-Omni -0.30.x release line uses vLLM 0.30.x. +vLLM-Omni depends on the matching major/minor release of vLLM. This source +checkout targets vLLM 0.31.0. The pre-built Omni wheel instructions below install +the published 0.30.0 release with its matching vLLM dependency. The Dockerfile's `BASE_IMAGE` pin applies only to Docker builds. The `vllm-omni` package does not install vLLM as a dependency, so non-Docker source @@ -49,13 +50,14 @@ uv pip install onnxruntime-rocm #### Installation of vLLM -If you do not need to modify source code of vLLM, you can directly install the stable 0.30.0 release version of the library +If you do not need to modify source code of vLLM, install the final 0.31.0 +release wheel for ROCm 7.2.3: ```bash -uv pip install vllm==0.30.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.30.0/rocm723 +uv pip install vllm==0.31.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.31.0/rocm723 ``` -The pre-built 0.30.0 vLLM wheel targets ROCm 7.2.3. If you need a different ROCm stack or want to reuse an existing PyTorch installation, build vLLM from source instead. +The pre-built 0.31.0 vLLM wheel targets ROCm 7.2.3. If you need a different ROCm stack or want to reuse an existing PyTorch installation, build vLLM from source instead. #### Installation of vLLM-Omni @@ -75,7 +77,7 @@ If you want to check, modify or debug with source code of vLLM, install the libr ```bash git clone https://github.com/vllm-project/vllm.git cd vllm -git checkout v0.30.0 +git checkout v0.31.0 python3 -m pip install -r requirements/rocm.txt python3 setup.py develop ``` @@ -87,7 +89,7 @@ python3 setup.py develop #### Build docker image The source build defaults to the published upstream base image -`vllm/vllm-openai-rocm:v0.30.0`, aligned with the vLLM release used by CI. +`vllm/vllm-openai-rocm:v0.31.0`, aligned with this source checkout. Older or custom bases must provide the vLLM APIs checked by the Dockerfile's image-build canary. This upstream base is distinct from the prebuilt `vllm/vllm-omni-rocm` images discussed below; their published-tag availability @@ -102,7 +104,7 @@ To select the upstream ROCm base explicitly, pass `BASE_IMAGE`: ```bash DOCKER_BUILDKIT=1 docker build \ -f docker/Dockerfile.rocm \ - --build-arg BASE_IMAGE=vllm/vllm-openai-rocm:v0.30.0 \ + --build-arg BASE_IMAGE=vllm/vllm-openai-rocm:v0.31.0 \ -t vllm-omni-rocm . ``` diff --git a/docs/getting_started/installation/gpu/xpu.inc.md b/docs/getting_started/installation/gpu/xpu.inc.md index 0232c17bf..49bbbf50f 100644 --- a/docs/getting_started/installation/gpu/xpu.inc.md +++ b/docs/getting_started/installation/gpu/xpu.inc.md @@ -20,12 +20,16 @@ vLLM-Omni currently recommends using the Docker image setup steps below. #### Build docker image ```bash -DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.xpu -t vllm-omni-xpu --shm-size=4g . +export VLLM_VERSION=v0.31.0 +DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.xpu -t vllm-omni-xpu --shm-size=4g \ + --build-arg VLLM_VERSION="$VLLM_VERSION" . ``` This layers vLLM-Omni on top of the published `vllm/vllm-openai-xpu:` -base image (v0.30.0 by default), which Docker pulls automatically. To target a different vLLM release, -pass `--build-arg VLLM_VERSION=`. If that tag has not been published yet, build +base image, which Docker pulls automatically. The Dockerfile still defaults to +v0.30.0; the command above explicitly selects v0.31.0 to match this source +checkout. To target a different vLLM release, pass +`--build-arg VLLM_VERSION=`. If that tag has not been published yet, build the base from upstream's own Dockerfile first and point the build at it: ```bash diff --git a/docs/getting_started/installation/npu/npu.inc.md b/docs/getting_started/installation/npu/npu.inc.md index ad29c8447..9bf7613f1 100644 --- a/docs/getting_started/installation/npu/npu.inc.md +++ b/docs/getting_started/installation/npu/npu.inc.md @@ -71,10 +71,14 @@ We are keeping [issue #886](https://github.com/vllm-project/vllm-omni/issues/886 # --8<-- [start:installation-main] -You can also install vLLM-Omni from the latest main branch if you want to use the latest features or bug fixes. Use the aligned vLLM-Ascend v0.30.0 image described above, and check [issue #886](https://github.com/vllm-project/vllm-omni/issues/886) for the status of the latest vLLM-Omni commit on NPU. +You can also install vLLM-Omni from source if you want to use the latest features +or bug fixes. This source checkout targets vLLM 0.31.0 and needs a matching +vLLM-Ascend runtime. The v0.30.0 images above are for the published Omni v0.30.0 +release. Check [issue #886](https://github.com/vllm-project/vllm-omni/issues/886) +for aligned versions and NPU validation before selecting a development image. ```bash -# Inside the vLLM-Ascend v0.30.0 container, install vLLM-Omni from main +# Inside a container with matching vLLM and vLLM-Ascend, install Omni from source cd /vllm-workspace git clone https://github.com/vllm-project/vllm-omni.git cd vllm-omni diff --git a/docs/getting_started/quickstart.md b/docs/getting_started/quickstart.md index 0e363c533..e3df99bcd 100644 --- a/docs/getting_started/quickstart.md +++ b/docs/getting_started/quickstart.md @@ -19,17 +19,20 @@ uv venv --python 3.12 --seed source .venv/bin/activate # On CUDA -uv pip install vllm==0.30.0 --torch-backend=auto +uv pip install vllm==0.31.0 --torch-backend=auto \ + --extra-index-url https://wheels.vllm.ai/db9527a46873454610df6dbedf79a36d6bf1a7f6 # On ROCm -uv pip install vllm==0.30.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.30.0/rocm723 +uv pip install vllm==0.31.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.31.0/rocm723 git clone https://github.com/vllm-project/vllm-omni.git cd vllm-omni uv pip install -e . ``` -For additional installation methods — please see the [installation guide](installation/README.md). +These commands install vLLM 0.31.0 for this source checkout. The CUDA index is +pinned to the final release commit. For published vLLM-Omni wheels and additional +installation methods, see the [installation guide](installation/README.md). !!! note It is important to install the same major & minor version of vLLM and vLLM Omni, otherwise things may not work as expected. If the versions are misaligned, you will see a warning when you import vLLM Omni.