anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

Align source installation documentation with vLLM 0.31.0

Browse Source 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
This commit is contained in:
tzhouam committed 2026-10-05 11:52:40 +08:00
1 parent df14622fb4
commit 07326d0922
8 files changed
+50 -30
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/configuration/README.md
+1 -1
View File
@@ -2,7 +2,7 @@
This section lists the most common options for running vLLM-Omni.
For options within a vLLM Engine, please refer to the [vLLM 0.30 configuration guide](https://docs.vllm.ai/en/v0.30.0/configuration/index.html).
For options within a vLLM Engine, please refer to the [vLLM 0.31 configuration guide](https://docs.vllm.ai/en/v0.31.0/configuration/index.html).
Each model defines fixed topology in a registered `PipelineConfig` and runtime
overrides in a deploy YAML.
docs/configuration/environment_variables.md
+1 -1
View File
@@ -186,7 +186,7 @@ their keys only.
## Inherited vLLM variables
vLLM-Omni also reads variables through its aligned vLLM dependency. Refer to
the [vLLM 0.30 environment-variable reference](https://docs.vllm.ai/en/v0.30.0/configuration/env_vars.html)
the [vLLM 0.31 environment-variable reference](https://docs.vllm.ai/en/v0.31.0/configuration/env_vars.html)
for their definitions. This includes vLLM launch, cache, logging, plugin, ROCm,
XPU, ModelScope, and FlashInfer workspace settings.
docs/getting_started/installation/README.md
+1 -1
View File
@@ -1,7 +1,7 @@
# Installation
!!! important
vLLM-Omni is released against the matching upstream vLLM major/minor version. The vLLM-Omni 0.30.x release line uses vLLM 0.30.x; the source-install instructions pin vLLM 0.30.0.
vLLM-Omni is released against the matching upstream vLLM major/minor version. This source checkout targets vLLM 0.31.0. The published vLLM-Omni 0.30.0 wheel instructions use vLLM 0.30.0; choose the installation path that matches your Omni revision.
vLLM-Omni supports the following hardware platforms:
docs/getting_started/installation/gpu/cuda.inc.md
+18 -11
View File
@@ -1,11 +1,13 @@
# --8<-- [start:requirements]
- GPU: compute capability 7.0 or higher (e.g., V100, T4, RTX20xx, A100, L4, H100, etc.)
- GPU: compute capability 7.5 or higher (e.g., T4, RTX20xx, A100, L4, H100, etc.)
# --8<-- [end:requirements]
# --8<-- [start:set-up-using-python]
vLLM-Omni depends on the matching major/minor release of vLLM. The vLLM-Omni 0.30.x release line uses vLLM 0.30.x.
vLLM-Omni depends on the matching major/minor release of vLLM. This source
checkout targets vLLM 0.31.0. The pre-built Omni wheel instructions below install
the published 0.30.0 release with its matching vLLM dependency.
!!! note
PyTorch installed via `conda` will statically link `NCCL` library, which can cause issues when vLLM tries to use `NCCL`. See <gh-issue:8420> for more details.
@@ -41,13 +43,17 @@ uv pip install 'vllm-omni[demo]==0.30.0'
# --8<-- [start:build-wheel-from-source]
#### Installation of vLLM
If you do not need to modify source code of vLLM, you can directly install the stable 0.30.0 release version of the library
If you do not need to modify source code of vLLM, install the final 0.31.0
release wheel from its commit-pinned index:
```bash
uv pip install vllm==0.30.0 --torch-backend=auto
uv pip install vllm==0.31.0 --torch-backend=auto \
--extra-index-url https://wheels.vllm.ai/db9527a46873454610df6dbedf79a36d6bf1a7f6
```
The 0.30.0 release of vLLM ships CUDA 13.0-compatible binaries by default. If you need a different CUDA variant or want to reuse an existing PyTorch installation, build vLLM from source instead.
The default 0.31.0 wheel contains CUDA 12.9 binaries. If you need a different
CUDA variant or want to reuse an existing PyTorch installation, build vLLM
from source instead.
#### Installation of vLLM-Omni
Since vllm-omni is rapidly evolving, it's recommended to install it from source
@@ -68,16 +74,16 @@ If you want to check, modify or debug with source code of vLLM, install the libr
```bash
git clone https://github.com/vllm-project/vllm.git
cd vllm
git checkout v0.30.0
git checkout v0.31.0
```
Set up environment variables to get pre-built wheels. If there are internet problems, just download the whl file manually. And set `VLLM_PRECOMPILED_WHEEL_LOCATION` as your local absolute path of whl file.
```bash
#For CUDA 13.0 (the default for v0.30.0; the wheel filename has no `+cu130` suffix)
export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.30.0/vllm-0.30.0-cp38-abi3-manylinux_2_28_x86_64.whl
# Default CUDA 12.9 wheel for v0.31.0 on x86_64
export VLLM_PRECOMPILED_WHEEL_LOCATION=https://wheels.vllm.ai/db9527a46873454610df6dbedf79a36d6bf1a7f6/vllm-0.31.0-cp38-abi3-manylinux_2_28_x86_64.whl
```
Install vllm with command below (If you have no existing PyTorch).
```bash
uv pip install --editable .
VLLM_USE_PRECOMPILED=1 uv pip install --editable . --torch-backend=auto
```
Install vllm with command below (If you already have PyTorch).
```bash
@@ -128,12 +134,13 @@ docker run --runtime nvidia --gpus 2 \
DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.cuda -t vllm-omni-cuda .
```
If you want to specify the base vLLM version:
The source build defaults to `vllm/vllm-openai:v0.31.0`. To select that
upstream release image explicitly:
```bash
DOCKER_BUILDKIT=1 docker build \
-f docker/Dockerfile.cuda \
--build-arg BASE_IMAGE=vllm/vllm-openai:v0.30.0 \
--build-arg BASE_IMAGE=vllm/vllm-openai:v0.31.0 \
-t vllm-omni-cuda .
```
docs/getting_started/installation/gpu/rocm.inc.md
+10 -8
View File
@@ -10,8 +10,9 @@
For ROCm, vLLM-Omni currently recommends the setup steps through Docker Images.
vLLM-Omni depends on the matching major/minor release of vLLM. The vLLM-Omni
0.30.x release line uses vLLM 0.30.x.
vLLM-Omni depends on the matching major/minor release of vLLM. This source
checkout targets vLLM 0.31.0. The pre-built Omni wheel instructions below install
the published 0.30.0 release with its matching vLLM dependency.
The Dockerfile's `BASE_IMAGE` pin applies only to Docker builds. The
`vllm-omni` package does not install vLLM as a dependency, so non-Docker source
@@ -49,13 +50,14 @@ uv pip install onnxruntime-rocm
#### Installation of vLLM
If you do not need to modify source code of vLLM, you can directly install the stable 0.30.0 release version of the library
If you do not need to modify source code of vLLM, install the final 0.31.0
release wheel for ROCm 7.2.3:
```bash
uv pip install vllm==0.30.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.30.0/rocm723
uv pip install vllm==0.31.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.31.0/rocm723
```
The pre-built 0.30.0 vLLM wheel targets ROCm 7.2.3. If you need a different ROCm stack or want to reuse an existing PyTorch installation, build vLLM from source instead.
The pre-built 0.31.0 vLLM wheel targets ROCm 7.2.3. If you need a different ROCm stack or want to reuse an existing PyTorch installation, build vLLM from source instead.
#### Installation of vLLM-Omni
@@ -75,7 +77,7 @@ If you want to check, modify or debug with source code of vLLM, install the libr
```bash
git clone https://github.com/vllm-project/vllm.git
cd vllm
git checkout v0.30.0
git checkout v0.31.0
python3 -m pip install -r requirements/rocm.txt
python3 setup.py develop
```
@@ -87,7 +89,7 @@ python3 setup.py develop
#### Build docker image
The source build defaults to the published upstream base image
`vllm/vllm-openai-rocm:v0.30.0`, aligned with the vLLM release used by CI.
`vllm/vllm-openai-rocm:v0.31.0`, aligned with this source checkout.
Older or custom bases must provide the vLLM APIs checked by the Dockerfile's
image-build canary. This upstream base is distinct from the prebuilt
`vllm/vllm-omni-rocm` images discussed below; their published-tag availability
@@ -102,7 +104,7 @@ To select the upstream ROCm base explicitly, pass `BASE_IMAGE`:
```bash
DOCKER_BUILDKIT=1 docker build \
-f docker/Dockerfile.rocm \
--build-arg BASE_IMAGE=vllm/vllm-openai-rocm:v0.30.0 \
--build-arg BASE_IMAGE=vllm/vllm-openai-rocm:v0.31.0 \
-t vllm-omni-rocm .
```
docs/getting_started/installation/gpu/xpu.inc.md
+7 -3
View File
@@ -20,12 +20,16 @@ vLLM-Omni currently recommends using the Docker image setup steps below.
#### Build docker image
```bash
DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.xpu -t vllm-omni-xpu --shm-size=4g .
export VLLM_VERSION=v0.31.0
DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.xpu -t vllm-omni-xpu --shm-size=4g \
--build-arg VLLM_VERSION="$VLLM_VERSION" .
```
This layers vLLM-Omni on top of the published `vllm/vllm-openai-xpu:<VLLM_VERSION>`
base image (v0.30.0 by default), which Docker pulls automatically. To target a different vLLM release,
pass `--build-arg VLLM_VERSION=<tag>`. If that tag has not been published yet, build
base image, which Docker pulls automatically. The Dockerfile still defaults to
v0.30.0; the command above explicitly selects v0.31.0 to match this source
checkout. To target a different vLLM release, pass
`--build-arg VLLM_VERSION=<tag>`. If that tag has not been published yet, build
the base from upstream's own Dockerfile first and point the build at it:
```bash
docs/getting_started/installation/npu/npu.inc.md
+6 -2
View File
@@ -71,10 +71,14 @@ We are keeping [issue #886](https://github.com/vllm-project/vllm-omni/issues/886
# --8<-- [start:installation-main]
You can also install vLLM-Omni from the latest main branch if you want to use the latest features or bug fixes. Use the aligned vLLM-Ascend v0.30.0 image described above, and check [issue #886](https://github.com/vllm-project/vllm-omni/issues/886) for the status of the latest vLLM-Omni commit on NPU.
You can also install vLLM-Omni from source if you want to use the latest features
or bug fixes. This source checkout targets vLLM 0.31.0 and needs a matching
vLLM-Ascend runtime. The v0.30.0 images above are for the published Omni v0.30.0
release. Check [issue #886](https://github.com/vllm-project/vllm-omni/issues/886)
for aligned versions and NPU validation before selecting a development image.
```bash
# Inside the vLLM-Ascend v0.30.0 container, install vLLM-Omni from main
# Inside a container with matching vLLM and vLLM-Ascend, install Omni from source
cd /vllm-workspace
git clone https://github.com/vllm-project/vllm-omni.git
cd vllm-omni
docs/getting_started/quickstart.md
+6 -3
View File
@@ -19,17 +19,20 @@ uv venv --python 3.12 --seed
source .venv/bin/activate
# On CUDA
uv pip install vllm==0.30.0 --torch-backend=auto
uv pip install vllm==0.31.0 --torch-backend=auto \
--extra-index-url https://wheels.vllm.ai/db9527a46873454610df6dbedf79a36d6bf1a7f6
# On ROCm
uv pip install vllm==0.30.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.30.0/rocm723
uv pip install vllm==0.31.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.31.0/rocm723
git clone https://github.com/vllm-project/vllm-omni.git
cd vllm-omni
uv pip install -e .
```
For additional installation methods — please see the [installation guide](installation/README.md).
These commands install vLLM 0.31.0 for this source checkout. The CUDA index is
pinned to the final release commit. For published vLLM-Omni wheels and additional
installation methods, see the [installation guide](installation/README.md).
!!! note
It is important to install the same major & minor version of vLLM and vLLM Omni, otherwise things may not work as expected. If the versions are misaligned, you will see a warning when you import vLLM Omni.