[Rebase] Align vLLM-Omni with vLLM v0.31.0 (#8459)

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
This commit is contained in:
Zhou Taichang authored and GitHub committed 2026-10-05 14:11:01 +08:00
1 parent 1f47fa2f29
commit 0b0d2d7009
141 files changed
+2885 -721

No files matched your search

@@ -10,8 +10,9 @@
For ROCm, vLLM-Omni currently recommends the setup steps through Docker Images.
vLLM-Omni depends on the matching major/minor release of vLLM. The vLLM-Omni
0.30.x release line uses vLLM 0.30.x.
vLLM-Omni depends on the matching major/minor release of vLLM. This source
checkout targets vLLM 0.31.0. The pre-built Omni wheel instructions below install
the published 0.30.0 release with its matching vLLM dependency.
The Dockerfile's `BASE_IMAGE` pin applies only to Docker builds. The
`vllm-omni` package does not install vLLM as a dependency, so non-Docker source
@@ -49,13 +50,14 @@ uv pip install onnxruntime-rocm
#### Installation of vLLM
If you do not need to modify source code of vLLM, you can directly install the stable 0.30.0 release version of the library
If you do not need to modify source code of vLLM, install the final 0.31.0
release wheel for ROCm 7.2.3:
```bash
uv pip install vllm==0.30.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.30.0/rocm723
uv pip install vllm==0.31.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.31.0/rocm723
```
The pre-built 0.30.0 vLLM wheel targets ROCm 7.2.3. If you need a different ROCm stack or want to reuse an existing PyTorch installation, build vLLM from source instead.
The pre-built 0.31.0 vLLM wheel targets ROCm 7.2.3. If you need a different ROCm stack or want to reuse an existing PyTorch installation, build vLLM from source instead.
#### Installation of vLLM-Omni
@@ -75,7 +77,7 @@ If you want to check, modify or debug with source code of vLLM, install the libr
```bash
git clone https://github.com/vllm-project/vllm.git
cd vllm
git checkout v0.30.0
git checkout v0.31.0
python3 -m pip install -r requirements/rocm.txt
python3 setup.py develop
```
@@ -87,7 +89,7 @@ python3 setup.py develop
#### Build docker image
The source build defaults to the published upstream base image
`vllm/vllm-openai-rocm:v0.30.0`, aligned with the vLLM release used by CI.
`vllm/vllm-openai-rocm:v0.31.0`, aligned with this source checkout.
Older or custom bases must provide the vLLM APIs checked by the Dockerfile's
image-build canary. This upstream base is distinct from the prebuilt
`vllm/vllm-omni-rocm` images discussed below; their published-tag availability
@@ -102,7 +104,7 @@ To select the upstream ROCm base explicitly, pass `BASE_IMAGE`:
```bash
DOCKER_BUILDKIT=1 docker build \
-f docker/Dockerfile.rocm \
--build-arg BASE_IMAGE=vllm/vllm-openai-rocm:v0.30.0 \
--build-arg BASE_IMAGE=vllm/vllm-openai-rocm:v0.31.0 \
-t vllm-omni-rocm .
```