[Rebase] Rebase to vllm 0.12.0 (#335)

Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
This commit is contained in:
authored and GitHub committed 2025-12-19 16:42:02 +08:00
1 parent 9464e1479d
commit 8939eb835d
48 files changed
+1649 -2473

No files matched your search

@@ -18,9 +18,9 @@ Therefore, it is recommended to install vLLM and vLLM-Omni with a **fresh new**
#### Installation of vLLM
vLLM-Omni is built based on vLLM v0.11.0. Please install it with command below.
vLLM-Omni is built based on vLLM v0.12.0. Please install it with command below.
```bash
uv pip install vllm==0.11.0 --torch-backend=auto
uv pip install vllm==0.12.0 --torch-backend=auto
```
#### Installation of vLLM-Omni
@@ -34,10 +34,10 @@ uv pip install vllm-omni
# --8<-- [start:build-wheel-from-source]
#### Installation of vLLM
If you do not need to modify source code of vLLM, you can directly install the stable 0.11.0 release version of the library
If you do not need to modify source code of vLLM, you can directly install the stable 0.12.0 release version of the library
```bash
uv pip install vllm==0.11.0 --torch-backend=auto
uv pip install vllm==0.12.0 --torch-backend=auto
```
#### Installation of vLLM-Omni
@@ -54,11 +54,11 @@ If you want to check, modify or debug with source code of vLLM, install the libr
```bash
git clone https://github.com/vllm-project/vllm.git
cd vllm
git checkout v0.11.0
git checkout v0.12.0
```
Set up environment variables to get pre-built wheels. If there are internet problems, just download the whl file manually. And set `VLLM_PRECOMPILED_WHEEL_LOCATION` as your local absolute path of whl file.
```bash
export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.11.0/vllm-0.11.0-cp38-abi3-manylinux1_x86_64.whl
export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.12.0/vllm-0.12.0-cp38-abi3-manylinux_2_31_x86_64.whl
```
Install vllm with command below (If you have no existing PyTorch).
```bash
@@ -89,7 +89,7 @@ docker run --runtime nvidia --gpus 2 \
--env "HF_TOKEN=$HF_TOKEN" \
-p 8091:8091 \
--ipc=host \
vllm/vllm-omni:v0.11.0rc1 \
vllm/vllm-omni:v0.12.0rc1 \
--model Qwen/Qwen3-Omni-30B-A3B-Instruct --port 8091
```
@@ -19,7 +19,7 @@ vLLM-Omni current recommends the steps in under setup through Docker Images.
#### Get the correct docker environment
The v0.11.0 releases is on Oct 3, 2025, so we will pull a AMD nightly docker image close to the release period. `rocm/vllm-dev:nightly_main_20251005` is picked as it has a correct `transformers` version that supports qwen3_omni.
The v0.12.0 releases is on Dec 3, 2025, so we will pull a AMD nightly docker image close to the release period. `rocm/vllm-dev:nightly_main_20251205` is picked as it has a correct `transformers` version that supports qwen3_omni.
`docker pull rocm/vllm-dev:nightly_main_20251005`
@@ -51,18 +51,18 @@ sudo apt install ffmpeg -y
```
#### Installation of vLLM
Install the v0.11.0 release of vLLM as vLLM-Omni is currently depend on v0.11.0.
Install the v0.12.0 release of vLLM as vLLM-Omni is currently depend on v0.12.0.
```bash
# uninstall existing vllm which is based on the main branch, not on the releases branch
python3 -m pip uninstall -y vllm
git clone https://github.com/vllm-project/vllm.git vllm0110
cd vllm0110/
git clone https://github.com/vllm-project/vllm.git vllm0120
cd vllm0120/
# checkout the version tag v0.11.0
git checkout -b v0.11.0 v0.11.0
# checkout the version tag v0.12.0
git checkout -b v0.12.0 v0.12.0
# you should see 0.11.0
# you should see 0.12.0
python -c "import setuptools_scm; print(setuptools_scm.get_version())"
# PYTORCH_ROCM_ARCH=<your-gpu-arch> python3 setup.py develop
@@ -92,7 +92,7 @@ pip install vllm-omni
# clone and checkout to the release version.
git clone https://github.com/vllm-project/vllm-omni.git
cd vllm-omni
git checkout v0.11.0rc1
git checkout v0.12.0rc1
pip install -e .
```