[Rebase] Rebase to vllm 0.12.0 (#335)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com> Signed-off-by: tzhouam <tzhouam@connect.ust.hk> Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> Co-authored-by: Rein Yang <ruiruyang2@gmail.com> Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
This commit is contained in:
48 files changed
+1649
-2473
No files matched your search
@@ -18,9 +18,9 @@ Therefore, it is recommended to install vLLM and vLLM-Omni with a **fresh new**
|
||||
|
||||
#### Installation of vLLM
|
||||
|
||||
vLLM-Omni is built based on vLLM v0.11.0. Please install it with command below.
|
||||
vLLM-Omni is built based on vLLM v0.12.0. Please install it with command below.
|
||||
```bash
|
||||
uv pip install vllm==0.11.0 --torch-backend=auto
|
||||
uv pip install vllm==0.12.0 --torch-backend=auto
|
||||
```
|
||||
|
||||
#### Installation of vLLM-Omni
|
||||
@@ -34,10 +34,10 @@ uv pip install vllm-omni
|
||||
# --8<-- [start:build-wheel-from-source]
|
||||
|
||||
#### Installation of vLLM
|
||||
If you do not need to modify source code of vLLM, you can directly install the stable 0.11.0 release version of the library
|
||||
If you do not need to modify source code of vLLM, you can directly install the stable 0.12.0 release version of the library
|
||||
|
||||
```bash
|
||||
uv pip install vllm==0.11.0 --torch-backend=auto
|
||||
uv pip install vllm==0.12.0 --torch-backend=auto
|
||||
```
|
||||
|
||||
#### Installation of vLLM-Omni
|
||||
@@ -54,11 +54,11 @@ If you want to check, modify or debug with source code of vLLM, install the libr
|
||||
```bash
|
||||
git clone https://github.com/vllm-project/vllm.git
|
||||
cd vllm
|
||||
git checkout v0.11.0
|
||||
git checkout v0.12.0
|
||||
```
|
||||
Set up environment variables to get pre-built wheels. If there are internet problems, just download the whl file manually. And set `VLLM_PRECOMPILED_WHEEL_LOCATION` as your local absolute path of whl file.
|
||||
```bash
|
||||
export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.11.0/vllm-0.11.0-cp38-abi3-manylinux1_x86_64.whl
|
||||
export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.12.0/vllm-0.12.0-cp38-abi3-manylinux_2_31_x86_64.whl
|
||||
```
|
||||
Install vllm with command below (If you have no existing PyTorch).
|
||||
```bash
|
||||
@@ -89,7 +89,7 @@ docker run --runtime nvidia --gpus 2 \
|
||||
--env "HF_TOKEN=$HF_TOKEN" \
|
||||
-p 8091:8091 \
|
||||
--ipc=host \
|
||||
vllm/vllm-omni:v0.11.0rc1 \
|
||||
vllm/vllm-omni:v0.12.0rc1 \
|
||||
--model Qwen/Qwen3-Omni-30B-A3B-Instruct --port 8091
|
||||
```
|
||||
|
||||
|
||||
@@ -19,7 +19,7 @@ vLLM-Omni current recommends the steps in under setup through Docker Images.
|
||||
|
||||
#### Get the correct docker environment
|
||||
|
||||
The v0.11.0 releases is on Oct 3, 2025, so we will pull a AMD nightly docker image close to the release period. `rocm/vllm-dev:nightly_main_20251005` is picked as it has a correct `transformers` version that supports qwen3_omni.
|
||||
The v0.12.0 releases is on Dec 3, 2025, so we will pull a AMD nightly docker image close to the release period. `rocm/vllm-dev:nightly_main_20251205` is picked as it has a correct `transformers` version that supports qwen3_omni.
|
||||
|
||||
`docker pull rocm/vllm-dev:nightly_main_20251005`
|
||||
|
||||
@@ -51,18 +51,18 @@ sudo apt install ffmpeg -y
|
||||
```
|
||||
|
||||
#### Installation of vLLM
|
||||
Install the v0.11.0 release of vLLM as vLLM-Omni is currently depend on v0.11.0.
|
||||
Install the v0.12.0 release of vLLM as vLLM-Omni is currently depend on v0.12.0.
|
||||
|
||||
```bash
|
||||
# uninstall existing vllm which is based on the main branch, not on the releases branch
|
||||
python3 -m pip uninstall -y vllm
|
||||
git clone https://github.com/vllm-project/vllm.git vllm0110
|
||||
cd vllm0110/
|
||||
git clone https://github.com/vllm-project/vllm.git vllm0120
|
||||
cd vllm0120/
|
||||
|
||||
# checkout the version tag v0.11.0
|
||||
git checkout -b v0.11.0 v0.11.0
|
||||
# checkout the version tag v0.12.0
|
||||
git checkout -b v0.12.0 v0.12.0
|
||||
|
||||
# you should see 0.11.0
|
||||
# you should see 0.12.0
|
||||
python -c "import setuptools_scm; print(setuptools_scm.get_version())"
|
||||
|
||||
# PYTORCH_ROCM_ARCH=<your-gpu-arch> python3 setup.py develop
|
||||
@@ -92,7 +92,7 @@ pip install vllm-omni
|
||||
# clone and checkout to the release version.
|
||||
git clone https://github.com/vllm-project/vllm-omni.git
|
||||
cd vllm-omni
|
||||
git checkout v0.11.0rc1
|
||||
git checkout v0.12.0rc1
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
|
||||
Reference in new issue
Block a user