anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Rebase] Rebase to vllm 0.12.0 (#335)

Browse Source 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
This commit is contained in:
Zhou TaichangRein YangIsotr0py
 authored and GitHub committed 2025-12-19 16:42:02 +08:00
1 parent 9464e1479d
commit 8939eb835d
48 files changed
+1649 -2473
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/getting_started/installation/gpu/cuda.inc.md
+7 -7
View File
@@ -18,9 +18,9 @@ Therefore, it is recommended to install vLLM and vLLM-Omni with a **fresh new**
#### Installation of vLLM
vLLM-Omni is built based on vLLM v0.11.0. Please install it with command below.
vLLM-Omni is built based on vLLM v0.12.0. Please install it with command below.
```bash
uv pip install vllm==0.11.0 --torch-backend=auto
uv pip install vllm==0.12.0 --torch-backend=auto
```
#### Installation of vLLM-Omni
@@ -34,10 +34,10 @@ uv pip install vllm-omni
# --8<-- [start:build-wheel-from-source]
#### Installation of vLLM
If you do not need to modify source code of vLLM, you can directly install the stable 0.11.0 release version of the library
If you do not need to modify source code of vLLM, you can directly install the stable 0.12.0 release version of the library
```bash
uv pip install vllm==0.11.0 --torch-backend=auto
uv pip install vllm==0.12.0 --torch-backend=auto
```
#### Installation of vLLM-Omni
@@ -54,11 +54,11 @@ If you want to check, modify or debug with source code of vLLM, install the libr
```bash
git clone https://github.com/vllm-project/vllm.git
cd vllm
git checkout v0.11.0
git checkout v0.12.0
```
Set up environment variables to get pre-built wheels. If there are internet problems, just download the whl file manually. And set `VLLM_PRECOMPILED_WHEEL_LOCATION` as your local absolute path of whl file.
```bash
export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.11.0/vllm-0.11.0-cp38-abi3-manylinux1_x86_64.whl
export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.12.0/vllm-0.12.0-cp38-abi3-manylinux_2_31_x86_64.whl
```
Install vllm with command below (If you have no existing PyTorch).
```bash
@@ -89,7 +89,7 @@ docker run --runtime nvidia --gpus 2 \
--env "HF_TOKEN=$HF_TOKEN" \
-p 8091:8091 \
--ipc=host \
vllm/vllm-omni:v0.11.0rc1 \
vllm/vllm-omni:v0.12.0rc1 \
--model Qwen/Qwen3-Omni-30B-A3B-Instruct --port 8091
```