Signed-off-by: Roger Wang <hey@rogerw.io>
This commit is contained in:
Roger Wang committed 2026-03-01 05:16:03 +00:00
1 parent 3d9fa8d53f
commit 171826dd8b
4 files changed
+6 -5

No files matched your search

@@ -40,7 +40,7 @@ If you do not need to modify source code of vLLM, you can directly install the s
uv pip install vllm==0.16.0 --torch-backend=auto
```
The release 0.14.0 of vLLM is based on PyTorch 2.9.0 which requires CUDA 12.9 environment.
The release 0.16.0 of vLLM is based on PyTorch 2.9.0 which requires CUDA 12.9 environment.
#### Installation of vLLM-Omni
Since vllm-omni is rapidly evolving, it's recommended to install it from source
@@ -91,7 +91,7 @@ docker run --runtime nvidia --gpus 2 \
--env "HF_TOKEN=$HF_TOKEN" \
-p 8091:8091 \
--ipc=host \
vllm/vllm-omni:v0.14.0 \
vllm/vllm-omni:v0.16.0 \
--model Qwen/Qwen3-Omni-30B-A3B-Instruct --port 8091
```
@@ -130,7 +130,7 @@ docker run --rm \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=$HF_TOKEN" \
-p 8091:8091 \
vllm/vllm-omni-rocm:v0.14.0 \
vllm/vllm-omni-rocm:v0.16.0 \
--model Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091
```
@@ -149,7 +149,7 @@ docker run --rm -it \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=$HF_TOKEN" \
--entrypoint bash \
vllm/vllm-omni-rocm:v0.14.0
vllm/vllm-omni-rocm:v0.16.0
```
# --8<-- [end:pre-built-images]