[Doc] Add installation part for pre built docker. (#141)

Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
Signed-off-by: Alicia Wang <115451386+congw729@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
This commit is contained in:
AliciaandRoger Wang authored and GitHub committed 2025-12-04 00:04:18 -08:00
1 parent 863cdb82b6
commit c0b88a87ba
2 files changed
+25 -1

No files matched your search

+4 -1
View File
@@ -37,4 +37,7 @@ vLLM-Omni is a Python library that supports the following GPU variants. The libr
## Set up using Docker
### Pre-built images
To be released... Stay tuned!
=== "NVIDIA CUDA"
--8<-- "docs/getting_started/installation/gpu/cuda.inc.md:pre-built-images"
@@ -73,3 +73,24 @@ uv pip install --no-build-isolation --editable .
</details>
# --8<-- [end:build-wheel-from-source]
# --8<-- [start:pre-built-images]
vLLM-Omni offers an official docker image for deployment. These images are built on top of vLLM docker images and available on Docker Hub as [vllm/vllm-omni](https://hub.docker.com/r/vllm/vllm-omni/tags). The version of vLLM-Omni indicates which release of vLLM it is based on.
Here's an example deployment command that has been verified on 2 x H100's:
```bash
docker run --runtime nvidia --gpus 2 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=$HF_TOKEN" \
-p 8091:8091 \
--ipc=host \
vllm/vllm-omni:v0.11.0rc1 \
--model Qwen/Qwen3-Omni-30B-A3B-Instruct --port 8091
```
!!! tip
You can use this docker image to serve models the same way you would with in vLLM! To do so, make sure you overwrite the default entrypoint (`vllm serve --omni`) which works only for models supported in the vLLM-Omni project.
# --8<-- [end:pre-built-images]