[Doc] Update README and docs for v0.30.0 (#8175)

Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
This commit is contained in:
Hongsheng Liuandhsliu_ustc authored and GitHub committed 2026-09-27 10:07:58 +08:00
1 parent 6715c57b2d
commit 7ab582ae25
9 files changed
+25 -25

No files matched your search

+4 -3
View File
@@ -16,6 +16,7 @@ Easy, fast, and cheap omni-modality model serving for everyone
*Latest News* 🔥 *Latest News* 🔥
- [2026/09] We released [0.30.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.30.0), rebased onto vLLM 0.30.0, featuring a unified full-duplex serving framework around engine-owned sessions for [MiniCPM-o 4.5](recipes/OpenBMB/MiniCPM-o-4_5.md) and AURA, native cross-stage KV and multimodal payload transfer (Mooncake AR-to-DiT handoff and NIXL connectors), interactive world-model serving with [LingBot World](recipes/Robbyant/LingBot-World-2.0.md), and realtime [MiniMax H3](recipes/MiniMaxAI/MiniMax-H3.md) Turbo inference on NVIDIA Blackwell and Ascend 950.
- [2026/08] We released [0.28.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.28.0), featuring production-ready [MiniMax H3](recipes/MiniMaxAI/MiniMax-H3.md) serving on GPU and NPU, a unified AR/DiT paged KV cache runtime, and enhanced realtime full-duplex serving for the [MiniCPM-o series](recipes/OpenBMB/MiniCPM-o-4_5.md). - [2026/08] We released [0.28.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.28.0), featuring production-ready [MiniMax H3](recipes/MiniMaxAI/MiniMax-H3.md) serving on GPU and NPU, a unified AR/DiT paged KV cache runtime, and enhanced realtime full-duplex serving for the [MiniCPM-o series](recipes/OpenBMB/MiniCPM-o-4_5.md).
- [2026/08] [VeRL-Omni](https://github.com/verl-project/verl-omni) `v0.2.0` is released: faster diffusion RL powered by vLLM-Omni (request-level/step-wise batching with FA3), rebuilt Qwen3-Omni multimodal training (DPO & GSPO), plus LTX-2.3, Qwen-Image-Edit support and more. See the [release notes](https://github.com/verl-project/verl-omni/releases/tag/v0.2.0). - [2026/08] [VeRL-Omni](https://github.com/verl-project/verl-omni) `v0.2.0` is released: faster diffusion RL powered by vLLM-Omni (request-level/step-wise batching with FA3), rebuilt Qwen3-Omni multimodal training (DPO & GSPO), plus LTX-2.3, Qwen-Image-Edit support and more. See the [release notes](https://github.com/verl-project/verl-omni/releases/tag/v0.2.0).
- [2026/08] We released [0.26.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.26.0) - aligned with the vLLM 0.26 release line, featuring [MiniMax H3](recipes/MiniMaxAI/MiniMax-H3.md) joint video/audio generation, an experimental full-duplex realtime runtime for [MiniCPM-o 4.5](recipes/OpenBMB/MiniCPM-o-4_5.md), distributed layerwise diffusion offload, and broader model, hardware, streaming, TTS, and quantization support. - [2026/08] We released [0.26.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.26.0) - aligned with the vLLM 0.26 release line, featuring [MiniMax H3](recipes/MiniMaxAI/MiniMax-H3.md) joint video/audio generation, an experimental full-duplex realtime runtime for [MiniCPM-o 4.5](recipes/OpenBMB/MiniCPM-o-4_5.md), distributed layerwise diffusion offload, and broader model, hardware, streaming, TTS, and quantization support.
@@ -58,9 +59,9 @@ vLLM-Omni is flexible and easy to use with:
vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, including: vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, including:
- **Omni-modality models** (e.g. Qwen3-Omni, MiniCPM-o 4.5, Cosmos3, HunyuanImage, BAGEL) - **Omni-modality models** (e.g. Qwen3-Omni, MiniCPM-o 4.5, Cosmos3, HunyuanImage, BAGEL)
- **TTS models** (e.g. Qwen3-TTS, IndexTTS 2.5, CosyVoice3) - **TTS models** (e.g. Qwen3-TTS, Tencent AuK, Breeze-TTS-2, CosyVoice3)
- **Diffusion models** — image, video, and audio generation (e.g. MiniMax H3, LTX-2.5, SANA-Video, Wan2.2) - **Diffusion models** — image, video, and audio generation (e.g. MiniMax H3, LingBot World, MAGI-2, LTX-2.5, Wan2.2)
- **Robot-policy and action models** (e.g. π0, GR00T-N1.7, DreamZero-DROID, InternVLA-A1) - **Robot-policy and action models** (e.g. π0.5, GR00T-N1.7, DreamZero-DROID, InternVLA-A1)
## Getting Started ## Getting Started
+3 -3
View File
@@ -56,9 +56,9 @@ vLLM-Omni is flexible and easy to use with:
vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, including: vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, including:
- Omni-modality models (e.g. Qwen3-Omni, MiniCPM-o 4.5, Cosmos3, HunyuanImage, BAGEL) - Omni-modality models (e.g. Qwen3-Omni, MiniCPM-o 4.5, Cosmos3, HunyuanImage, BAGEL)
- TTS models (e.g. Qwen3-TTS, IndexTTS 2.5, CosyVoice3) - TTS models (e.g. Qwen3-TTS, Tencent AuK, Breeze-TTS-2, CosyVoice3)
- Diffusion models — image, video, and audio generation (e.g. MiniMax H3, LTX-2.5, SANA-Video, Wan2.2) - Diffusion models — image, video, and audio generation (e.g. MiniMax H3, LingBot World, MAGI-2, LTX-2.5, Wan2.2)
- Robot-policy and action models (e.g. π0, GR00T-N1.7, DreamZero-DROID, InternVLA-A1) - Robot-policy and action models (e.g. π0.5, GR00T-N1.7, DreamZero-DROID, InternVLA-A1)
For more information, checkout the following: For more information, checkout the following:
+1 -1
View File
@@ -2,7 +2,7 @@
This section lists the most common options for running vLLM-Omni. This section lists the most common options for running vLLM-Omni.
For options within a vLLM Engine, please refer to the [vLLM 0.29 configuration guide](https://docs.vllm.ai/en/v0.29.0/configuration/index.html). For options within a vLLM Engine, please refer to the [vLLM 0.30 configuration guide](https://docs.vllm.ai/en/v0.30.0/configuration/index.html).
Each model defines fixed topology in a registered `PipelineConfig` and runtime Each model defines fixed topology in a registered `PipelineConfig` and runtime
overrides in a deploy YAML. overrides in a deploy YAML.
+1 -1
View File
@@ -178,7 +178,7 @@ their keys only.
## Inherited vLLM variables ## Inherited vLLM variables
vLLM-Omni also reads variables through its aligned vLLM dependency. Refer to vLLM-Omni also reads variables through its aligned vLLM dependency. Refer to
the [vLLM 0.29 environment-variable reference](https://docs.vllm.ai/en/v0.29.0/configuration/env_vars.html) the [vLLM 0.30 environment-variable reference](https://docs.vllm.ai/en/v0.30.0/configuration/env_vars.html)
for their definitions. This includes vLLM launch, cache, logging, plugin, ROCm, for their definitions. This includes vLLM launch, cache, logging, plugin, ROCm,
XPU, ModelScope, and FlashInfer workspace settings. XPU, ModelScope, and FlashInfer workspace settings.
+1 -1
View File
@@ -1,7 +1,7 @@
# Installation # Installation
!!! important !!! important
vLLM-Omni is released against the matching upstream vLLM major/minor version. The 0.30 development line requires vLLM 0.30.x; the source-install instructions pin vLLM 0.30.0. Published vLLM-Omni 0.28.0 wheels and images still require vLLM 0.28.x. vLLM-Omni is released against the matching upstream vLLM major/minor version. The vLLM-Omni 0.30.x release line uses vLLM 0.30.x; the source-install instructions pin vLLM 0.30.0.
vLLM-Omni supports the following hardware platforms: vLLM-Omni supports the following hardware platforms:
+1 -3
View File
@@ -34,9 +34,7 @@ vLLM-Omni is a Python library that supports the following GPU variants. The libr
### Pre-built wheels ### Pre-built wheels
The source-install instructions target vLLM 0.30.0. Until vLLM-Omni 0.30.0 artifacts are published, the CUDA/ROCm pre-built examples below remain on the matching 0.28.0 release pair. Note: Pre-built wheels are currently available for vLLM-Omni 0.11.0rc1, 0.12.0rc1, 0.14.0rc1, 0.14.0, 0.16.0, 0.18.0, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.25.0, 0.26.0, 0.28.0, and 0.30.0. If you need a newer unreleased revision, please [build from source](https://docs.vllm.ai/projects/vllm-omni/en/latest/getting_started/installation/gpu/#build-wheel-from-source).
Note: Pre-built wheels are currently available for vLLM-Omni 0.11.0rc1, 0.12.0rc1, 0.14.0rc1, 0.14.0, 0.16.0, 0.18.0, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.25.0, 0.26.0, and 0.28.0. If you need a newer unreleased revision, please [build from source](https://docs.vllm.ai/projects/vllm-omni/en/latest/getting_started/installation/gpu/#build-wheel-from-source).
=== "NVIDIA CUDA" === "NVIDIA CUDA"
@@ -5,7 +5,7 @@
# --8<-- [end:requirements] # --8<-- [end:requirements]
# --8<-- [start:set-up-using-python] # --8<-- [start:set-up-using-python]
vLLM-Omni depends on the matching major/minor release of vLLM. The 0.30 development line uses vLLM 0.30.x. Published 0.28.0 wheels and images use vLLM 0.28.x. vLLM-Omni depends on the matching major/minor release of vLLM. The vLLM-Omni 0.30.x release line uses vLLM 0.30.x.
!!! note !!! note
PyTorch installed via `conda` will statically link `NCCL` library, which can cause issues when vLLM tries to use `NCCL`. See <gh-issue:8420> for more details. PyTorch installed via `conda` will statically link `NCCL` library, which can cause issues when vLLM tries to use `NCCL`. See <gh-issue:8420> for more details.
@@ -18,22 +18,22 @@ Therefore, it is recommended to install vLLM and vLLM-Omni with a **fresh new**
#### Installation of vLLM #### Installation of vLLM
These pre-built wheel instructions install the published vLLM-Omni 0.28.0 release. For the 0.30 development line, use the source-install instructions below. These pre-built wheel instructions install the published vLLM-Omni 0.30.0 release.
vLLM-Omni is built based on vLLM. Please install it with command below. vLLM-Omni is built based on vLLM. Please install it with command below.
```bash ```bash
uv pip install vllm==0.28.0 --torch-backend=auto uv pip install vllm==0.30.0 --torch-backend=auto
``` ```
#### Installation of vLLM-Omni #### Installation of vLLM-Omni
```bash ```bash
uv pip install vllm-omni==0.28.0 uv pip install vllm-omni==0.30.0
``` ```
To run Gradio demos, also install the optional extras: To run Gradio demos, also install the optional extras:
```bash ```bash
uv pip install 'vllm-omni[demo]==0.28.0' uv pip install 'vllm-omni[demo]==0.30.0'
``` ```
# --8<-- [end:pre-built-wheels] # --8<-- [end:pre-built-wheels]
@@ -111,7 +111,7 @@ docker run --runtime nvidia --gpus 2 \
--env "HF_TOKEN=$HF_TOKEN" \ --env "HF_TOKEN=$HF_TOKEN" \
-p 8091:8091 \ -p 8091:8091 \
--ipc=host \ --ipc=host \
vllm/vllm-omni:v0.28.0 \ vllm/vllm-omni:v0.30.0 \
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091
``` ```
@@ -10,9 +10,8 @@
For ROCm, vLLM-Omni currently recommends the setup steps through Docker Images. For ROCm, vLLM-Omni currently recommends the setup steps through Docker Images.
vLLM-Omni depends on the matching major/minor release of vLLM. The 0.30 vLLM-Omni depends on the matching major/minor release of vLLM. The vLLM-Omni
development line uses vLLM 0.30.x. Published 0.28.0 wheels and images use vLLM 0.30.x release line uses vLLM 0.30.x.
0.28.x.
The Dockerfile's `BASE_IMAGE` pin applies only to Docker builds. The The Dockerfile's `BASE_IMAGE` pin applies only to Docker builds. The
`vllm-omni` package does not install vLLM as a dependency, so non-Docker source `vllm-omni` package does not install vLLM as a dependency, so non-Docker source
@@ -23,12 +22,12 @@ installing vLLM-Omni, as shown below.
#### Installation of vLLM #### Installation of vLLM
These pre-built wheel instructions install the published vLLM-Omni 0.28.0 release. For the 0.30 development line, use the source-install instructions below. These pre-built wheel instructions install the published vLLM-Omni 0.30.0 release.
vLLM-Omni is built based on vLLM. Please install it with command below. vLLM-Omni is built based on vLLM. Please install it with command below.
```bash ```bash
uv pip install vllm==0.28.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.28.0/rocm723 uv pip install vllm==0.30.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.30.0/rocm723
``` ```
#### Installation of vLLM-Omni #### Installation of vLLM-Omni
@@ -37,7 +36,7 @@ uv pip install vllm==0.28.0+rocm723 --extra-index-url https://wheels.vllm.ai/roc
# we need to add --no-build-isolation as the torch # we need to add --no-build-isolation as the torch
# is not obtained from pypi, we have to install using the # is not obtained from pypi, we have to install using the
# torch installed in our environment # torch installed in our environment
uv pip install vllm-omni==0.28.0 uv pip install vllm-omni==0.30.0
# Optional if want to run Qwen3 TTS # Optional if want to run Qwen3 TTS
uv pip uninstall onnxruntime # should be removed before we can install onnxruntime-rocm uv pip uninstall onnxruntime # should be removed before we can install onnxruntime-rocm
@@ -150,6 +149,8 @@ vllm-omni-rocm
vLLM-Omni offers an official docker image for deployment. These images are built on top of vLLM docker images and available on Docker Hub as [vllm/vllm-omni-rocm](https://hub.docker.com/r/vllm/vllm-omni-rocm/tags). The version of vLLM-Omni indicates which release of vLLM it is based on. vLLM-Omni offers an official docker image for deployment. These images are built on top of vLLM docker images and available on Docker Hub as [vllm/vllm-omni-rocm](https://hub.docker.com/r/vllm/vllm-omni-rocm/tags). The version of vLLM-Omni indicates which release of vLLM it is based on.
The prebuilt ROCm image is published separately from the release pipeline: `v0.28.0` is the latest tag currently available on Docker Hub, and newer-tag availability is tracked in [#7405](https://github.com/vllm-project/vllm-omni/issues/7405).
#### Launch vLLM-Omni Server #### Launch vLLM-Omni Server
Here's an example deployment command that has been verified on 2 x MI300's: Here's an example deployment command that has been verified on 2 x MI300's:
@@ -53,7 +53,7 @@ docker run --rm \
# Inside the container, install vLLM-Omni from source # Inside the container, install vLLM-Omni from source
cd /vllm-workspace cd /vllm-workspace
git clone -b v0.30.0rc1 https://github.com/vllm-project/vllm-omni.git git clone -b v0.30.0 https://github.com/vllm-project/vllm-omni.git
cd vllm-omni cd vllm-omni
pip install -v -e . --no-build-isolation pip install -v -e . --no-build-isolation
# or VLLM_OMNI_TARGET_DEVICE=npu pip install -v -e . # or VLLM_OMNI_TARGET_DEVICE=npu pip install -v -e .