diff --git a/README.md b/README.md index 4afe6e9b6..25f0c148f 100644 --- a/README.md +++ b/README.md @@ -18,9 +18,7 @@ Easy, fast, and cheap omni-modality model serving for everyone *Latest News* 🔥 - [2026/08] We released [0.26.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.26.0) - aligned with the vLLM 0.26 release line, featuring [MiniMax H3](recipes/MiniMaxAI/MiniMax-H3.md) joint video/audio generation, an experimental full-duplex realtime runtime for [MiniCPM-o 4.5](recipes/OpenBMB/MiniCPM-o-4_5.md), distributed layerwise diffusion offload, and broader model, hardware, streaming, TTS, and quantization support. - [2026/07] We released [0.24.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.24.0) - aligned with the vLLM 0.24 release line, expanding production-ready coverage across TTS, speech, diffusion, image/video generation, and robot-policy serving, with major Omni stage runtime refactoring, diffusion request-level batching, async output materialization, quantization/cache/memory improvements, and broad CUDA/ROCm/XPU/NPU support. -- [2026/06] We released [0.22.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.22.0) - an **omnimodal world-model** release aligned with vLLM 0.22, featuring [Nvidia Cosmos3](recipes/cosmos3/Cosmos3-Nano.md)/DreamZero world model support, expanded quantization coverage across Blackwell/NPU/XPU, TTS production improvements, new models including MiniCPM-o 4.5, MOSS-TTS, and Lance, plus RL integration with [VeRL-Omni](https://github.com/verl-project/verl-omni). -- [2026/05] We released [0.20.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.20.0) - refreshes the serving/runtime stack for large-scale omni workloads, and improves diffusion model performance, quantization, and hardware readiness across CUDA, ROCm, MUSA, NPU, and XPU backends. -- [2026/03] Starting with [0.14.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.14.0), vLLM-Omni publishes a stable release for every even-numbered upstream vLLM minor version, aligned with the corresponding vLLM release. [0.16.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.16.0) and [0.18.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.18.0) continued this cadence with broader omni, TTS, diffusion, distributed execution, performance, hardware, and production-readiness improvements. +- [2026/06] Starting with [0.14.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.14.0), vLLM-Omni publishes a stable release aligned with every even-numbered upstream vLLM minor version. [0.16.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.16.0), [0.18.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.18.0), [0.20.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.20.0), and [0.22.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.22.0) continued this cadence, expanding omni and world-model support with [NVIDIA Cosmos3](recipes/cosmos3/Cosmos3-Nano.md) and DreamZero, adding models such as MiniCPM-o 4.5, MOSS-TTS, and Lance, and advancing TTS, diffusion, distributed execution, quantization, RL integration through [VeRL-Omni](https://github.com/verl-project/verl-omni), and CUDA/ROCm/MUSA/NPU/XPU coverage. - [2026/03] Check out our first public [project deepdive](https://youtu.be/sgwNfsNnR9I) at the vLLM Hong Kong Meetup! - [2026/03] **[vllm-omni-skills](https://github.com/hsliuustc0106/vllm-omni-skills)** is a community-driven collection of AI assistant skills that help developers work with vLLM-Omni more effectively. These skills can be used with popular agentic AI coding assistants like **Cursor IDE**, **Claude**, **Codex**, and more. - [2025/11] vLLM community officially released [vllm-project/vllm-omni](https://github.com/vllm-project/vllm-omni) in order to support omni-modality models serving. @@ -53,8 +51,8 @@ vLLM-Omni is flexible and easy to use with: - Seamless integration with popular Hugging Face models - Tensor, pipeline, data and expert parallelism support for distributed inference - Streaming outputs -- **Experimental:** Full-duplex realtime serving with streaming audio input and output for MiniCPM-o 4.5 - OpenAI-compatible API server +- **Experimental:** Full-duplex realtime serving with streaming audio input and output vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, including: diff --git a/docs/README.md b/docs/README.md index 6715c591d..fc787b635 100644 --- a/docs/README.md +++ b/docs/README.md @@ -51,8 +51,8 @@ vLLM-Omni is flexible and easy to use with: - Seamless integration with popular Hugging Face models - Tensor, pipeline, data and expert parallelism support for distributed inference - Streaming outputs -- **Experimental:** Full-duplex realtime serving with streaming audio input and output for MiniCPM-o 4.5 - OpenAI-compatible API server +- **Experimental:** Full-duplex realtime serving with streaming audio input and output vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, including: diff --git a/docs/configuration/README.md b/docs/configuration/README.md index 390176e9c..09772ccd1 100644 --- a/docs/configuration/README.md +++ b/docs/configuration/README.md @@ -2,7 +2,7 @@ This section lists the most common options for running vLLM-Omni. -For options within a vLLM Engine. Please refer to [vLLM Configuration](https://docs.vllm.ai/en/v0.16.0/configuration/index.html) +For options within a vLLM Engine, please refer to the [vLLM 0.26 configuration guide](https://docs.vllm.ai/en/v0.26.0/configuration/index.html). Currently, the main options are maintained by stage configs for each model. diff --git a/docs/getting_started/installation/README.md b/docs/getting_started/installation/README.md index 89562c53c..b49cee794 100644 --- a/docs/getting_started/installation/README.md +++ b/docs/getting_started/installation/README.md @@ -1,5 +1,8 @@ # Installation +!!! important + vLLM-Omni is released against the matching upstream vLLM major/minor version. vLLM-Omni 0.26.x requires vLLM 0.26.x; the stable v0.26.0 instructions pin vLLM 0.26.0. + vLLM-Omni supports the following hardware platforms: - [GPU](gpu.md) diff --git a/docs/getting_started/installation/gpu/cuda.inc.md b/docs/getting_started/installation/gpu/cuda.inc.md index 9d568bde3..a68bfdb58 100644 --- a/docs/getting_started/installation/gpu/cuda.inc.md +++ b/docs/getting_started/installation/gpu/cuda.inc.md @@ -5,7 +5,7 @@ # --8<-- [end:requirements] # --8<-- [start:set-up-using-python] -vLLM-Omni depends vLLM. So please follow instructions below mainly for vLLM. +vLLM-Omni depends on the matching major/minor release of vLLM. The vLLM-Omni 0.26.x release line uses vLLM 0.26.x. !!! note PyTorch installed via `conda` will statically link `NCCL` library, which can cause issues when vLLM tries to use `NCCL`. See for more details. @@ -124,7 +124,7 @@ If you want to specify the base vLLM version: ```bash DOCKER_BUILDKIT=1 docker build \ -f docker/Dockerfile.cuda \ - --build-arg BASE_IMAGE=vllm/vllm-openai:v0.22.1 \ + --build-arg BASE_IMAGE=vllm/vllm-openai:v0.26.0 \ -t vllm-omni-cuda . ``` diff --git a/docs/getting_started/installation/gpu/musa.inc.md b/docs/getting_started/installation/gpu/musa.inc.md index a7cbc848f..ae9ba25a1 100644 --- a/docs/getting_started/installation/gpu/musa.inc.md +++ b/docs/getting_started/installation/gpu/musa.inc.md @@ -28,7 +28,7 @@ vLLM-Omni for MUSA requires building from source. Pre-built wheels are not curre ```bash git clone https://github.com/MooreThreads/vllm-musa.git cd vllm-musa -git checkout v0.18.0-dev +git checkout v0.26.0-dev pip install . --no-build-isolation -v ``` diff --git a/docs/getting_started/installation/npu/npu.inc.md b/docs/getting_started/installation/npu/npu.inc.md index b24487f3f..a0d4c86b0 100644 --- a/docs/getting_started/installation/npu/npu.inc.md +++ b/docs/getting_started/installation/npu/npu.inc.md @@ -53,16 +53,20 @@ We are keeping [issue #886](https://github.com/vllm-project/vllm-omni/issues/886 You can also build vLLM-Omni from the latest main branch if you want to use the latest features or bug fixes. (But sometimes it will break for a while. You can check [issue #886](https://github.com/vllm-project/vllm-omni/issues/886) for the status of the latest commit of vLLM-Omni main branch on NPU.) ```bash -# Pin vLLM version to 0.18.0 -git clone -b v0.18.0 https://github.com/vllm-project/vllm.git +# Pin vLLM and vLLM-Ascend to the v0.26 release line +git clone -b v0.26.0 https://github.com/vllm-project/vllm.git +cd vllm VLLM_TARGET_DEVICE=empty pip install -v -e . +cd .. -git clone -b v0.18.0rc1 https://github.com/vllm-project/vllm-ascend.git +git clone -b releases/v0.26.0rc https://github.com/vllm-project/vllm-ascend.git +cd vllm-ascend pip install -v -e . +cd .. # Install vLLM-Omni from the latest main branch git clone https://github.com/vllm-project/vllm-omni.git -cd /vllm-workspace/vllm-omni +cd vllm-omni pip install -v -e . --no-build-isolation # or VLLM_OMNI_TARGET_DEVICE=npu pip install -v -e . export VLLM_WORKER_MULTIPROC_METHOD=spawn diff --git a/docs/getting_started/quickstart.md b/docs/getting_started/quickstart.md index 15ceb395e..9480245c4 100644 --- a/docs/getting_started/quickstart.md +++ b/docs/getting_started/quickstart.md @@ -32,9 +32,9 @@ uv pip install -e . For additional installation methods — please see the [installation guide](installation/README.md). !!! note - It is important to install the same major & minor version of vLLM and vLLM Omni, otherwise things may not work as expected. If the versions are misaligned, you will see a warning when you import vLLM Omni. + vLLM-Omni is released against the matching upstream vLLM major/minor version. vLLM-Omni 0.26.x requires vLLM 0.26.x; the commands above pin vLLM 0.26.0 for the vLLM-Omni 0.26 release line. If the versions are misaligned, you will see a warning when you import vLLM-Omni. - If you are seeing strange behavior with the `vllm` command not handling the `--omni` flag correctly, you most likely have a version mismatch with vLLM < `0.26.0` and vLLM Omni `0.26.0`, as vLLM Omni no longer hijacks the vLLM entrypoint. Updating vLLM should resolve this issue. + If you are seeing strange behavior with the `vllm` command not handling the `--omni` flag correctly, you most likely have a version mismatch with vLLM < `0.26.0` and vLLM-Omni `0.26.0`, as vLLM-Omni no longer hijacks the vLLM entrypoint. Updating vLLM should resolve this issue. ## Offline Inference