Compare commits

..
Author SHA1 Message Date
Zhou Taichang d6f93b0f37 [Debug] Correct Unreasonable Long Timeout (#1175)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-03 08:49:51 +00:00
Zhou Taichang 7fcbaa794c [Debug] Clear Dockerfile.ci to accelerate build image (#1172)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-03 13:43:45 +08:00
Hongsheng Liuandhsliu_ustc 3fb3c8aba4 Docs update paper link (#1169)
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
2026-02-03 13:19:36 +08:00
5 changed files with 27 additions and 14 deletions

No files matched your search

+15 -1
View File
@@ -15,7 +15,8 @@ Easy, fast, and cheap omni-modality model serving for everyone
---
*Latest News* 🔥
- [2026/02] We released [0.14.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.14.0) - This is the first **stable release** of vLLM-Omni that expands Omni’s diffusion / image-video generation and audio / TTS stack, improves distributed execution and memory efficiency, and broadens platform/backend coverage (GPU/ROCm/NPU/XPU). It also brings meaningful upgrades to serving APIs, profiling & benchmarking, and overall stability.
- [2026/02] We released [0.14.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.14.0) - This is the first **stable release** of vLLM-Omni that expands Omni’s diffusion / image-video generation and audio / TTS stack, improves distributed execution and memory efficiency, and broadens platform/backend coverage (GPU/ROCm/NPU/XPU). It also brings meaningful upgrades to serving APIs, profiling & benchmarking, and overall stability. Please check our latest [paper](https://arxiv.org/abs/2602.02204) for architecture design and performance results.
- [2026/01] We released [0.12.0rc1](https://github.com/vllm-project/vllm-omni/releases/tag/v0.12.0rc1) - a major RC milestone focused on maturing the diffusion stack, strengthening OpenAI-compatible serving, expanding omni-model coverage, and improving stability across platforms (GPU/NPU/ROCm), please check our latest [design](https://docs.google.com/presentation/d/1qv4qMW1rKAqDREMXiUDLIgqqHQe7TDPj/edit?usp=sharing&ouid=110473603432222024453&rtpof=true&sd=true).
- [2025/11] vLLM community officially released [vllm-project/vllm-omni](https://github.com/vllm-project/vllm-omni) in order to support omni-modality models serving.
@@ -67,6 +68,19 @@ Visit our [documentation](https://vllm-omni.readthedocs.io/en/latest/) to learn
We welcome and value any contributions and collaborations.
Please check out [Contributing to vLLM-Omni](https://vllm-omni.readthedocs.io/en/latest/contributing/) for how to get involved.
## Citation
If you use vLLM-Omni for your research, please cite our [paper](https://arxiv.org/abs/2602.02204):
```bibtex
@article{yin2026vllmomni,
title={vLLM-Omni: Fully Disaggregated Serving for Any-to-Any Multimodal Models},
author={Peiqi Yin, Jiangyun Zhu, Han Gao, Chenguang Zheng, Yongxiang Huang, Taichang Zhou, Ruirui Yang, Weizhi Liu, Weiqing Chen, Canlin Guo, Didan Deng, Zifeng Mo, Cong Wang, James Cheng, Roger Wang, Hongsheng Liu},
journal={arXiv preprint arXiv:2602.02204},
year={2026}
}
```
## Join the Community
Feel free to ask questions, provide feedbacks and discuss with fellow users of vLLM-Omni in `#sig-omni` slack channel at [slack.vllm.ai](https://slack.vllm.ai) or vLLM user forum at [discuss.vllm.ai](https://discuss.vllm.ai).
+8 -9
View File
@@ -3,19 +3,18 @@ ARG VLLM_BASE_TAG=v0.15.0
FROM ${VLLM_BASE_IMAGE}:${VLLM_BASE_TAG}
ARG APP_DIR=/workspace/vllm-omni
WORKDIR ${APP_DIR}
COPY . .
# Install system dependencies
RUN apt-get update && \
apt-get install -y ffmpeg && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Install vLLM wheel for the specific commit
# URL format: https://wheels.vllm.ai/<commit_hash>/
# Current vLLM commit: f176443446f659dbab5315e056e605d8984fd976 (v0.15.0)
RUN uv pip install --system --upgrade --force-reinstall vllm --extra-index-url https://wheels.vllm.ai/f176443446f659dbab5315e056e605d8984fd976
apt-get install -y ffmpeg && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Install vllm-omni into the same uv-managed Python environment used by the base image.
RUN uv pip install --system --no-cache-dir ".[dev]"
RUN uv pip install --python "$(python3 -c 'import sys; print(sys.executable)')" --no-cache-dir ".[dev]"
RUN ln -sf /usr/bin/python3 /usr/bin/python
ENTRYPOINT []
+1 -1
View File
@@ -253,7 +253,7 @@ class DiffusionEngine:
try:
# Give worker enough time — export + compression + table can be slow
results = self.collective_rpc(method="stop_profile", timeout=60000)
results = self.collective_rpc(method="stop_profile", timeout=600)
except Exception:
logger.error("Failed to stop profiling on workers", exc_info=True)
return {"traces": [], "tables": []}
+1 -1
View File
@@ -97,7 +97,7 @@ class OmniServeCommand(CLISubcommand):
omni_config_group.add_argument(
"--init-timeout",
type=int,
default=60000,
default=600,
help="The timeout for initializing the stages.",
)
omni_config_group.add_argument(
+2 -2
View File
@@ -221,8 +221,8 @@ class OmniStage:
# Wait for result from worker
try:
# Profiling stop might take time to flush files, give it 180s
response = self._out_q.get(timeout=60000)
# Profiling stop might take time to flush files, give it 600s
response = self._out_q.get(timeout=600)
if isinstance(response, dict):
if response.get("type") == "profiler_result":