Compare commits

...
Author SHA1 Message Date
Zhou Taichang d6f93b0f37 [Debug] Correct Unreasonable Long Timeout (#1175)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-03 08:49:51 +00:00
Zhou Taichang 7fcbaa794c [Debug] Clear Dockerfile.ci to accelerate build image (#1172)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-03 13:43:45 +08:00
Hongsheng Liuandhsliu_ustc 3fb3c8aba4 Docs update paper link (#1169)
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
2026-02-03 13:19:36 +08:00
5 changed files with 27 additions and 14 deletions

No files matched your search

+15 -1
View File
@@ -15,7 +15,8 @@ Easy, fast, and cheap omni-modality model serving for everyone
--- ---
*Latest News* 🔥 *Latest News* 🔥
- [2026/02] We released [0.14.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.14.0) - This is the first **stable release** of vLLM-Omni that expands Omni’s diffusion / image-video generation and audio / TTS stack, improves distributed execution and memory efficiency, and broadens platform/backend coverage (GPU/ROCm/NPU/XPU). It also brings meaningful upgrades to serving APIs, profiling & benchmarking, and overall stability.
- [2026/02] We released [0.14.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.14.0) - This is the first **stable release** of vLLM-Omni that expands Omni’s diffusion / image-video generation and audio / TTS stack, improves distributed execution and memory efficiency, and broadens platform/backend coverage (GPU/ROCm/NPU/XPU). It also brings meaningful upgrades to serving APIs, profiling & benchmarking, and overall stability. Please check our latest [paper](https://arxiv.org/abs/2602.02204) for architecture design and performance results.
- [2026/01] We released [0.12.0rc1](https://github.com/vllm-project/vllm-omni/releases/tag/v0.12.0rc1) - a major RC milestone focused on maturing the diffusion stack, strengthening OpenAI-compatible serving, expanding omni-model coverage, and improving stability across platforms (GPU/NPU/ROCm), please check our latest [design](https://docs.google.com/presentation/d/1qv4qMW1rKAqDREMXiUDLIgqqHQe7TDPj/edit?usp=sharing&ouid=110473603432222024453&rtpof=true&sd=true). - [2026/01] We released [0.12.0rc1](https://github.com/vllm-project/vllm-omni/releases/tag/v0.12.0rc1) - a major RC milestone focused on maturing the diffusion stack, strengthening OpenAI-compatible serving, expanding omni-model coverage, and improving stability across platforms (GPU/NPU/ROCm), please check our latest [design](https://docs.google.com/presentation/d/1qv4qMW1rKAqDREMXiUDLIgqqHQe7TDPj/edit?usp=sharing&ouid=110473603432222024453&rtpof=true&sd=true).
- [2025/11] vLLM community officially released [vllm-project/vllm-omni](https://github.com/vllm-project/vllm-omni) in order to support omni-modality models serving. - [2025/11] vLLM community officially released [vllm-project/vllm-omni](https://github.com/vllm-project/vllm-omni) in order to support omni-modality models serving.
@@ -67,6 +68,19 @@ Visit our [documentation](https://vllm-omni.readthedocs.io/en/latest/) to learn
We welcome and value any contributions and collaborations. We welcome and value any contributions and collaborations.
Please check out [Contributing to vLLM-Omni](https://vllm-omni.readthedocs.io/en/latest/contributing/) for how to get involved. Please check out [Contributing to vLLM-Omni](https://vllm-omni.readthedocs.io/en/latest/contributing/) for how to get involved.
## Citation
If you use vLLM-Omni for your research, please cite our [paper](https://arxiv.org/abs/2602.02204):
```bibtex
@article{yin2026vllmomni,
title={vLLM-Omni: Fully Disaggregated Serving for Any-to-Any Multimodal Models},
author={Peiqi Yin, Jiangyun Zhu, Han Gao, Chenguang Zheng, Yongxiang Huang, Taichang Zhou, Ruirui Yang, Weizhi Liu, Weiqing Chen, Canlin Guo, Didan Deng, Zifeng Mo, Cong Wang, James Cheng, Roger Wang, Hongsheng Liu},
journal={arXiv preprint arXiv:2602.02204},
year={2026}
}
```
## Join the Community ## Join the Community
Feel free to ask questions, provide feedbacks and discuss with fellow users of vLLM-Omni in `#sig-omni` slack channel at [slack.vllm.ai](https://slack.vllm.ai) or vLLM user forum at [discuss.vllm.ai](https://discuss.vllm.ai). Feel free to ask questions, provide feedbacks and discuss with fellow users of vLLM-Omni in `#sig-omni` slack channel at [slack.vllm.ai](https://slack.vllm.ai) or vLLM user forum at [discuss.vllm.ai](https://discuss.vllm.ai).
+8 -9
View File
@@ -3,19 +3,18 @@ ARG VLLM_BASE_TAG=v0.15.0
FROM ${VLLM_BASE_IMAGE}:${VLLM_BASE_TAG} FROM ${VLLM_BASE_IMAGE}:${VLLM_BASE_TAG}
ARG APP_DIR=/workspace/vllm-omni ARG APP_DIR=/workspace/vllm-omni
WORKDIR ${APP_DIR} WORKDIR ${APP_DIR}
COPY . . COPY . .
# Install system dependencies # Install system dependencies
RUN apt-get update && \ RUN apt-get update && \
apt-get install -y ffmpeg && \ apt-get install -y ffmpeg && \
apt-get clean && \ apt-get clean && \
rm -rf /var/lib/apt/lists/* rm -rf /var/lib/apt/lists/*
# Install vLLM wheel for the specific commit
# URL format: https://wheels.vllm.ai/<commit_hash>/
# Current vLLM commit: f176443446f659dbab5315e056e605d8984fd976 (v0.15.0)
RUN uv pip install --system --upgrade --force-reinstall vllm --extra-index-url https://wheels.vllm.ai/f176443446f659dbab5315e056e605d8984fd976
# Install vllm-omni into the same uv-managed Python environment used by the base image. # Install vllm-omni into the same uv-managed Python environment used by the base image.
RUN uv pip install --system --no-cache-dir ".[dev]" RUN uv pip install --python "$(python3 -c 'import sys; print(sys.executable)')" --no-cache-dir ".[dev]"
RUN ln -sf /usr/bin/python3 /usr/bin/python RUN ln -sf /usr/bin/python3 /usr/bin/python
ENTRYPOINT [] ENTRYPOINT []
+1 -1
View File
@@ -253,7 +253,7 @@ class DiffusionEngine:
try: try:
# Give worker enough time — export + compression + table can be slow # Give worker enough time — export + compression + table can be slow
results = self.collective_rpc(method="stop_profile", timeout=60000) results = self.collective_rpc(method="stop_profile", timeout=600)
except Exception: except Exception:
logger.error("Failed to stop profiling on workers", exc_info=True) logger.error("Failed to stop profiling on workers", exc_info=True)
return {"traces": [], "tables": []} return {"traces": [], "tables": []}
+1 -1
View File
@@ -97,7 +97,7 @@ class OmniServeCommand(CLISubcommand):
omni_config_group.add_argument( omni_config_group.add_argument(
"--init-timeout", "--init-timeout",
type=int, type=int,
default=60000, default=600,
help="The timeout for initializing the stages.", help="The timeout for initializing the stages.",
) )
omni_config_group.add_argument( omni_config_group.add_argument(
+2 -2
View File
@@ -221,8 +221,8 @@ class OmniStage:
# Wait for result from worker # Wait for result from worker
try: try:
# Profiling stop might take time to flush files, give it 180s # Profiling stop might take time to flush files, give it 600s
response = self._out_q.get(timeout=60000) response = self._out_q.get(timeout=600)
if isinstance(response, dict): if isinstance(response, dict):
if response.get("type") == "profiler_result": if response.get("type") == "profiler_result":