Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d6f93b0f37 | ||
|
|
7fcbaa794c | ||
|
|
3fb3c8aba4 |
No files matched your search
@@ -15,7 +15,8 @@ Easy, fast, and cheap omni-modality model serving for everyone
|
||||
---
|
||||
|
||||
*Latest News* 🔥
|
||||
- [2026/02] We released [0.14.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.14.0) - This is the first **stable release** of vLLM-Omni that expands Omni’s diffusion / image-video generation and audio / TTS stack, improves distributed execution and memory efficiency, and broadens platform/backend coverage (GPU/ROCm/NPU/XPU). It also brings meaningful upgrades to serving APIs, profiling & benchmarking, and overall stability.
|
||||
|
||||
- [2026/02] We released [0.14.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.14.0) - This is the first **stable release** of vLLM-Omni that expands Omni’s diffusion / image-video generation and audio / TTS stack, improves distributed execution and memory efficiency, and broadens platform/backend coverage (GPU/ROCm/NPU/XPU). It also brings meaningful upgrades to serving APIs, profiling & benchmarking, and overall stability. Please check our latest [paper](https://arxiv.org/abs/2602.02204) for architecture design and performance results.
|
||||
- [2026/01] We released [0.12.0rc1](https://github.com/vllm-project/vllm-omni/releases/tag/v0.12.0rc1) - a major RC milestone focused on maturing the diffusion stack, strengthening OpenAI-compatible serving, expanding omni-model coverage, and improving stability across platforms (GPU/NPU/ROCm), please check our latest [design](https://docs.google.com/presentation/d/1qv4qMW1rKAqDREMXiUDLIgqqHQe7TDPj/edit?usp=sharing&ouid=110473603432222024453&rtpof=true&sd=true).
|
||||
- [2025/11] vLLM community officially released [vllm-project/vllm-omni](https://github.com/vllm-project/vllm-omni) in order to support omni-modality models serving.
|
||||
|
||||
@@ -67,6 +68,19 @@ Visit our [documentation](https://vllm-omni.readthedocs.io/en/latest/) to learn
|
||||
We welcome and value any contributions and collaborations.
|
||||
Please check out [Contributing to vLLM-Omni](https://vllm-omni.readthedocs.io/en/latest/contributing/) for how to get involved.
|
||||
|
||||
## Citation
|
||||
|
||||
If you use vLLM-Omni for your research, please cite our [paper](https://arxiv.org/abs/2602.02204):
|
||||
|
||||
```bibtex
|
||||
@article{yin2026vllmomni,
|
||||
title={vLLM-Omni: Fully Disaggregated Serving for Any-to-Any Multimodal Models},
|
||||
author={Peiqi Yin, Jiangyun Zhu, Han Gao, Chenguang Zheng, Yongxiang Huang, Taichang Zhou, Ruirui Yang, Weizhi Liu, Weiqing Chen, Canlin Guo, Didan Deng, Zifeng Mo, Cong Wang, James Cheng, Roger Wang, Hongsheng Liu},
|
||||
journal={arXiv preprint arXiv:2602.02204},
|
||||
year={2026}
|
||||
}
|
||||
```
|
||||
|
||||
## Join the Community
|
||||
Feel free to ask questions, provide feedbacks and discuss with fellow users of vLLM-Omni in `#sig-omni` slack channel at [slack.vllm.ai](https://slack.vllm.ai) or vLLM user forum at [discuss.vllm.ai](https://discuss.vllm.ai).
|
||||
|
||||
|
||||
@@ -3,19 +3,18 @@ ARG VLLM_BASE_TAG=v0.15.0
|
||||
FROM ${VLLM_BASE_IMAGE}:${VLLM_BASE_TAG}
|
||||
ARG APP_DIR=/workspace/vllm-omni
|
||||
WORKDIR ${APP_DIR}
|
||||
|
||||
COPY . .
|
||||
|
||||
# Install system dependencies
|
||||
RUN apt-get update && \
|
||||
apt-get install -y ffmpeg && \
|
||||
apt-get clean && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install vLLM wheel for the specific commit
|
||||
# URL format: https://wheels.vllm.ai/<commit_hash>/
|
||||
# Current vLLM commit: f176443446f659dbab5315e056e605d8984fd976 (v0.15.0)
|
||||
RUN uv pip install --system --upgrade --force-reinstall vllm --extra-index-url https://wheels.vllm.ai/f176443446f659dbab5315e056e605d8984fd976
|
||||
apt-get install -y ffmpeg && \
|
||||
apt-get clean && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install vllm-omni into the same uv-managed Python environment used by the base image.
|
||||
RUN uv pip install --system --no-cache-dir ".[dev]"
|
||||
RUN uv pip install --python "$(python3 -c 'import sys; print(sys.executable)')" --no-cache-dir ".[dev]"
|
||||
|
||||
RUN ln -sf /usr/bin/python3 /usr/bin/python
|
||||
|
||||
ENTRYPOINT []
|
||||
@@ -253,7 +253,7 @@ class DiffusionEngine:
|
||||
|
||||
try:
|
||||
# Give worker enough time — export + compression + table can be slow
|
||||
results = self.collective_rpc(method="stop_profile", timeout=60000)
|
||||
results = self.collective_rpc(method="stop_profile", timeout=600)
|
||||
except Exception:
|
||||
logger.error("Failed to stop profiling on workers", exc_info=True)
|
||||
return {"traces": [], "tables": []}
|
||||
|
||||
@@ -97,7 +97,7 @@ class OmniServeCommand(CLISubcommand):
|
||||
omni_config_group.add_argument(
|
||||
"--init-timeout",
|
||||
type=int,
|
||||
default=60000,
|
||||
default=600,
|
||||
help="The timeout for initializing the stages.",
|
||||
)
|
||||
omni_config_group.add_argument(
|
||||
|
||||
@@ -221,8 +221,8 @@ class OmniStage:
|
||||
|
||||
# Wait for result from worker
|
||||
try:
|
||||
# Profiling stop might take time to flush files, give it 180s
|
||||
response = self._out_q.get(timeout=60000)
|
||||
# Profiling stop might take time to flush files, give it 600s
|
||||
response = self._out_q.get(timeout=600)
|
||||
|
||||
if isinstance(response, dict):
|
||||
if response.get("type") == "profiler_result":
|
||||
|
||||
Reference in new issue
Block a user