anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
855 Commits 235 Branches 31 Tags
62c8695dfd831f129b2e3021ae6d5c1673c51cce
Commit Graph
855 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam 62c8695dfd Update GPUGenerationModelRunner to clarify KV connector metadata handling during draft model runs 
- Changed variable name to better reflect its purpose in managing KV connector metadata.
- Updated comments for improved clarity regarding the handling of speculative configurations.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 03:16:33 +00:00
tzhouam d911475b22 Fix output queue handling in OmniStage to return None if queue is closed during shutdown 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 01:59:04 +00:00
tzhouam e17fb0e97e Refactor OpenAI serving components and enhance multimodal handling 
- Updated import for OpenAIServingEmbedding in api_server.py.
- Enhanced omni_init_app_state to initialize renderer for engine_client.
- Added handle_oov_mm_token parameter to multiple model classes for better multimodal token handling.
- Improved comments for clarity in various model files.
- Adjusted GPUARModelRunner to ensure proper handling of late interaction runner attributes.

Signed-off-by: [Your Name] <[Your Email]>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 01:55:17 +00:00
tzhouam 72cb4f661b Adjust GPU memory utilization and device settings in qwen2_5_omni_ci.yaml for improved performance 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-12 13:57:52 +00:00
tzhouam 11b6c16007 rebase: align vllm-omni with vLLM 84e436ed1c94 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-11 19:24:44 +00:00
tzhouam 08a2673c66 rebase: align vllm-omni with vLLM 84e436ed1c94 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-11 07:29:52 +00:00
tzhouam e47c5dd7fb Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-11 02:50:45 +00:00
Didan DengandCanlin Guo 7543f2ff3b [Bugfix]: SP attention not enabling when _sp_plan hooks are not applied (#1704) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
 2026-03-11 10:49:42 +08:00
Ting FU 6ac85d4607 [Feat] Enable expert parallel for diffusion MoE layers (#1323) 
Signed-off-by: Semmer2 <semmer@live.cn>
 2026-03-11 10:45:36 +08:00
tzhouam 91135f13aa Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-11 02:43:59 +00:00
rein yang 4725593a4b [Docs] update async chunk performance diagram (#1741) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-03-11 10:39:23 +08:00
tzhouam ad12921cc2 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-11 01:44:32 +00:00
SYLARandlinyueqian c51b4fd7b8 [Feat][Qwen3-tts]: Add Gradio demo for online serving (#1231) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
 2026-03-11 09:36:02 +08:00
Junhong Liu 2ee4a07adb [Refactor][Perf] Qwen3-omni: code predictor with re-prefill + SDPA and eliminate decode hot-path CPU round-trips (#1758) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-03-11 07:52:21 +08:00
Juan Pablo Zuluagaandpablo 1d0a97f0d1 [feat][Qwen3TTS] Simple dynamic TTFA based on Code2Wav load (#1714) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-03-10 21:02:33 +00:00
Baoyuan Qi 2fdc647ed7 [BUGFIX] Add compatibility for mimo-audio with vLLM 0.17.0 (#1752) 
Signed-off-by: Baoyuan Qi <qibaoyuan@xiaomi.com>
 2026-03-10 20:57:26 +00:00
Hongsheng Liu 4027534dbf Revert "Add online serving to Stable Audio Diffusion and introduce v1/audio/generate endpoint" (#1789) 2026-03-10 22:40:48 +08:00
Zhou Taichang汪志鹏Hongsheng LiuCyrus Leung
 		e449cc411e [Doc][skip-ci] Update installation instructions (#1762) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-10 22:29:16 +08:00
tzhouam 20c102d3cf debug: empty commit to test CI pipeline (vLLM ddbb0d230a35) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 10:04:29 +00:00
tzhouam a9862f17f3 Revert "Refactor Dockerfile.ci to consolidate package installation commands into a single RUN statement, improving readability and efficiency of the CI environment setup." 
This reverts commit 1c0a71e30d.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 09:41:45 +00:00
zhumingjue138 5d62fcd717 [CI]: Add core_model and cpu markers for L1 use case. (#1709) 
Signed-off-by: zhumingjue <zhumingjue@huawei.com>
 2026-03-10 17:39:37 +08:00
tzhouam 1c0a71e30d Refactor Dockerfile.ci to consolidate package installation commands into a single RUN statement, improving readability and efficiency of the CI environment setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 09:13:24 +00:00
tzhouam cee2b4bd11 debug: empty commit to test CI pipeline (vLLM ddbb0d230a35) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 08:58:06 +00:00
tzhouam b0cf788d8f debug: empty commit to test CI pipeline (vLLM ddbb0d230a35) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 08:56:23 +00:00
tzhouam dd0ec557b1 debug: empty commit to test CI pipeline (vLLM ddbb0d230a35) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 08:55:56 +00:00
tzhouam 4014e76795 debug: empty commit to test CI pipeline (vLLM ddbb0d230a35) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 08:54:54 +00:00
tzhouam 5c845ce50b debug: empty commit to test CI pipeline (vLLM ddbb0d230a35) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 08:54:06 +00:00
tzhouam 9dc5d15f1d debug: empty commit to test CI pipeline (vLLM ddbb0d230a35) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 08:09:53 +00:00
tzhouam cf0a7a5475 Update Dockerfile.ci to install additional dependencies for flashinfer and numpy, and create a symlink for python3. This enhances the CI environment setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 08:05:28 +00:00
Daniel Huang 745ef312a7 [Enhancement][pytest] Check for process running during start server (#1559) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
 2026-03-10 15:44:54 +08:00
tzhouam 8ac369c84a debug: empty commit to test CI pipeline (vLLM 156e33553ccd) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 07:13:09 +00:00
tzhouam cfbdb57ec6 debug: empty commit to test CI pipeline (vLLM 156e33553ccd) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 07:11:31 +00:00
tzhouam 7706132d8a debug: empty commit to test CI pipeline (vLLM 156e33553ccd) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 07:08:42 +00:00
tzhouam e150a1b28b debug: empty commit to test CI pipeline (vLLM 156e33553ccd) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 06:55:27 +00:00
tzhouam 3630bdfaa7 Update Dockerfile.ci to install vLLM from a new commit and simplify worker_type retrieval in omni_stage.py. Removed unnecessary dependency installations for flashinfer and numpy. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 01:55:12 +00:00
tzhouam dd0c893e79 Align dev/vllm-align with upstream vLLM changes 
- Update Dockerfile.ci to install vLLM from specific commit wheel,
  add flashinfer/cublas/numpy dependencies
- Fix worker_type leak in omni_stage by using pop instead of get

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-10 01:46:53 +00:00
Ekagra Ranjan 7a560492a2 Add online serving to Stable Audio Diffusion and introduce v1/audio/generate endpoint (#1255) 2026-03-09 13:36:46 -04:00
Sy03 761eff930a [Fix][Qwen3-TTS] Preserve ref_code decoder context for Base ICL (#1731) 2026-03-09 13:36:27 -04:00
Ricardo Noriega 112a9aa237 [Feature] Add Helm Chart to deploy vLLM-Omni on Kubernetes (#1337) 
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com>
 2026-03-09 16:02:23 +00:00
Didan Deng e79ad8d59a [Doc] CLI Args Naming Style Correction (#1750) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-09 23:35:45 +08:00
Yuanheng Zhao 82f63a4403 [BugFix] Enable CPU offloading and Cache-DiT together on Diffusion Model (#1723) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-03-09 22:25:41 +08:00
汪志鹏 7271116973 [Bagel]: Support multistage img2img (#1669) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-09 22:23:50 +08:00
wangyu 5628d41242 [Test] Solving the Issue of Whisper Model's GPU Memory Not Being Successfully Cleared and the Occasional Accuracy Problem of the Qwen3-omni Model Test (#1744) 
Signed-off-by: yenuo26 <410167048@qq.com>
 2026-03-09 21:52:56 +08:00
Samit 155856ff30 [Perf] Reduce IPC overhead for single-stage diffusion serving for Wan2.2 (#1715) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
v0.17.0rc1 		2026-03-10 00:03:51 +13:00
Zhou Taichang汪志鹏Hongsheng LiuCyrus Leung
 		89b527e533 Dev/rebase v0170 (#1639) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-09 17:29:38 +08:00
Yueqian Lin 4db10d6dec Disable mm processor cache in CI stage configs (#1739) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-09 15:42:15 +08:00
Canlin Guo 29357027b0 [Bugfix] Allow to enable HSDP alone (#1567) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-09 06:36:13 +00:00
Sy03Yueqian Linlinyueqian
 		d6871db6a3 [Feat]: Offline inference supports async_chunk (#1415) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
 2026-03-09 11:33:38 +08:00
Ding Zuhaoandprincepride fb717a4071 [Bagel] Eliminate broadcast in CFG parallel denoising loop (#1695) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
 2026-03-09 03:26:21 +00:00
zhaotyerlinyueqianYueqian Lin
 		41964cb90a feat(tts): add voice upload API for Qwen3-TTS (#1201) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-09 11:02:11 +08:00