anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
957 Commits 235 Branches 31 Tags
034d579d770bb39fa2fb8d2bd1caca27f1affae3
Commit Graph
237 Commits
Author SHA1 Message Date
tzhouam 034d579d77 Update vLLM precompiled wheel version and refactor various components for improved functionality and error handling. Changes include modifying the get_name method in DiffusionQuantizationConfig to use class-level attributes, enhancing metadata handling in StageEngineCoreClient, and adjusting the AsyncOmni class constructor for better parameter management. Additionally, improved error handling in omni_snapshot_download and optimized initial chunk size computation in talker2code2wav_async_chunk for dynamic load adaptation. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-19 05:14:34 +00:00
tzhouam 342a209924 Merge remote-tracking branch 'origin/main' into dev/vllm-align 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-19 02:06:26 +00:00
TJian c85acb1773 [CI] [ROCm] Bugfix device environment issue (#1984) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-03-19 05:11:28 +08:00
Chenguang ZhengyinpePeiqi YinwuhangPeiqi Yinlinyueqianprincepridelishunyang
 		3158e9d28f [Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring (#1908) 
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Signed-off-by: Peiqi Yin <yinpeiqi809@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: Peiqi Yin <yinpeiqi809@gmail.com>
Co-authored-by: wuhang <whlbx@hotmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-03-18 17:33:53 +08:00
tzhouam 004c60a031 Refactor shutdown and cleanup logic in OmniRunner and OmniStage to improve process termination and resource management. The changes ensure that stage workers are stopped before closing queues, preventing potential orphaned subprocesses. Additionally, enhanced error handling during process termination provides clearer logging for failed shutdown attempts. This update aims to increase the reliability and stability of the shutdown process across the application. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 09:33:28 +00:00
tzhouam cc4ade4ba7 Refactor shutdown logic in OmniLLM and stage worker to ensure proper engine closure. The shutdown method now checks for the presence of engine_core before invoking shutdown, enhancing resource management. Additionally, the stage worker now explicitly calls stage_engine.close() upon receiving a shutdown signal. This improves the overall stability and reliability of the shutdown process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 08:19:56 +00:00
tzhouam de6bcc734c Update Dockerfile.ci to use a new vllm wheel URL and refactor MooncakeConnector import logic in monkey_patch.py for improved compatibility. Adjust OpenAIServingChat to call the harmony request method from the correct context. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 08:33:28 +00:00
tzhouam a26e568dfc Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-16 07:30:43 +00:00
tzhouam 9c22e3afb0 Refactor import statement for OpenAIServingEmbedding in api_server.py to use aliasing for clarity 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 06:13:07 +00:00
Jinheng 88caaf182b [Feature] Split #1303 Part 1: PD disaggregation scaffolding (#1863) 
Signed-off-by: Jinheng Li <ahengljh@gmail.com>
 2026-03-16 14:05:19 +08:00
tzhouam aae101f495 Refactor import statement for OpenAIServingEmbedding in api_server.py 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 03:39:51 +00:00
tzhouam 3a508d7b7a rebase: align vllm-omni with vLLM e9163b536e72 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 02:32:39 +00:00
tzhouam f137a1e3ad Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-16 01:49:46 +00:00
Lancer af012babf8 [Bugfix] Fix cpu offload and quantization compatibility (#1473) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-14 18:55:30 +00:00
tzhouam 97899c6c98 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-13 19:03:57 +00:00
Yueqian Lin cebf78ba3e [Bugfix] Restore voice upload API and profiler endpoints reverted by #1719 (#1879) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-14 04:37:08 +13:00
dengyunyang 3e8cb6c631 [Diffusion] [Model] Dreamid omni (#1855) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-03-13 21:53:06 +08:00
Ian EavesSamitGao Han
 		92f771d455 [Frontend] Rewrite video API for async job lifecycle (#1665) 
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2026-03-13 17:11:06 +08:00
汪志鹏 e0b71056ba [BugFix]: Fix bagel online inference bug (#1804) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-13 08:58:44 +00:00
tzhouam d3db476f2d Merge branch 'main' into dev/vllm-align 2026-03-13 06:51:22 +00:00
tzhouam d911475b22 Fix output queue handling in OmniStage to return None if queue is closed during shutdown 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 01:59:04 +00:00
tzhouam e17fb0e97e Refactor OpenAI serving components and enhance multimodal handling 
- Updated import for OpenAIServingEmbedding in api_server.py.
- Enhanced omni_init_app_state to initialize renderer for engine_client.
- Added handle_oov_mm_token parameter to multiple model classes for better multimodal token handling.
- Improved comments for clarity in various model files.
- Adjusted GPUARModelRunner to ensure proper handling of late interaction runner attributes.

Signed-off-by: [Your Name] <[Your Email]>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 01:55:17 +00:00
Yueqian Lin 366b3368ea Add Fish Speech S2 Pro support with online serving and voice cloning (#1798) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-12 16:20:52 -04:00
SYLAR c8c60e1970 [Config Refactor][1/2] Model Pipeline Configuration System (#1115) 
Signed-off-by: lishunyang <lishunyang12@163.com>
 2026-03-12 21:09:34 +08:00
汪志鹏 a52867abde [BugFix]: Revert #1582 (#1842) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-12 19:19:57 +08:00
rongfu.leng 7c5159037f [Feat] add wav response_format when stream is true in /v1/audio/speec… (#1819) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-12 17:45:13 +08:00
汪志鹏 11d9ca8f41 [Feature]: Enable directly use OmniLLM init AR model (#1821) 2026-03-12 04:24:10 +00:00
Vensen 3306fa6611 [Feat][Diffusion]: Implement Component-Level VRAM Quota and Resource Domain Isolation (#1582) 
Signed-off-by: vensen <vensenmu@gmail.com>
 2026-03-12 12:17:18 +08:00
Daniel HuangandSYLAR 0e005817f7 [Enhancement] Patch OmniStage.try_collect() with ray alive checks (#1561) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
Signed-off-by: Daniel Huang <pilotflyer824@gmail.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-03-12 12:08:34 +08:00
Sy03Yueqian Linlinyueqian
 		5d28b6a40c [Feat][Qwen3-TTS] Support streaming audio output for websocket (#1719) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
 2026-03-11 23:48:41 -04:00
Daniel Huang ac17431745 [Enhancement] Patch OmniStage.try_collect() with _proc alive checks (#1560) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
 2026-03-12 10:40:42 +08:00
Yueqian Lin e1efcb7560 Fix duplicate get_supported_tasks definition in async_omni.py (#1825) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-11 19:43:37 -04:00
Yueqian Lin a6b5519616 [Bugfix] Report supported tasks for omni models to skip unnecessary chat init (#1645) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-12 01:06:20 +08:00
Lancer 0f7c9c755d [Bugfix] Expose base_model_paths property in _DiffusionServingModels (#1771) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-11 20:52:10 +08:00
tzhouam 08a2673c66 rebase: align vllm-omni with vLLM 84e436ed1c94 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-11 07:29:52 +00:00
Ting FU 6ac85d4607 [Feat] Enable expert parallel for diffusion MoE layers (#1323) 
Signed-off-by: Semmer2 <semmer@live.cn>
 2026-03-11 10:45:36 +08:00
Juan Pablo Zuluagaandpablo 1d0a97f0d1 [feat][Qwen3TTS] Simple dynamic TTFA based on Code2Wav load (#1714) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-03-10 21:02:33 +00:00
Hongsheng Liu 4027534dbf Revert "Add online serving to Stable Audio Diffusion and introduce v1/audio/generate endpoint" (#1789) 2026-03-10 22:40:48 +08:00
Ekagra Ranjan 7a560492a2 Add online serving to Stable Audio Diffusion and introduce v1/audio/generate endpoint (#1255) 2026-03-09 13:36:46 -04:00
Sy03 761eff930a [Fix][Qwen3-TTS] Preserve ref_code decoder context for Base ICL (#1731) 2026-03-09 13:36:27 -04:00
Samit 155856ff30 [Perf] Reduce IPC overhead for single-stage diffusion serving for Wan2.2 (#1715) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-03-10 00:03:51 +13:00
Zhou Taichang汪志鹏Hongsheng LiuCyrus Leung
 		89b527e533 Dev/rebase v0170 (#1639) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-09 17:29:38 +08:00
zhaotyerlinyueqianYueqian Lin
 		41964cb90a feat(tts): add voice upload API for Qwen3-TTS (#1201) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-09 11:02:11 +08:00
knlnguyen1802andSamit 92e1a54419 [RL] Support collective rpc api to entrypoint && Support custom input output (#1646) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-03-09 10:12:43 +08:00
Yueqian Lin a9f694b08a [Bugfix] Use upstream MediaConnector for ref_audio resolution (#1661) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-09 01:38:26 +00:00
Canlin Guo 074af30a9b Revert "[Profile] Adding metrics for Diffusion/DiT Single diffusion Pipeline (#668)" (#1724) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-08 00:11:47 +13:00
Didan Deng 30191ed112 [Feature]: Add vae-patch-parallel CLI argument in online serving (#1716) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-07 01:45:02 +08:00
erfgss b7fcc9d07e [Profile] Adding metrics for Diffusion/DiT Single diffusion Pipeline (#668) 
Signed-off-by: Chen Yang <2082464740@qq.com>
Signed-off-by: erfgss <97771661+erfgss@users.noreply.github.com>
 2026-03-06 15:39:19 +08:00
Canlin Guo 846167b237 [1/N][Refactor] Clean up dead code in output processor (#1579) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-06 14:45:53 +08:00
Jack Leegoogle-labs-jules[bot]JackLeeHal
 		c0b714a8ac [Feature] Support flexible task_type configuration for Qwen3-TTS models (#1197) 
Signed-off-by: Jack Lee <695697442@qq.com>
Signed-off-by: jackleehal <695697442@qq.com>
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: JackLeeHal <15664755+JackLeeHal@users.noreply.github.com>
 2026-03-05 13:39:09 -05:00