anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
894 Commits 235 Branches 31 Tags
d3db476f2dfb466ed26af065342d208ab06d374d
Commit Graph
894 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam d3db476f2d Merge branch 'main' into dev/vllm-align 2026-03-13 06:51:22 +00:00
tzhouam 62c8695dfd Update GPUGenerationModelRunner to clarify KV connector metadata handling during draft model runs 
- Changed variable name to better reflect its purpose in managing KV connector metadata.
- Updated comments for improved clarity regarding the handling of speculative configurations.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 03:16:33 +00:00
Alex BrooksandDidan Deng c8cf8a799d [Refactor] Use SP Plan for LongCat Sequence Parallelism (#1772) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-13 11:13:34 +08:00
dsinghviandYueqian Lin 0540af327e [BugFix] Readme and example runner file for cosyvoice3 missed in refactoring (#1685) 
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-13 10:34:29 +08:00
tzhouam d911475b22 Fix output queue handling in OmniStage to return None if queue is closed during shutdown 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 01:59:04 +00:00
tzhouam e17fb0e97e Refactor OpenAI serving components and enhance multimodal handling 
- Updated import for OpenAIServingEmbedding in api_server.py.
- Enhanced omni_init_app_state to initialize renderer for engine_client.
- Added handle_oov_mm_token parameter to multiple model classes for better multimodal token handling.
- Improved comments for clarity in various model files.
- Adjusted GPUARModelRunner to ensure proper handling of late interaction runner attributes.

Signed-off-by: [Your Name] <[Your Email]>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 01:55:17 +00:00
NATURE 54a999d031 [skip CI][Docs] add connector design document (#1737) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-03-13 09:23:32 +08:00
Yueqian Lin 366b3368ea Add Fish Speech S2 Pro support with online serving and voice cloning (#1798) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-12 16:20:52 -04:00
Gao Han 36c6cb9634 [CI failure] Comment out test_zimage_vae_patch_parallel_tp2 (#1856) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-03-13 01:00:16 +08:00
Yueqian Lin 9c6fbd76be Enable async_scheduling by default for Qwen3-TTS (#1853) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-12 12:30:20 -04:00
Du Bin 8ad8988e2b [Bugfix] Fix Helios text_encoder embed_tokens all-zeros due to untied weights (#1728) 
Signed-off-by: OSS Scout <scout@oss-scout.local>
 2026-03-12 15:28:21 +00:00
tzhouam 72cb4f661b Adjust GPU memory utilization and device settings in qwen2_5_omni_ci.yaml for improved performance 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-12 13:57:52 +00:00
usberkeleyprinceprideHongsheng Liu
 		b3cced310a [bugfix] Add Interleaved 2D Rotary Embedding for HunyuanImage3 (#1784) 
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-12 21:33:03 +08:00
Zeyu Huang | 黃澤宇andHongsheng Liu 21e621f84c [Test] Reduce SP & Offloading test cases for L2 (#1839) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-12 13:32:43 +00:00
SYLAR c8c60e1970 [Config Refactor][1/2] Model Pipeline Configuration System (#1115) 
Signed-off-by: lishunyang <lishunyang12@163.com>
 2026-03-12 21:09:34 +08:00
Jiaping Wuskf1999Just-itSemmer2gcanlin
 		28dd1a699d [Model] Extend NPU support for HunyuanImage3 Diffusion Model (#1689) 
Signed-off-by: ElleElleWu <1608928702@qq.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: skf1999 <13234016272@163.com>
Co-authored-by: Just-it <1161406585@qq.com>
Co-authored-by: Semmer2 <semmer@live.cn>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-12 20:49:17 +08:00
Hongsheng Liuandhsliu_ustc ff3365f6a2 [skip ci] update readme slides link (#1850) 
Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
 2026-03-12 20:27:42 +08:00
JohnJanwuzhongjianGao Han
 		fe7e8770db [Feature]: support Flux.2-dev cache_dit (#1814) 
Co-authored-by: wuzhongjian <wuzhongjian@cmss.chinamobile.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2026-03-12 19:53:47 +08:00
汪志鹏 a52867abde [BugFix]: Revert #1582 (#1842) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-12 19:19:57 +08:00
rongfu.leng 7c5159037f [Feat] add wav response_format when stream is true in /v1/audio/speec… (#1819) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-12 17:45:13 +08:00
wangyu 1c7f963a2d [Bugfix] Modify _resolve_pytest_target to support glob patterns and return multiple paths (#1843) 
Signed-off-by: yenuo26 <410167048@qq.com>
 2026-03-12 09:29:08 +00:00
Samit 4dbaa74f2b [Enhancement] Upgrade cache-dit from 1.2.0 to 1.3.0 (#1834) 
Signed-off-by: samithuang <285365963@qq.com>
 2026-03-12 15:39:06 +08:00
汪志鹏 11d9ca8f41 [Feature]: Enable directly use OmniLLM init AR model (#1821) 2026-03-12 04:24:10 +00:00
Vensen 3306fa6611 [Feat][Diffusion]: Implement Component-Level VRAM Quota and Resource Domain Isolation (#1582) 
Signed-off-by: vensen <vensenmu@gmail.com>
 2026-03-12 12:17:18 +08:00
Daniel HuangandSYLAR 0e005817f7 [Enhancement] Patch OmniStage.try_collect() with ray alive checks (#1561) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
Signed-off-by: Daniel Huang <pilotflyer824@gmail.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-03-12 12:08:34 +08:00
wangyu c501c2d512 [Test] Nightly Buildkite Pytest Test Case Statistics And Send HTML Report By Email (#1674) 
Signed-off-by: yenuo26 <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
 2026-03-12 12:08:11 +08:00
Sy03Yueqian Linlinyueqian
 		5d28b6a40c [Feat][Qwen3-TTS] Support streaming audio output for websocket (#1719) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
 2026-03-11 23:48:41 -04:00
Alicia 09b66bf6db [Doc][skip ci] Update readme with Video link for vLLM HK First Meetup (#1833) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-12 16:06:36 +13:00
Daniel Huang ac17431745 [Enhancement] Patch OmniStage.try_collect() with _proc alive checks (#1560) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
 2026-03-12 10:40:42 +08:00
Yueqian Lin e1efcb7560 Fix duplicate get_supported_tasks definition in async_omni.py (#1825) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-11 19:43:37 -04:00
tzhouam 11b6c16007 rebase: align vllm-omni with vLLM 84e436ed1c94 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-11 19:24:44 +00:00
wangyu da4a077155 [Test] Skip the qwen3-omni relevant validation for a known issue 1367. (#1812) 
Signed-off-by: yenuo26 <410167048@qq.com>
 2026-03-11 15:00:13 -04:00
Yueqian Lin acde431668 Add Qwen3-TTS benchmark scripts (#1573) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-12 01:09:26 +08:00
Yueqian Lin f144f5effe [Test] Add Qwen3-TTS nightly performance benchmark (#1700) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-12 01:07:47 +08:00
Yueqian Lin a6b5519616 [Bugfix] Report supported tasks for omni models to skip unnecessary chat init (#1645) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-12 01:06:20 +08:00
Lancer 0f7c9c755d [Bugfix] Expose base_model_paths property in _DiffusionServingModels (#1771) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-11 20:52:10 +08:00
Didan Deng af9784c03f [Bugfix] Adjust Z-Image Tensor Parallelism Diff Threshold (#1808) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-11 20:30:54 +08:00
Zeyu Huang | 黃澤宇 74eedd3ef1 [Frontend] ComfyUI video & LoRA support (#1596) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-03-11 10:22:45 +00:00
Zeyu Huang | 黃澤宇andHongsheng Liu a19098233b [Test] L4 complete diffusion feature test for Qwen-Image-Edit models (#1682) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-11 18:10:06 +08:00
Samit b313279af7 [skip ci][Docs] doc fix for example snippets (#1811) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-03-11 17:32:12 +08:00
tzhouam 08a2673c66 rebase: align vllm-omni with vLLM 84e436ed1c94 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-11 07:29:52 +00:00
JohnJanandwuzhongjian 4d89ebaa07 [Model]: add FLUX.2-dev model (#1629) 
Co-authored-by: wuzhongjian <wuzhongjian@cmss.chinamobile.com>
 2026-03-11 14:44:18 +08:00
NumberWan 617b1d6053 Add coordinator module and corresponding unit test (#1465) 
Signed-off-by: Jeff Wan <wantszkin2003@gmail.com>
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-03-11 14:36:48 +08:00
Chendi.XueandSYLAR ca37f221a8 update GpuMemoryMonitor to DeviceMemoryMonitor for all HW (#1526) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-03-11 12:19:36 +08:00
WeiQing Chenanddavid6666666 2b5a3dd6b3 [skip ci] [Docs] Update WeChat QR code for community support (#1802) 
Signed-off-by: david6666666 <david6666666@users.noreply.github.com>
Co-authored-by: david6666666 <david6666666@users.noreply.github.com>
 2026-03-11 12:19:02 +08:00
tzhouam e47c5dd7fb Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-11 02:50:45 +00:00
Didan DengandCanlin Guo 7543f2ff3b [Bugfix]: SP attention not enabling when _sp_plan hooks are not applied (#1704) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
 2026-03-11 10:49:42 +08:00
Ting FU 6ac85d4607 [Feat] Enable expert parallel for diffusion MoE layers (#1323) 
Signed-off-by: Semmer2 <semmer@live.cn>
 2026-03-11 10:45:36 +08:00
tzhouam 91135f13aa Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-11 02:43:59 +00:00
rein yang 4725593a4b [Docs] update async chunk performance diagram (#1741) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-03-11 10:39:23 +08:00