anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,775 Commits 235 Branches 31 Tags
c69ed4037dcbae7306cba9094cb42bf94de464f0
Commit Graph
73 Commits
Author SHA1 Message Date
+20
Zhou TaichangClaude Opus 4.7NumberWandsinghviDnoobYueqian LinHongsheng Liuknlnguyen1802Samitrein yangNick Caozhumingjue138Ricardo Noriegalyj-jjjCursorgcanlinwangyuweizhoublueweizhoubluedengyunyang汪志鹏WeiQing ChenJie LiuYueqian LinNATUREbjf-frzamy-why-3459Y. FisherZeyu Huang | 黃澤宇Copilot Autofix powered by AI
 		eb4e60ee64 [Rebase] Rebase to vllm v0.21.0 (#3530) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Dnoob <dxpouo@gmail.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: rein yang <ruiruyang2@gmail.com>
Signed-off-by: Nick Cao <ncao@redhat.com>
Signed-off-by: zhumingjue <zhumingjue@huawei.com>
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com>
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: weizhoublue <weizhoublue@github.com>
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io>
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: NumberWan <wantszkin2003@gmail.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Dnoob <dxpouo@gmail.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com>
Co-authored-by: Ricardo Noriega <rnoriega@redhat.com>
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: weizhoublue <45163302+weizhoublue@users.noreply.github.com>
Co-authored-by: weizhoublue <weizhoublue@github.com>
Co-authored-by: dengyunyang <584797741@qq.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: NATURE <wzliu@connect.hku.hk>
Co-authored-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Y. Fisher <yukexiong1@huawei.com>
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-05-16 18:18:43 +08:00
wuhang 0d9d57acd9 [Entrypoint][Refactor] Make field type hint more concrete (#3139) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-05-14 11:25:55 +08:00
Alex BrooksSy03Yueqian LinYueqian Lin
 		e03ab82dd2 [Refactor/Bugfix] Use Delta Messages for Streaming (#2911) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Sy03 <1370724210@qq.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-26 15:11:53 +12:00
wuhang fd9a0ee27f [Feature] Failure message shows more details (#2961) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-04-23 04:32:43 +08:00
Daniel HuangChenguang ZhengZhou TaichangHongsheng Liubjf-frz
 		52b53367ef [Enhancement] Engine runtime errors (#2426) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
Signed-off-by: Daniel Huang <pilotflyer824@gmail.com>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Chenguang Zheng <645327136@qq.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: bjf-frz <frz123db@gmail.com>
 2026-04-21 14:46:05 +08:00
Vensen e076378df1 [Feat][sleepmode] add omni sleepmode and ack protocol (#2022) 2026-04-21 08:19:53 +08:00
ChenWenjing 4124a1f858 [Feature]Support audio streaming input and output-phase2 (#2581) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-04-18 15:42:41 +08:00
Bingyu (Spencer) Liu a5a4998ece [Feature] Support Prefill-Decode disaggregation via vLLM KV transfer (#2220) 
Signed-off-by: LiuBingyu <liubingyu62@gmail.com>
 2026-04-17 18:43:17 +08:00
amy-why-3459 2b70e89535 [Revert] Revert "[Log] Wire stat loggers into AsyncOmniEngine to match AsyncLL… (#2716) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-13 14:55:16 +08:00
Canlin Guo 5d58abb11e [Log] Wire stat loggers into AsyncOmniEngine to match AsyncLLM (#2551) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-04-12 19:54:23 +08:00
Zhou Taichang 2804a85e58 [Rebase] Rebase to vllm v0.19.0 (#2475) 2026-04-04 15:03:32 +08:00
ChenWenjingandHongsheng Liu 728cf6d023 [Feature] add session based audio streaming input (#2208) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-02 22:51:41 +08:00
WeiQing Chen 1eee8435e1 [Bugfix] fix: return 400 for unsupported multi-image edits such as Qwen-Image-Layered (#2298) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-03-30 20:24:57 +08:00
Canlin GuoSamitlishunyang lishunyang12@163.com
 		7217557950 [Refactor] Unify torch profiler for omni and diffusion models (#2099) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: lishunyang lishunyang12@163.com
 2026-03-24 15:25:52 +08:00
knlnguyen1802 65d76a4127 [Feature][RL] Support batching for QwenImage in async mode (#1593) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
 2026-03-23 10:08:24 +00:00
Zhou Taichangandgcanlin a90a7690b8 [Rebase] Rebase to vllm v0.18.0 (#2037) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-21 20:37:38 +08:00
Chenguang ZhengyinpePeiqi YinwuhangPeiqi Yinlinyueqianprincepridelishunyang
 		3158e9d28f [Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring (#1908) 
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Signed-off-by: Peiqi Yin <yinpeiqi809@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: Peiqi Yin <yinpeiqi809@gmail.com>
Co-authored-by: wuhang <whlbx@hotmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-03-18 17:33:53 +08:00
Lancer af012babf8 [Bugfix] Fix cpu offload and quantization compatibility (#1473) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-14 18:55:30 +00:00
汪志鹏 e0b71056ba [BugFix]: Fix bagel online inference bug (#1804) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-13 08:58:44 +00:00
Yueqian Lin e1efcb7560 Fix duplicate get_supported_tasks definition in async_omni.py (#1825) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-11 19:43:37 -04:00
Yueqian Lin a6b5519616 [Bugfix] Report supported tasks for omni models to skip unnecessary chat init (#1645) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-12 01:06:20 +08:00
Ting FU 6ac85d4607 [Feat] Enable expert parallel for diffusion MoE layers (#1323) 
Signed-off-by: Semmer2 <semmer@live.cn>
 2026-03-11 10:45:36 +08:00
Samit 155856ff30 [Perf] Reduce IPC overhead for single-stage diffusion serving for Wan2.2 (#1715) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-03-10 00:03:51 +13:00
Zhou Taichang汪志鹏Hongsheng LiuCyrus Leung
 		89b527e533 Dev/rebase v0170 (#1639) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-09 17:29:38 +08:00
knlnguyen1802andSamit 92e1a54419 [RL] Support collective rpc api to entrypoint && Support custom input output (#1646) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-03-09 10:12:43 +08:00
Didan Deng 30191ed112 [Feature]: Add vae-patch-parallel CLI argument in online serving (#1716) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-07 01:45:02 +08:00
WeiQing Chen 83fe7d005b [Model] support LTX-2 text-to-video image-to-video (#841) 
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-03-05 23:31:48 +08:00
Samit 22a51a7cdb [Feature][Wan2.2] Speed up diffusion model startup by multi-thread weight loading (#1504) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-03-02 14:53:25 +00:00
Canlin GuoandHongsheng Liu f53152d340 [Feature] Support HSDP for diffusion models (#1339) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 21:31:10 +08:00
Juan Pablo ZuluagapabloHongsheng Liu
 		03c01e0415 [Qwen3TTS][Feat] Streaming output (#1438) 
Signed-off-by: pablo <pablo@agigo.ai>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: pablo <pablo@agigo.ai>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 21:02:35 +08:00
wuhang汪志鹏Copilot
 		36b8f80266 [Feature] Support Stage Based Deployment CLI (#939) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-24 16:55:23 +08:00
Sy03andHongsheng Liu efbe4116ff [Feature] : Support disaggregated inference pipeline for Qwen3_TTS (#1161) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-21 07:36:44 +08:00
knlnguyen1802 9994594782 [Misc] Support WorkerWrapperBase and CustomPipeline for Diffusion Worker (#764) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
 2026-02-12 19:23:51 +08:00
Yuanheng ZhaoandSamit 8bf849c399 Refactor CPU Offloading Backend Pattern (#1223) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-02-12 17:29:55 +08:00
Junhong Liu 82fd2c3637 Fix yield token metrics and opt metrics record stats (#1292) 2026-02-12 00:52:05 +08:00
Junhong LiuandHongsheng Liu 9af6fb9a1f [Feature] Opt metrics structure (#891) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-09 19:17:32 +08:00
Junhong LiuandHongsheng Liu 959b3ca8c8 [Bugfix] E2E metric incorrect qwen3-omni with async chunk feature (#1018) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-04 15:27:30 +08:00
Zhou Taichangand汪志鹏 70ef548448 Dev/rebase v0.15.0 (#1159) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-03 12:24:45 +08:00
Yuanheng ZhaoHongsheng LiuCopilot
 		2923d5a91e [Feature] Support DiT Layerwise (Blockwise) CPU Offloading (#858) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-30 17:35:09 +08:00
amy-why-3459andRein Yang dd518973de [BugFix] Fix the accuracy issue of multimodal input. (#1020) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
 2026-01-29 19:30:11 +08:00
Samit c0beebbfe3 [Feature] Support cache-dit for Wan 2.2 inference (#1021) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-01-28 20:07:50 +08:00
Zeyu Huang | 黃澤宇 d64bbde561 [Frontend][Model] Support batch request with refined OmniDiffusionReq… (#797) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-01-28 16:07:42 +08:00
Andy Zhoudongbo910220Samit
 		5037af1033 [Feature] Diffusion LoRA Adapter Support (PEFT compatible) for vLLM alignment (#758) 
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
Signed-off-by: Andy Zhou <jzhoubc@connect.ust.hk>
Signed-off-by: dongbo910220 <1275604947@qq.com>
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-01-27 10:22:40 +08:00
amy-why-3459Rein YangCHENHongsheng Liu
 		52bf1e2f40 [Feature]Support async computation and communication across stages by chunks (#727) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-26 11:21:24 +08:00
D!NE$H 5e7035e183 [Feature]: Add CFG param to online serving (#824) 
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
 2026-01-19 04:23:15 +00:00
Yuhan Liu e213bdd0a4 [BugFix]Remove duplicate error handling for request results (#781) 
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
 2026-01-14 20:08:23 +08:00
JustQJ 9cdf592782 [Misc] Enable tensor_parallel_size argument with online serving cmd (#761) 
Signed-off-by: TangPeng <85704592@qq.com>
 2026-01-13 15:55:25 +08:00
Prajwal Aandzjy0516 779f5987d9 [feature] cpu offloading support for diffusion (#497) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Prajwal A <prajwalanagani@gmail.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-01-12 10:41:18 +00:00
Ziming Huang cddcc400d3 [BugFix] Fix stage engine outputs mismatch bug in online batching (#691) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-09 04:01:57 +13:00
Junhong Liu bb13467adc [Bug fix] fix e2e_total_tokens and e2e_total_time_ms (#648) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-01-08 20:25:04 +08:00