anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,511 Commits 235 Branches 31 Tags
2ce15ab7a023d4fb800629e36c06bbe75a4324e2
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Zhou TaichangandClaude Opus 5 2ce15ab7a0 [Rebase] Rebase to vllm 0.27.0 (#5976) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-11 15:23:46 +00:00
Zhou TaichangandClaude Fable 5 2c4e9a945b [BugFix] Make pure-TP KV receive fallback all-or-nothing across ranks (#5636) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-01 15:43:48 +00:00
Zhou TaichangtzhouamClaude Fable 5
 		a951dd0cb4 [BugFix] Never dispatch diffusion with an incomplete CFG companion bundle (#5482) 
Signed-off-by: tzhouam <tzhouam@gmail.com>
Co-authored-by: tzhouam <tzhouam@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-30 08:17:24 -07:00
Zhou TaichangtzhouamClaude Fable 5Gao Han
 		c2b55de5b9 [Test] Give the logprobs scheduler doubles the vLLM 0.26 contracts (#5483) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: tzhouam <tzhouam@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-07-28 15:25:57 +00:00
Zhou TaichangandClaude Opus 5 66a9b0c84f [Bugfix] Select the AR-Diffusion engine in the DreamZero CFG-parallel deploy config (#5439) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-28 17:14:26 +08:00
Zhou TaichangClaude Fable 5tzhouam
 		cbab1d4d8f [Rebase] Rebase to vllm 0.26.0 (#5443) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: tzhouam <tzhouam@gmail.com>
 2026-07-27 23:35:49 +08:00
Zhou TaichangandClaude Opus 4.8 776424f0de [Refactor][WIP] Add missing test for modelrunner (G1/N) (#5333) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-27 16:24:47 +08:00
Zhou TaichangandClaude Opus 4.8 d3c47efc4f [Rebase] Rebase to vllm v0.25.0 (#5042) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-12 23:39:53 +08:00
Zhou Taichang 86bdcaf3d5 [BDE] Phase 1: engine-level KV cache management for DreamZero (RFC #4366) (#4534) 2026-07-05 07:17:58 +08:00
Zhou Taichang a560ed184d [Rebase] Rebase to vllm v0.24.0 (#4709) 2026-06-30 19:41:06 +08:00
Zhou TaichangandClaude Opus 4.8 8a6786731e [CI][Rebase] Fix Ready/Merge pipeline upload by stripping source_file_dependencies (#4532) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-22 15:22:11 +00:00
Zhou TaichangandClaude Opus 4.8 a62420450c [CI][Rebase]: run full e2e suite in rebase Ready/Merge pipelines in the rebase pipeline[skip-ci] (#4478) 
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-16 16:49:19 +08:00
Zhou Taichang 7b837944a1 [Rebase] Rebase to vllm 0.23.0 (#4286) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-06-13 16:43:02 +00:00
Zhou Taichang af02713952 [Bugfix] harden diffusion model prefetch against transformers v5 shard-resolution race (#4076) 2026-06-04 12:14:25 +08:00
+1
Zhou TaichangClaude Opus 4.7skfZheng WengangHongsheng LiuWeiQing ChenSYLARYueqian Linnainiu258JasonYueqian Lin
 		f92d84fcf9 [Rebase] Rebase to vllm releases/v0.22.0 (#3891) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: nainiu258 <cperfect02@163.com>
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com>
Co-authored-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-06-01 11:08:12 +08:00
+20
Zhou TaichangClaude Opus 4.7NumberWandsinghviDnoobYueqian LinHongsheng Liuknlnguyen1802Samitrein yangNick Caozhumingjue138Ricardo Noriegalyj-jjjCursorgcanlinwangyuweizhoublueweizhoubluedengyunyang汪志鹏WeiQing ChenJie LiuYueqian LinNATUREbjf-frzamy-why-3459Y. FisherZeyu Huang | 黃澤宇Copilot Autofix powered by AI
 		eb4e60ee64 [Rebase] Rebase to vllm v0.21.0 (#3530) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Dnoob <dxpouo@gmail.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: rein yang <ruiruyang2@gmail.com>
Signed-off-by: Nick Cao <ncao@redhat.com>
Signed-off-by: zhumingjue <zhumingjue@huawei.com>
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com>
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: weizhoublue <weizhoublue@github.com>
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io>
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: NumberWan <wantszkin2003@gmail.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Dnoob <dxpouo@gmail.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com>
Co-authored-by: Ricardo Noriega <rnoriega@redhat.com>
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: weizhoublue <45163302+weizhoublue@users.noreply.github.com>
Co-authored-by: weizhoublue <weizhoublue@github.com>
Co-authored-by: dengyunyang <584797741@qq.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: NATURE <wzliu@connect.hku.hk>
Co-authored-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Y. Fisher <yukexiong1@huawei.com>
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-05-16 18:18:43 +08:00
Zhou TaichangandYuanheng Zhao 7f2b1e9386 [Rebase] Rebase to vllm 0.20.0 (#3232) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com>
 2026-04-30 09:56:59 +12:00
Zhou Taichang edb4f2fdcf [Test] Add ModelRunner V2 with Qwen3-TTS Base E2E Test to CI pipeline (#2321) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-04-17 20:08:23 +08:00
Zhou Taichang 2804a85e58 [Rebase] Rebase to vllm v0.19.0 (#2475) 2026-04-04 15:03:32 +08:00
Zhou TaichangandGao Han d6e338f9d6 [Doc] Update docs and dockerfiles for rebase of vllm v0.18.0 (#2038) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-03-21 13:21:33 +00:00
Zhou Taichangandgcanlin a90a7690b8 [Rebase] Rebase to vllm v0.18.0 (#2037) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-21 20:37:38 +08:00
Zhou Taichang 05d8de813e [Bugfix] Remove duplicated config keyword max batch size (#1851) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-20 16:13:24 +08:00
Zhou Taichang汪志鹏Hongsheng LiuCyrus Leung
 		e449cc411e [Doc][skip-ci] Update installation instructions (#1762) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-10 22:29:16 +08:00
Zhou Taichang汪志鹏Hongsheng LiuCyrus Leung
 		89b527e533 Dev/rebase v0170 (#1639) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-09 17:29:38 +08:00
Zhou Taichang 90a48f4444 [Debug] Enable curl retry aligned with openai (#1539) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 11:56:42 +08:00
Zhou Taichang fe0c02ee78 [Debug] Merge vllm pull 35368 (#1534) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-28 10:39:25 +08:00
Zhou Taichang bc13f7cf38 [Doc] Update installation instructions for vllm 0.16.0 (#1505) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-27 16:06:30 +08:00
Zhou Taichang 986b356077 [CI] Update Dockerfile for vllm-omni CI image and remove obsolete dep… (#1491) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-26 14:36:03 +08:00
Zhou Taichang 8e950daf6a [Debug] Multi-Request for Qwen 3 Omni use_audio_in_video (#1433) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-26 09:26:25 +08:00
Zhou Taichang 296d995ec2 [CI] Remove overwhelming debug log (#1463) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-25 12:06:01 +08:00
Zhou Taichang b42287d14b [Doc] Update GPU installation commands (#1434) 2026-02-23 13:54:57 +08:00
Zhou Taichang汪志鹏Hongsheng LiuIsotr0pyZJY0516youkaichao
 		75770c9309 [Rebase] Rebase to vllm v0.16.0 (#1357)
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Details 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2026-02-13 18:05:07 +08:00
Zhou Taichang a854f0b7ff [Debug] Add trigger to concurrent stage init (#1274) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 14:18:18 +08:00
Zhou Taichang f1960d8dfd [Feature] Enable use_audio_in_video for Qwen 3 Omni Online (#1198) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-04 20:48:25 +08:00
Zhou Taichang d6f93b0f37 [Debug] Correct Unreasonable Long Timeout (#1175) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-03 08:49:51 +00:00
Zhou Taichang 7fcbaa794c [Debug] Clear Dockerfile.ci to accelerate build image (#1172) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-03 13:43:45 +08:00
Zhou Taichangand汪志鹏 70ef548448 Dev/rebase v0.15.0 (#1159) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-03 12:24:45 +08:00
Zhou Taichangandroot def39561ae [Debug] Update GLM-Image Pipeline (#1049) 
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-29 16:03:27 +08:00
Zhou Taichang e3b6254090 debug Qwen TTS (#902) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-01-22 15:28:56 +00:00
Zhou Taichang d2a72f380f [Doc] Update rebase doc (#878) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-21 03:59:02 +00:00
+13
Zhou TaichangJaredforRealrootJustQJXU MingshiSihyeon JangJiangyun ZhucatcatZiming HuangYuhan LiuAliciaSamitYueqian LinWeiQing ChenPeiqi YinDidan DengHongsheng Liuwangyuwangyu31577汪志鹏John Liu BUAAD!NE$Hroot
 		776c3a794e Dev/rebase 0.14.0 (#847) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-21 00:25:25 +08:00
Zhou Taichangandroot 2e13bddf87 [Debug] Clean code in Qwen 3 Omni and add warning for talker temperature. (#688) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-12 23:13:45 +13:00
Zhou Taichang 4a26b2eca9 [Docs] Remove multi-request streaming design document and update ray-based execution documentation structure (#641) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-05 19:09:05 +08:00
Zhou Taichang ec8ff95e22 Dev/add i2i bash (#623) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-01-05 14:17:08 +08:00
Zhou TaichangCopilotCanlin GuoPeiqi YinHongsheng LiuSYLARJiangyun ZhudengyunyangSamit
 		f0b8d3db82 [Feature] Control the stage init timeout threshold by --stage-init-timeout (#393) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Li Shunyang <gmsv2544@nus.edu.sg>
Signed-off-by: Li Shunyang <lishunyang12@163.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: dengyunyang <584797741@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2025-12-30 23:37:10 +08:00
Zhou Taichang 59ae174bba [Debug] Debug qwen3 mix modality output empty string (#431) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-12-30 15:32:15 +08:00
Zhou TaichangRein YangIsotr0py
 		8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-19 16:42:02 +08:00
Zhou Taichang 1f4e1badc6 [Benchmark] Benchmark Running Samples for Qwen3 Omni and Dataset Preparation (#212) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-12-18 03:51:37 +00:00
Zhou TaichangandGao Han c70b167f70 [Engine] Remove Diffusion_XX which duplicates with Generation_XX (#163) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-12-05 03:04:23 +00:00
Zhou Taichang 228f4e5070 [Bug-fix] Fix Bugs in Qwen3/Qwen2.5 Omni Rebased Support (#114) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-30 18:15:10 +08:00
tzhouam 8f6280f4f6 adjust the doc 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-12 15:51:56 +08:00
tzhouam 26fcae5def Debug and Update doc 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 14:49:45 +08:00
tzhouam c12bd006bc Make Example Easier To Use 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 11:56:08 +08:00
tzhouam bc69d8e931 rename util further rename imports 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:57:15 +08:00
tzhouam 12cc2cd245 rename util 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:55:57 +08:00
tzhouam b943abb22f add data download instructions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:49:53 +08:00
tzhouam edae58b137 modify code based on review suggestions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:03:26 +08:00
tzhouam 5052bfa0b9 Merge branch 'feat/multi-request-stream-new' of https://github.com/tzhouam/vllm-omni into feat/multi-request-stream-new 2025-11-10 16:43:11 +08:00
tzhouam c83d223e91 Squash 7 commits: 36ada0478a 0ef1fefd45 519c140d20 c18fc40f10 5f3f53a39d da385277a2 edefcc244b 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 16:42:58 +08:00
tzhouam edefcc244b Merge branch 'feat/multi-request-stream-new' of https://github.com/tzhouam/vllm-omni into feat/multi-request-stream-new 2025-11-10 16:37:49 +08:00
tzhouam da385277a2 Revert "Revert "Update run script to include initialization sleep time and support for loading prompts from a .pt file."" 
This reverts commit 5f3f53a39d.
 2025-11-10 16:37:37 +08:00
tzhouam 5f3f53a39d Revert "Update run script to include initialization sleep time and support for loading prompts from a .pt file." 
This reverts commit c18fc40f10.
 2025-11-10 16:36:55 +08:00
tzhouam c18fc40f10 Update run script to include initialization sleep time and support for loading prompts from a .pt file. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 16:36:00 +08:00
tzhouam 519c140d20 Update run script to include initialization sleep time and support for loading prompts from a .pt file. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 15:47:28 +08:00
tzhouam 0ef1fefd45 add prompt file 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 15:44:44 +08:00
tzhouam 36ada0478a add back the sleep time for the stage init to avoid single card conflict 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 15:07:01 +08:00
tzhouam 08aef1f964 remove text output for audio output 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 14:41:44 +08:00
tzhouam 8e8783d78a Refactor end-to-end inference script and enhance argument parsing. Added support for loading prompts from a .pt file, introduced new command-line arguments for initialization and output handling, and improved error handling for prompt loading. Removed deprecated files related to previous implementations. 2025-11-10 11:58:47 +08:00
tzhouam c5631ca753 Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class. 2025-11-07 18:05:00 +08:00
tzhouam 3c2ad1bf12 debug ar model runner 2025-11-07 16:25:48 +08:00
tzhouam 6a7a27801b Port changes from feat/multi-request-stream (vs main) 2025-11-07 15:48:47 +08:00
Zhou Taichang 6d34a3ed11 Merge pull request #46 from Gaohan123/fix_install_v2 
[Bugfix] Further supplement installation guide
 2025-10-30 17:25:14 +08:00
tzhouam f6158e7576 debug 2025-10-24 18:01:02 +08:00
tzhouam 605d1de013 Update entry points in vllm_omni module 
- Reintroduced AsyncOmniLLM to the main entry points.
- Added import for patch module.
- Cleaned up comments regarding AsyncOmniLLM support.
 2025-10-24 17:54:19 +08:00
tzhouam d45f8a6be7 debug typo 2025-10-24 17:47:49 +08:00
tzhouam 61b344dc39 Add import for OmniModelsRegistry in model executor initialization 2025-10-24 17:44:31 +08:00
tzhouam 3b1324ea39 Merge branch 'main' into end2end_fixup 2025-10-24 17:41:03 +08:00
tzhouam 3265e857eb Refactor OmniLLM and MultimodalOutputProcessor initialization 
- Removed AsyncOmniLLM from the main entry points as it is not supported yet.
- Updated the constructor of MultimodalOutputProcessor to set a default value for engine_core_output_type.
- Adjusted output type handling in various methods to utilize the new default engine_core_output_type for consistency.
 2025-10-24 17:15:34 +08:00
tzhouam eb006f66e5 Enhance OmniScheduler error handling with logging 
- Added logging for exceptions when wrapping scheduled requests in OmniScheduler.
- Removed unnecessary blank lines for cleaner code structure.
 2025-10-24 12:00:36 +08:00
tzhouam 87b3ae7e70 Merge branch 'feat/omni-schedulers' of https://github.com/tzhouam/vllm-omni into feat/omni-schedulers 2025-10-24 11:57:45 +08:00
tzhouam c23d24ca19 Refactor comments in diffusion_scheduler.py for clarity and consistency 
- Translated comments from Chinese to English for better accessibility.
- Improved clarity of comments related to request handling and scheduling logic.
- Ensured comments align with the code functionality and maintain a uniform style.
 2025-10-24 11:57:39 +08:00
Zhou Taichang 582639c77a Merge branch 'main' into feat/hidden-state-input-request-input-processor-patch 
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 11:52:45 +08:00
tzhouam 63c72f4c2b Merge branch 'feat/hidden-state-input-request-input-processor-patch' of https://github.com/tzhouam/vllm-omni into feat/hidden-state-input-request-input-processor-patch 2025-10-24 11:40:49 +08:00
tzhouam a97409efef Add dtype mapping method in OmniProcessor 
- Introduced a static method `_dtype_to_name` to map PyTorch data types to their string representations.
- Updated the `dtype_str` assignments in the `OmniProcessor` class to utilize the new mapping method for improved readability and maintainability.
 2025-10-24 11:40:43 +08:00
tzhouam 4cd3b36fa0 Refactor OmniProcessor and update parsing logic 
- Removed redundant imports of `torch` and `numpy` in `OmniProcessor` class.
- Improved comments in `parse_singleton_prompt_omni` function for clarity on token handling.
- Ensured `current_platform` is imported only once for better code organization.
 2025-10-24 11:27:30 +08:00
Zhou TaichangandCopilot d0423129f8 Apply suggestion from @Copilot 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 11:14:58 +08:00
Zhou TaichangandCopilot b04f114773 Apply suggestion from @Copilot 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 11:14:41 +08:00
tzhouam ee4822327b Remove unnecessary numpy imports and update return type in extract_multimodal_outputs method in OmniGPUModelRunner 
- Eliminated redundant numpy import statements to streamline the code.
- Changed the return type of extract_multimodal_outputs from dict to tuple for better clarity on output structure.
 2025-10-24 11:08:42 +08:00
tzhouam 6b196dee24 Enhance error handling in output processing for multimodal tensors 
- Added logging for exceptions during tensor movement to CPU in OmniRequestState and MultimodalOutputProcessor.
- Improved robustness by ensuring the output pipeline continues without crashing on errors.
- Updated comments for clarity on error handling behavior.
 2025-10-24 11:01:32 +08:00
Zhou TaichangandCopilot 9ebc1dd652 Apply suggestion from @Copilot 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 10:41:54 +08:00
tzhouam 634915652d Remove unused code and comments in GPUDiffusionModelRunner for improved clarity 
- Deleted the `has_kv_transfer_group` import as it was not utilized.
- Cleaned up commented-out code related to model methods, focusing on the forward method.
- Added a TODO comment to implement the diffuse method for additional model support.
 2025-10-24 10:34:41 +08:00
tzhouam ae3b7504b2 Merge branch 'feat/qwen_2_5_omni_models' of https://github.com/tzhouam/vllm-omni into feat/qwen_2_5_omni_models 2025-10-24 10:02:58 +08:00
tzhouam 52905a0d17 Enhance Qwen2.5-Omni model by defining TALKER_CODEC_EOS_TOKEN_ID and TALKER_CODEC_BOS_TOKEN_ID constants. Update audio generation logic to use these constants for improved clarity and maintainability. Refactor tensor initialization to utilize the talker's hidden size. Update comments for better understanding and consistency. 2025-10-24 10:02:51 +08:00
Zhou TaichangandCopilot 8a6e298243 Apply suggestion from @Copilot 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 09:43:45 +08:00
tzhouam 4acaef3ccb Merge branch 'feat/omni-schedulers' of https://github.com/tzhouam/vllm-omni into feat/omni-schedulers 2025-10-24 09:41:29 +08:00
tzhouam c37cd6f0e3 delete unused imports 2025-10-24 09:41:20 +08:00
tzhouam 28ae542eed match runner, worker name with file name 2025-10-23 14:35:23 +08:00
tzhouam a3a77b6cc4 Qwen2.5-Omni: squash 5 commits (add components, refactor, load thinker embedding from weights, remove unused code) 
Squashed commits:\n- 4e2d812 Add Qwen2.5-Omni model components\n- 351f833 load the thinker embedding from the weights, not rely on local .pt files\n- 2c32533 Refactor Qwen2.5-Omni model files\n- 9163a60 remove unused codes\n- 192bdf8 remove
 2025-10-23 14:24:03 +08:00
tzhouam ea16546ae9 Refactor GPU diffusion model classes for clarity and consistency 
- Renamed `DiffusionModelRunner` to `GPUDiffusionModelRunner` for better clarity in class purpose.
- Updated `DiffusionGPUWorker` to utilize the renamed `GPUDiffusionModelRunner`.
- Simplified output handling in `GPUDiffusionModelRunner` by removing unnecessary checks, enhancing code readability.
 2025-10-23 11:04:10 +08:00
tzhouam 3a27ff51d0 add back the diffusion engine as a placeholder for the actural diffusion engine in the future 2025-10-23 10:04:27 +08:00