Commit Graph
70 Commits
Author SHA1 Message Date
tzhouam 0099eebb3f Refactor OmniEngineArgs and AsyncOmniEngineArgs to improve compatibility with partial payloads by using getattr for various attributes. Update GPUGenerationModelRunner and OmniGPUModelRunner to replace the activate_lora parameter with num_active_loras for better clarity and backward compatibility. This change enhances the handling of LoRA activation during model runs.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-12 03:33:50 +00:00
tzhouam 7e4a604d4d reabse 2025/2/9
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-09 03:07:43 +00:00
tzhouamandCursor 03c6a58f69 Merge origin/main into dev/vllm-align; resolve omni_stage.py conflicts
Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-09 01:52:31 +00:00
dsinghvi 59024bee62 Process-Scoped GPU Memory Accounting (#1204)
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
2026-02-08 22:43:02 +08:00
tzhouam 24b04f58a0 rebase 2026/2/5 2026-02-05 10:52:28 +00:00
amy-why-3459 5304bc1e16 [Refactor] Refactor async chunk and fix the shape mismatch issue (#1151)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
2026-02-04 00:00:56 +08:00
Zhou Taichangand汪志鹏 70ef548448 Dev/rebase v0.15.0 (#1159)
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
2026-02-03 12:24:45 +08:00
f6cfc0d1e3 [BugFix] Fix Qwen3 Omni talker mtp torch.compile startup error (#1104)
Signed-off-by: ram16g <anlianfengjie@163.com>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: ram16g <anlianfengjie@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-01-30 23:25:49 +08:00
Ziming HuangandHongsheng Liu a909d36415 [Perf] Qwen3 Omni talker mtp optimization (#1005)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-01-29 21:23:19 +08:00
汪志鹏 741f7e2826 [Core]: KV Cache Transfer Encapsulation (#979)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
2026-01-28 23:12:45 +08:00
Canlin Guo c9392dff01 [Hardware] Support platforms and plugin system (#774)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2026-01-28 22:12:52 +08:00
Canlin Guo cb48968294 [Bugfix] Modify output to model_runner_output (#1026)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2026-01-28 18:42:39 +08:00
Canlin Guo cadd77262d [NPU] Upgrade to v0.14.0 (#820)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2026-01-27 16:45:32 +08:00
52bf1e2f40 [Feature]Support async computation and communication across stages by chunks (#727)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-01-26 11:21:24 +08:00
汪志鹏andwzliu 7f821be57b Support Bagel Model (#726)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: wzliu <wzliu@connect.hku.hk>
2026-01-22 23:09:55 +13:00
Gao Han a54c323952 [Model] Support Qwen3-TTS model series (#895)
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
2026-01-22 16:57:37 +08:00
+13 776c3a794e Dev/rebase 0.14.0 (#847)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
2026-01-21 00:25:25 +08:00
Ziming Huang b5cf8806f0 [BugFix] Fix incorrect mrope positions under cuda graph (#803)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
2026-01-16 03:24:21 +13:00
Ziming Huang 1444e1fc45 [Feature] Support Qwen3 Omni talker mtp batch inference (#722)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
2026-01-14 19:07:31 +08:00
Zhou Taichangandroot 2e13bddf87 [Debug] Clean code in Qwen 3 Omni and add warning for talker temperature. (#688)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
2026-01-12 23:13:45 +13:00
汪志鹏 0ec2da08de [BugFix] Fix assuming all stage model have talker (#730)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
2026-01-11 01:36:37 +13:00
Ziming Huang 978e3ad7d4 [Feature] Support Qwen3 Omni talker cudagraph (#669)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
2026-01-09 11:45:58 +08:00
Canlin Guo 46afa046ce [NPU] Support mixed modalities for Qwen3-Omni (#537)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2025-12-31 16:15:20 +08:00
Ziming HuangandHongsheng Liu 1ae0c64fe0 Support Qwen3 Omni thinker cuda graph (#523)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2025-12-30 23:46:13 +13:00
Zhou Taichang 59ae174bba [Debug] Debug qwen3 mix modality output empty string (#431)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2025-12-30 15:32:15 +08:00
Canlin Guo be81443c8a [Bugfix][NPU] Add _model_forward for ModelRunner (#505)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2025-12-28 04:09:34 +08:00
dsinghviandHongsheng Liu df4909630c model forward call logic fixes (#495)
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2025-12-27 14:22:07 +00:00
595e7c0032 [Core] Supports stage abstraction in the diffusion model (#391)
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2025-12-27 01:59:33 +13:00
dsinghviandHongsheng Liu abde501dfa [Misc] Fix type interface mismatch (#478)
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2025-12-26 15:28:49 +08:00
Canlin GuoandHongsheng Liu fe3d9a6a81 [NPU][Upgrade] Upgrade to v0.12.0 (#447)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2025-12-26 14:10:23 +08:00
Canlin Guo e1f9691120 [Clean] Remove the redundant decoding payloads logic (#404)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2025-12-26 04:48:34 +13:00
Peiqi Yin e9e6b3711c [Core] customize process between stages, remove the need of "request_ids" in model forward, rebase to 0.12.0 (#458)
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
2025-12-24 13:26:33 +00:00
8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2025-12-19 16:42:02 +08:00
Lancer 01549fb953 [Diffusion] fix warning in diffusion stage (#332)
Signed-off-by: Lancer <maruixiang6688@gmail.com>
2025-12-16 09:07:16 +00:00
Cyrus Leung c091a03d82 [Chore] Bump Python version and apply formatting changes for v0.12 release (#126)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2025-12-16 15:19:40 +08:00
NATURE f995211da0 [Feature] Omni Connector + ray supported (#215)
Signed-off-by: wzliu <wzliu@connect.hku.hk>
2025-12-10 20:15:03 +08:00
Peiqi Yin bbb5a72871 [BugFix] Fix qwen3omni thinker batching. (#207)
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
2025-12-08 04:44:13 +00:00
Canlin Guo 6079b14040 [NPU][Refactor] Rename Diffusion* to Generation* (#211)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2025-12-05 08:23:37 +00:00
Zhou TaichangandGao Han c70b167f70 [Engine] Remove Diffusion_XX which duplicates with Generation_XX (#163)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Gao Han <gaohan19@huawei.com>
2025-12-05 03:04:23 +00:00
Alicia a33bbcedea [Fix] Resolve comments & update vLLM-Omni name usages. (#122)
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
2025-12-01 08:05:24 +08:00
Canlin Guo ec9e04027d [Feat] Add NPU Backend support for vLLM-Omni (#89)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2025-11-30 20:39:06 +08:00
Zhou Taichang 228f4e5070 [Bug-fix] Fix Bugs in Qwen3/Qwen2.5 Omni Rebased Support (#114)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2025-11-30 18:15:10 +08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
2025-11-30 04:44:01 +08:00
Roger WangandGao Han a5800cd2c2 [Doc] Misc documentation polishing (#98)
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
2025-11-29 20:42:39 +08:00
Gao Han db81ee9f69 [Docs] optimize and supplement docs system (#86)
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
2025-11-28 11:03:52 +08:00
Gao Han 598b4ddb51 [Feature] support multimodal inputs with multiple requests (#76)
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
2025-11-24 16:02:44 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64)
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
2025-11-17 14:44:10 +08:00
tzhouam c5631ca753 Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class. 2025-11-07 18:05:00 +08:00
tzhouam 3c2ad1bf12 debug ar model runner 2025-11-07 16:25:48 +08:00