tzhouam
22e01c416b
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-02-25 08:15:55 +00:00
Zhou Taichang
296d995ec2
[CI] Remove overwhelming debug log ( #1463 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-25 12:06:01 +08:00
tzhouam
861698f88f
Update Dockerfile for vllm wheel version and enhance SHM connector tests
...
- Updated Dockerfile to use a new vllm wheel version.
- Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions.
- Added handling for speculative configuration in GPU model runners to delay clearing connector metadata.
- Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-24 05:45:48 +00:00
tzhouam
bf69cd94bb
Rebase 2026/02/23
...
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-23 03:36:01 +00:00
Sy03 and Hongsheng Liu
efbe4116ff
[Feature] : Support disaggregated inference pipeline for Qwen3_TTS ( #1161 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-21 07:36:44 +08:00
75770c9309
[Rebase] Rebase to vllm v0.16.0 ( #1357 )
...
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2026-02-13 18:05:07 +08:00
amy-why-3459
5304bc1e16
[Refactor] Refactor async chunk and fix the shape mismatch issue ( #1151 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-02-04 00:00:56 +08:00
Zhou Taichang and 汪志鹏
70ef548448
Dev/rebase v0.15.0 ( #1159 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-02-03 12:24:45 +08:00
f6cfc0d1e3
[BugFix] Fix Qwen3 Omni talker mtp torch.compile startup error ( #1104 )
...
Signed-off-by: ram16g <anlianfengjie@163.com >
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Co-authored-by: ram16g <anlianfengjie@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-30 23:25:49 +08:00
Ziming Huang and Hongsheng Liu
a909d36415
[Perf] Qwen3 Omni talker mtp optimization ( #1005 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-29 21:23:19 +08:00
52bf1e2f40
[Feature]Support async computation and communication across stages by chunks ( #727 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-26 11:21:24 +08:00
+13
776c3a794e
Dev/rebase 0.14.0 ( #847 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Signed-off-by: root <root@hk01dgx039.cm.cluster >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: TangPeng <85704592@qq.com >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: iwzbi <wzbi@zju.edu.cn >
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com >
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com >
Signed-off-by: Dinesh G <G.Dinesh@ibm.com >
Signed-off-by: gDINESH13 <dinesh13g@gmail.com >
Signed-off-by: root <root@hk01dgx028.cm.cluster >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: JaredforReal <w13431838023@gmail.com >
Co-authored-by: root <root@hk01dgx039.cm.cluster >
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com >
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com >
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com >
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com >
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
2026-01-21 00:25:25 +08:00
Ziming Huang
b5cf8806f0
[BugFix] Fix incorrect mrope positions under cuda graph ( #803 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-16 03:24:21 +13:00
Ziming Huang
1444e1fc45
[Feature] Support Qwen3 Omni talker mtp batch inference ( #722 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-14 19:07:31 +08:00
汪志鹏
0ec2da08de
[BugFix] Fix assuming all stage model have talker ( #730 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-01-11 01:36:37 +13:00
Ziming Huang
978e3ad7d4
[Feature] Support Qwen3 Omni talker cudagraph ( #669 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-09 11:45:58 +08:00
Ziming Huang and Hongsheng Liu
1ae0c64fe0
Support Qwen3 Omni thinker cuda graph ( #523 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-30 23:46:13 +13:00
Zhou Taichang
59ae174bba
[Debug] Debug qwen3 mix modality output empty string ( #431 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2025-12-30 15:32:15 +08:00
dsinghvi and Hongsheng Liu
abde501dfa
[Misc] Fix type interface mismatch ( #478 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-26 15:28:49 +08:00
Canlin Guo
e1f9691120
[Clean] Remove the redundant decoding payloads logic ( #404 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2025-12-26 04:48:34 +13:00
Peiqi Yin
e9e6b3711c
[Core] customize process between stages, remove the need of "request_ids" in model forward, rebase to 0.12.0 ( #458 )
...
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
2025-12-24 13:26:33 +00:00
8939eb835d
[Rebase] Rebase to vllm 0.12.0 ( #335 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-19 16:42:02 +08:00
Lancer
01549fb953
[Diffusion] fix warning in diffusion stage ( #332 )
...
Signed-off-by: Lancer <maruixiang6688@gmail.com >
2025-12-16 09:07:16 +00:00
Cyrus Leung
c091a03d82
[Chore] Bump Python version and apply formatting changes for v0.12 release ( #126 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-12-16 15:19:40 +08:00
Zhou Taichang
228f4e5070
[Bug-fix] Fix Bugs in Qwen3/Qwen2.5 Omni Rebased Support ( #114 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2025-11-30 18:15:10 +08:00
R2-Y and Rein
0a4e5a8710
[Feature] add support for Qwen3-omni ( #55 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein <rein@Reins-MacBook-Air.local >
2025-11-30 04:44:01 +08:00
Gao Han
db81ee9f69
[Docs] optimize and supplement docs system ( #86 )
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-28 11:03:52 +08:00
Gao Han
598b4ddb51
[Feature] support multimodal inputs with multiple requests ( #76 )
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-24 16:02:44 +08:00
Jiangyun Zhu
5707fc78d5
[CI] fix ci and format existing code ( #71 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2025-11-18 20:54:26 +08:00
Gao Han
c5a89302f1
[Feature] Support online inference ( #64 )
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-17 14:44:10 +08:00
Gaohan123
6e1478222e
Dependency refractored to vLLM v0.11.0
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-04 22:06:41 +08:00
Gaohan123
171f3c1ea7
fix conflicts
2025-10-24 16:49:19 +08:00
tzhouam
ee4822327b
Remove unnecessary numpy imports and update return type in extract_multimodal_outputs method in OmniGPUModelRunner
...
- Eliminated redundant numpy import statements to streamline the code.
- Changed the return type of extract_multimodal_outputs from dict to tuple for better clarity on output structure.
2025-10-24 11:08:42 +08:00
tzhouam
4609afa613
Refactor sampled_token_ids assignment in OmniGPUModelRunner
...
- Simplified the assignment of sampled_token_ids by removing the conditional check for "model_stage".
- Ensured consistent handling of sampled_token_ids for improved clarity and maintainability.
2025-10-22 17:42:29 +08:00
tzhouam
54977bd590
Add OmniGPUModelRunner and OmniModelInputForGPU classes
...
- Introduced OmniGPUModelRunner for enhanced GPU model execution with state management.
- Added OmniModelInputForGPU and its builder to support additional information in model inputs.
- Implemented methods for updating request states and handling multimodal inputs.
- Enhanced input preparation for CUDA graph execution and LoRA integration.
2025-10-22 15:33:11 +08:00
tzhouam