Commit Graph
49 Commits
Author SHA1 Message Date
Zhou TaichangandClaude Opus 5 2ce15ab7a0 [Rebase] Rebase to vllm 0.27.0 (#5976)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 15:23:46 +00:00
Caspian ChenandSYLAR 593b404539 [Quantization][Qwen2.5-Omni] Support thinker-only ModelOpt NVFP4 W4A4 checkpoints (#5073)
Signed-off-by: Caspian443 <scrisis843@gmail.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
2026-08-09 02:45:57 +08:00
cbab1d4d8f [Rebase] Rebase to vllm 0.26.0 (#5443)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: tzhouam <tzhouam@gmail.com>
2026-07-27 23:35:49 +08:00
Zheng Wengang b9b8282cf5 [BugFix] Couple audio+video mm-cache for use_audio_in_video (#5308)
Signed-off-by: ZhengWG <zwg0606@gmail.com>
2026-07-27 04:29:53 +00:00
Wei 10d3a25916 [Feature] Support per-video audio masks for Qwen Omni (#4656)
Signed-off-by: hbhflw2000 <417911774@qq.com>
2026-07-20 11:39:59 +08:00
Yukim1andHongsheng Liu 793d3e9ab7 [Config] Return VllmOmniConfig from create_from_model (#4818)
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-07-18 09:21:07 +00:00
Zheng Wengang 9fe26cfacb [Bugfix][Model] Fix Qwen3-Omni video metadata handling for use_audio_in_video (#4959)
Signed-off-by: ZhengWG <zwg0606@gmail.com>
2026-07-10 14:24:18 +08:00
akshatvishu d9cf85d5e2 [BugFix] Use KV cache scale mapper in custom loaders (#4810)
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
2026-07-03 04:11:28 +00:00
Ayaka Mikazuki deb9bd4efc Fix Qwen2.5-Omni AutoRound loading (#4781)
Signed-off-by: Ayaka <ayaka@mail.shn.hk>
2026-06-29 07:34:27 +00:00
Isaac Hung e9d4931781 [Core] Remove dead custom_process_input_func hooks in stage input processors (#4531)
Signed-off-by: Isaac Hung <manhei5001@gmail.com>
2026-06-29 05:17:25 +02:00
amy-why-3459 fc7393e485 [Refactor]Refactoring audio_in_video implementation (#3566)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
2026-06-08 14:14:02 +08:00
+1 f92d84fcf9 [Rebase] Rebase to vllm releases/v0.22.0 (#3891)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: nainiu258 <cperfect02@163.com>
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com>
Co-authored-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
2026-06-01 11:08:12 +08:00
NATURE 3a347b10b0 [3/5][core]refactor communication layer: PR 3 of 5, all other models in non async mode (#3719)
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
2026-05-28 14:06:41 +08:00
BeatSeat dab596e468 [Refactor] Unify Snake/SnakeBeta and alias-free activation into common modules (#3886)
Signed-off-by: BeatSeat <wendavid552@gmail.com>
2026-05-27 23:41:35 -05:00
Siyuan Kong 7d9d8aee79 [Bugfix] Fix qwen2_5_omni weight loading (#3598)
Signed-off-by: ksiyuan <ksiyuan@umich.edu>
2026-05-21 16:43:33 +08:00
Nick CaoandClaude dca369d448 [Perf] Remove dead audio_tower and visual from Qwen2.5-Omni talker stage (#3425)
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-05-12 23:24:13 +08:00
Peiqi Yin 5d4b16e6e3 [BugFix] Qwen2.5-Omni streaming code2wav input handling (#3396)
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
2026-05-07 11:00:20 +08:00
Nick CaoandClaude 9de3806624 [Cleanup] Extract shared SinusPositionEmbedding/DiTTimestepEmbedding/timestep_embedding layer (#3285)
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-05-01 20:19:42 +08:00
Zhou TaichangandYuanheng Zhao 7f2b1e9386 [Rebase] Rebase to vllm 0.20.0 (#3232)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com>
2026-04-30 09:56:59 +12:00
2390350533 [Refactor] Standardize data entry key names to {type}.{qualifier} format (#1829)
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-04-25 18:15:38 +08:00
SYLAR cd384d95f3 [Config Refactor 2.5/N] Centralize pipeline registry (#2915)
Signed-off-by: lishunyang <lishunyang12@163.com>
2026-04-20 01:18:03 +08:00
a3903810fe [Config Refactor][2/N] Pipeline + Deploy Config Schema (#2383)
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: xiaohajiayou <75477391+xiaohajiayou@users.noreply.github.com>
Co-authored-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-04-19 14:36:00 +12:00
Yiyang "Ian" LiuandCopilot 53a9cf49a6 fix: do not apply FP8 quant config to vision/audio encoders for pre-quantized checkpoints (#2702)
Signed-off-by: Yiyang Liu <37043548+ianliuy@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-14 13:52:32 +00:00
Junhong Liu f3d078e9f2 [CI] Add online e2e test for qwen2.5 omni (#1668)
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
2026-03-25 14:14:34 +08:00
rein yang 3cdf28828e [Feature] support to change the speaker of qwen3-omni (#1963)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
2026-03-24 13:53:24 +08:00
Zhou Taichangandgcanlin a90a7690b8 [Rebase] Rebase to vllm v0.18.0 (#2037)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
2026-03-21 20:37:38 +08:00
Zhou Taichang fe0c02ee78 [Debug] Merge vllm pull 35368 (#1534)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-28 10:39:25 +08:00
Junhong Liu 42d86af1f1 Fix no embed text spk tokens (#1540)
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
2026-02-28 09:33:58 +08:00
Zhou Taichang 8e950daf6a [Debug] Multi-Request for Qwen 3 Omni use_audio_in_video (#1433)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-26 09:26:25 +08:00
75770c9309 [Rebase] Rebase to vllm v0.16.0 (#1357)
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
2026-02-13 18:05:07 +08:00
Zhou Taichangand汪志鹏 70ef548448 Dev/rebase v0.15.0 (#1159)
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
2026-02-03 12:24:45 +08:00
Fanli LinandCopilot bfbf3e5b69 [Feat] Support XPU Backend in vLLM-Omni (#191)
Signed-off-by: Fanli Lin <fanli.lin@intel.com>
Signed-off-by: Fanli Lin <fanli0116@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-30 04:33:05 +13:00
Canlin Guo c9392dff01 [Hardware] Support platforms and plugin system (#774)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2026-01-28 22:12:52 +08:00
Yuanheng Zhao 2f5e32477c [NFC] Remove redundant torch.no_grad in models and pipelines (#854)
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
2026-01-27 12:36:54 +00:00
John Liu BUAA c3894585b5 [Perf] Fuse Q/K/V Linear with QKVParallelLinear in Qwen2.5Omni DiTAttention (#884)
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
2026-01-21 09:24:05 +00:00
+13 776c3a794e Dev/rebase 0.14.0 (#847)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
2026-01-21 00:25:25 +08:00
John Liu BUAA bb24e07a7a [Perf] Optimize the Qwen2.5-Omni Model thinker-to-talker-proj with nn.Linear (#825)
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
2026-01-17 19:47:11 +08:00
Peiqi Yin 35f994e255 [BugFix] Qwen2.5-omni supress end token and won't stop (#773)
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
2026-01-16 00:41:59 +08:00
Zhou Taichangandroot 2e13bddf87 [Debug] Clean code in Qwen 3 Omni and add warning for talker temperature. (#688)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
2026-01-12 23:13:45 +13:00
dengyunyang b60030092c [BugFix] token2wav code out of range (#655)
Signed-off-by: dengyunyang <584797741@qq.com>
2026-01-06 11:05:08 +08:00
Dong WandHongsheng Liu c1ebb60747 [Bugfix]Fix Qwen 2.5 omni hardcoded max_mel_frames to resolve shape mismatch (#543)
Signed-off-by: Dong Wang <dongw2019@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-01-01 17:12:03 +13:00
Zhou Taichang 59ae174bba [Debug] Debug qwen3 mix modality output empty string (#431)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2025-12-30 15:32:15 +08:00
Peiqi Yin e9e6b3711c [Core] customize process between stages, remove the need of "request_ids" in model forward, rebase to 0.12.0 (#458)
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
2025-12-24 13:26:33 +00:00
Peiqi Yin da7f25e14b [BugFix] Qwen3-Omni / Qwen2.5-Omni thinker batching with audio input (#451)
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
2025-12-24 18:43:00 +08:00
8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2025-12-19 16:42:02 +08:00
Cyrus Leung c091a03d82 [Chore] Bump Python version and apply formatting changes for v0.12 release (#126)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2025-12-16 15:19:40 +08:00
Canlin Guo ec9e04027d [Feat] Add NPU Backend support for vLLM-Omni (#89)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2025-11-30 20:39:06 +08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
2025-11-30 04:44:01 +08:00
Gao Han 598b4ddb51 [Feature] support multimodal inputs with multiple requests (#76)
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
2025-11-24 16:02:44 +08:00