Taichang Zhou and Claude Opus 4.8
9678ae58cf
[MR V2] Migrate to Model Runner V2
...
Squashed migration rebased onto current main. Collapses the 46-commit
dev/migrate-MR-v2 history (10 merge commits + superseded pre-rebase work)
into a single commit, so the diff reflects only the genuine MR-V2 feature
(worker_v2/ runners + model_states, omni schedulers, stage engine pipeline,
pooling_output serialization bridge). Content already merged to main
separately (step_audio2, tts_adapters, npu/_310p, etc.) drops out.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-06-20 07:19:40 +00:00
8f869c8ea5
[Refactor] Output Processor Phase 2: separate multimodal output channel ( #1601 ) ( #2744 )
...
Signed-off-by: Megha Agarwal <agarwalmegha1308@gmail.com >
Signed-off-by: meghaagr13 <divyanshsinghvi@gmail.com >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: meghaagr13 <divyanshsinghvi@gmail.com >
Co-authored-by: Boao Shi <aoibosh@connect.hku.hk >
Co-authored-by: Taichang Zhou <tzhouam@connect.ust.hk >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-06-12 15:19:14 +08:00
57227dc7f3
[Perf][Bugfix] qwen3-tts hot path: prefix-cache OOM guards + talker/orchestrator micro-opts ( #3689 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-06-09 10:36:57 -07:00
c33ca60208
[Perf][VoxCPM2] Optimize VoxCPM2 high-concurrency decode throughput. ( #3882 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-06-03 23:08:32 +08:00
TaffyOfficial and TaffyOfficial
ee33601571
[Perf] Deduplicate AR prefix cache hidden-state CPU staging ( #3734 )
...
Signed-off-by: TaffyOfficial <2324465096@qq.com >
Co-authored-by: TaffyOfficial <2324465096@qq.com >
2026-06-01 09:35:07 +00:00
Canlin Guo
412bf42a75
[Refactor] Unify _talker_mtp_forward across GPU and NPU model runners ( #3476 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-05-31 07:58:00 +08:00
NATURE
3a347b10b0
[3/5][core]refactor communication layer: PR 3 of 5, all other models in non async mode ( #3719 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-05-28 14:06:41 +08:00
c99df1ebd9
[TTS][Perf] Optimize Qwen3-TTS high-concurrency serving ( #3662 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-05-17 16:58:59 -04:00
+20
eb4e60ee64
[Rebase] Rebase to vllm v0.21.0 ( #3530 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: NumberWan <wantszkin2003@gmail.com >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: Dnoob <dxpouo@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Signed-off-by: rein yang <ruiruyang2@gmail.com >
Signed-off-by: Nick Cao <ncao@redhat.com >
Signed-off-by: zhumingjue <zhumingjue@huawei.com >
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com >
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: weizhoublue <weizhoublue@github.com >
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io >
Signed-off-by: dengyunyang <584797741@qq.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
Signed-off-by: bjf-frz <frz123db@gmail.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: KexiongYu <yukexiong1@huawei.com >
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: NumberWan <wantszkin2003@gmail.com >
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: Dnoob <dxpouo@gmail.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com >
Co-authored-by: Nick Cao <ncao@redhat.com >
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com >
Co-authored-by: Ricardo Noriega <rnoriega@redhat.com >
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: weizhoublue <45163302+weizhoublue@users.noreply.github.com >
Co-authored-by: weizhoublue <weizhoublue@github.com >
Co-authored-by: dengyunyang <584797741@qq.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: NATURE <wzliu@connect.hku.hk >
Co-authored-by: bjf-frz <frz123db@gmail.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Y. Fisher <yukexiong1@huawei.com >
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-05-16 18:18:43 +08:00
NATURE and Hongsheng Liu
82a0b3a467
[2/5] [core]refactor communication layer: PR 2 of 5 Qwen3 Omni non async ( #2677 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-15 14:14:08 +08:00
dsinghvi
e40076e872
[Refactor] msgspec standardisation for data entry key names and improved type checks ( #3149 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
2026-05-12 14:30:13 +08:00
Sy03
2c73cf3522
[Perf] Fix Qwen3-TTS latency regression ( #3485 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-05-10 23:07:38 +00:00
Nick Cao
e6466cf0e6
[Refactor] Replace and ban a few torch.cuda functions in favor of torch.accelerator replacements. ( #3365 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-05-07 23:23:59 +08:00
1e5f288a91
[FEAT] support multi-stage deployment ( #2396 )
...
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: Zheng Wengang <zwg0606@gmail.com >
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn >
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: Chenguang Zheng <645327136@qq.com >
2026-05-06 19:33:18 +08:00
Sy03
2b85474c23
[Bugfix] Propagate seed to Qwen3-TTS Fast AR sampler ( #3350 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-05-05 14:57:55 -04:00
amy-why-3459 and Hongsheng Liu
01f500a5bf
[perf]Qwen3-Omni performance optimization ( #3203 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-01 22:47:24 +08:00
2390350533
[Refactor] Standardize data entry key names to {type}.{qualifier} format ( #1829 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-25 18:15:38 +08:00
Haco
ad7c966b1d
[BugFix] add missing subtalker sampling config to Qwen3-TTS deploy YAML ( #2940 )
...
Signed-off-by: xiaohajiayou <923390377@qq.com >
2026-04-21 14:47:22 +08:00
NATURE and Hongsheng Liu
0d4e975e1b
[core]refactor communication layer: PR1(Added Refactor Infra Only) ( #1555 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-13 16:01:14 +08:00
Ziming Huang
d2aa9cf08a
Revert "[Fix] Fix slow hasattr in CUDAGraphWrapper.__getattr__ ( #1982 )" ( #2639 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-04-09 17:19:58 +08:00
Ziming Huang
89fff09628
[Fix] Fix slow hasattr in CUDAGraphWrapper.__getattr__ ( #1982 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-03-19 17:10:08 +08:00
89b527e533
Dev/rebase v0170 ( #1639 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-03-09 17:29:38 +08:00
dsinghvi and Zhou Taichang
68764cc4f4
[Refactor]: Phase1 for rebasing_additional_info ( #1394 )
...
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk >
2026-03-02 10:19:41 +08:00
+13
b19c227fb3
[New Model]: XiaomiMiMo/MiMo-Audio-7B-Instruct support ( #750 )
...
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com >
Signed-off-by: hsliu <liuhongsheng4@huawei.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: GG-li <3226868735@qq.com >
Signed-off-by: Sihao Li <111170255+GG-li@users.noreply.github.com >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com >
Signed-off-by: dongbo910220 <1275604947@qq.com >
Signed-off-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com >
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Signed-off-by: baoyuan qi <qibaoyuan@126.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Prajwal A <prajwalanagani@gmail.com >
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Signed-off-by: 丁宁 <nndding@gmail.com >
Signed-off-by: SHIJIN ZHANG <75300765+Dovis01@users.noreply.github.com >
Signed-off-by: dingning<dingning7@xiaomi.com >
Signed-off-by: dingning <dingning7@xiaomi.com >
Signed-off-by: dingning <dingning@xiaomi.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: Zhang Shijin <zhangshijin@xiaomi.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Sihao Li <111170255+GG-li@users.noreply.github.com >
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: JohnJan <wuzhongjian_yewu@cmss.chinamobile.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: Junhong Liu <ljh_lbj@163.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
Co-authored-by: shijin zhang <zsj1364226740@gmail.com >
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
Co-authored-by: Prajwal A <34590600+LawJarp-A@users.noreply.github.com >
Co-authored-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: dingning <dingning7@xiaomi.com >
Co-authored-by: ning ding <nndding@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-02-27 11:19:49 +08:00
Yuanheng Zhao
ed08a216c1
[Misc] Migrate L1 tests to use pytest-mock ( #1315 )
...
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
2026-02-26 17:08:39 +08:00
Sy03 and Hongsheng Liu
efbe4116ff
[Feature] : Support disaggregated inference pipeline for Qwen3_TTS ( #1161 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-21 07:36:44 +08:00
75770c9309
[Rebase] Rebase to vllm v0.16.0 ( #1357 )
...
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2026-02-13 18:05:07 +08:00
Alicia and Hongsheng Liu
d3ea94309d
[CI] Add pytest markers to current tests and update the doc. ( #577 )
...
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-09 16:49:45 +08:00
dsinghvi
59024bee62
Process-Scoped GPU Memory Accounting ( #1204 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
2026-02-08 22:43:02 +08:00
f6cfc0d1e3
[BugFix] Fix Qwen3 Omni talker mtp torch.compile startup error ( #1104 )
...
Signed-off-by: ram16g <anlianfengjie@163.com >
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Co-authored-by: ram16g <anlianfengjie@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-30 23:25:49 +08:00
Gao Han
a54c323952
[Model] Support Qwen3-TTS model series ( #895 )
...
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk >
2026-01-22 16:57:37 +08:00
+13
776c3a794e
Dev/rebase 0.14.0 ( #847 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Signed-off-by: root <root@hk01dgx039.cm.cluster >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: TangPeng <85704592@qq.com >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: iwzbi <wzbi@zju.edu.cn >
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com >
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com >
Signed-off-by: Dinesh G <G.Dinesh@ibm.com >
Signed-off-by: gDINESH13 <dinesh13g@gmail.com >
Signed-off-by: root <root@hk01dgx028.cm.cluster >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: JaredforReal <w13431838023@gmail.com >
Co-authored-by: root <root@hk01dgx039.cm.cluster >
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com >
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com >
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com >
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com >
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
2026-01-21 00:25:25 +08:00
Ziming Huang
1444e1fc45
[Feature] Support Qwen3 Omni talker mtp batch inference ( #722 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-14 19:07:31 +08:00
