Taichang Zhou and Claude Opus 4.8
9678ae58cf
[MR V2] Migrate to Model Runner V2
...
Squashed migration rebased onto current main. Collapses the 46-commit
dev/migrate-MR-v2 history (10 merge commits + superseded pre-rebase work)
into a single commit, so the diff reflects only the genuine MR-V2 feature
(worker_v2/ runners + model_states, omni schedulers, stage engine pipeline,
pooling_output serialization bridge). Content already merged to main
separately (step_audio2, tts_adapters, npu/_310p, etc.) drops out.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-06-20 07:19:40 +00:00
amy-why-3459
f05e03aea8
[Perf][DreamZero]dreamzero supports CUDA graph, torch.compile, and DiT Caching. ( #4213 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-06-19 14:24:37 +08:00
f75c437a60
[Refactor]Base class for dit pipelines with unified parameter declaration[1/N] ( #4225 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-06-18 17:14:18 +08:00
64a5bb92de
[Refactor] Omni Stage Runtime and Distributed Replica Control Plane ( #3855 )
...
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Chenguang ZHENG <645327136@qq.com >
2026-06-17 14:13:20 +08:00
Zheng Wengang and Hongsheng Liu
ba4932c364
[Bugfix] Fix multi-replica stage identity and clean up redundant deploy config ( #4410 )
...
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-16 14:08:11 +08:00
Hongsheng Liu
aff0af9754
[Refactor] Extract diffusion output formatting boundary ( #4407 )
...
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-06-15 22:50:50 +08:00
Zeng Chuang and Hongsheng Liu
10404e4c5b
[Perf] Restore parallel stage initialization for AR+DiT pipelines ( #3641 )
...
Signed-off-by: zengchuang <zengchuang3@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-15 09:37:11 +08:00
Zhou Taichang
7b837944a1
[Rebase] Rebase to vllm 0.23.0 ( #4286 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-06-13 16:43:02 +00:00
8f869c8ea5
[Refactor] Output Processor Phase 2: separate multimodal output channel ( #1601 ) ( #2744 )
...
Signed-off-by: Megha Agarwal <agarwalmegha1308@gmail.com >
Signed-off-by: meghaagr13 <divyanshsinghvi@gmail.com >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: meghaagr13 <divyanshsinghvi@gmail.com >
Co-authored-by: Boao Shi <aoibosh@connect.hku.hk >
Co-authored-by: Taichang Zhou <tzhouam@connect.ust.hk >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-06-12 15:19:14 +08:00
MaciejBalaNV and Hongsheng Liu
03cc888485
[refactor] Refactor guardrail error handling - add 400 error code ( #4297 )
...
Signed-off-by: Maciej Bala <mbala@nvidia.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-11 21:09:24 +00:00
dengyunyang and Hongsheng Liu
14879d6619
[Bugfix][HunyuanImage] fix accuracy in stream mode ( #4265 )
...
Signed-off-by: dengyunyang <584797741@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-11 10:39:00 +08:00
57ea047acf
Step audio R1 reasoning parser ( #2846 )
...
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: Mike Qiu <qdy220091330@gmail.com >
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-10 21:03:19 +00:00
akshatvishu and Hongsheng Liu
2fa4c7a4a9
[Model] Add Ming-omni-tts dense 0.5B pipeline ( #2906 )
...
Signed-off-by: akshatvishu <akshatnayak197@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-10 20:11:04 +08:00
e1b0b83238
add moriio transfer engine ( #1742 )
...
Signed-off-by: inkcherry <mingzhi.liu@amd.com >
Signed-off-by: zejwang <zejwang@amd.com >
Signed-off-by: junkang1991 <junkangchow@gmail.com >
Signed-off-by: Zejian Wang <zejianwang@sjtu.edu.cn >
Co-authored-by: zejwang <zejwang@amd.com >
Co-authored-by: junkang1991 <junkangchow@gmail.com >
Co-authored-by: Zejian Wang <zejianwang@sjtu.edu.cn >
Co-authored-by: Zejian Wang <Zejian.Wang@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-06-10 08:37:33 +08:00
57227dc7f3
[Perf][Bugfix] qwen3-tts hot path: prefix-cache OOM guards + talker/orchestrator micro-opts ( #3689 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-06-09 10:36:57 -07:00
ChenWenjing
e25239c2ec
[bugfix] fix realtime ci timeout error ( #4187 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
2026-06-06 00:05:07 +08:00
ZacheryAU
bfc7cedcf1
Benchmark data statistics for each stage of omni models ( #3628 )
...
Signed-off-by: ZacheryAU <70869627+ZacheryAU@users.noreply.github.com >
2026-06-04 21:43:35 +08:00
amy-why-3459
8949fdc440
[BugFix] Fix the issue of vllm failing to start. ( #4105 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-06-03 15:44:46 +00:00
rein yang and Hongsheng Liu
3a7c7f1422
[Bugfix] fix qwen3-omni performance regression ( #3575 )
...
Signed-off-by: rein yang <ruiruyang2@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-02 23:02:07 +08:00
Alex Brooks
12e75c6618
[Core] Integrate TrackingArgumentParser ( #3369 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-06-02 21:56:24 +08:00
bc794e625f
Add Cosmos3 model support ( #3454 )
...
Signed-off-by: Maciej Bala <mbala@nvidia.com >
Signed-off-by: MaciejBalaNV <mbala@nvidia.com >
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: lishunyang12 <lishunyang12@163.com >
2026-06-02 04:04:31 +08:00
2b7249fbb3
[Model]Support MiniCPM-o 4.5 ( #3642 )
...
Signed-off-by: tc-mb <tianchi_cai@icloud.com >
Co-authored-by: GKangaroo <1095103651@qq.com >
Co-authored-by: GKangaroo <gqx24@mails.tsinghua.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-01 22:12:24 +08:00
Yueqian Lin and Hongsheng Liu
c067485030
[Perf][TTS] Bounded-K active-stream window for Stage 1 (RFC #3535 ) ( #3592 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-01 21:59:10 +08:00
LHXuuu and vraiti
41a795b5d6
[Metrics] Add audio SLOs + cross-stage transfer families + per-(stage, replica) wrap for upstream vllm:* ( #3576 )
...
Signed-off-by: vraiti <vraiti@redhat.com >
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
Co-authored-by: vraiti <vraiti@redhat.com >
2026-06-01 20:20:48 +08:00
+1
f92d84fcf9
[Rebase] Rebase to vllm releases/v0.22.0 ( #3891 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: david6666666 <530634352@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: nainiu258 <cperfect02@163.com >
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com >
Co-authored-by: Zheng Wengang <zwg0606@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-06-01 11:08:12 +08:00
d4c139508f
[Feature] Add precomputed custom voices and Qwen3-TTS ref-context cache ( #3492 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Dan250124 <416947747@qq.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-31 17:51:24 +00:00
ChenWenjing
5dfdf584dd
[Feat]audio streaming input for async chunk ( #3614 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
2026-05-31 17:39:25 +08:00
b690a10e3a
[Feat] Support MagCache ( #1287 )
...
Signed-off-by: Lancer <maruixiang6688@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-05-29 22:26:23 +08:00
c69ed4037d
[feat] Support prompt embedding caching for diffusion model ( #2962 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-29 15:36:03 +08:00
Lancer and Alicia
5c3c81920b
[Bugfix] Ensure stage and diffusion subprocesses exit when parent dies unexpectedly ( #3751 )
...
Signed-off-by: Lancer <maruixiang6688@gmail.com >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
2026-05-28 08:22:04 +08:00
Haco and zzhuoxin1508
37eebff674
[BugFix] Fix diffusion parallel_config YAML override and add deploy config field allowlist ( #3483 )
...
Signed-off-by: xiaohajiayou <923390377@qq.com >
Co-authored-by: zzhuoxin1508 <zzhuoxin1508@users.noreply.github.com >
2026-05-28 01:00:32 +08:00
a698269e47
[Model] Add Ming-flash-omni-2.0 Image Generation (Diffusion) Stage ( #2875 )
...
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: Zheng Wengang <zwg0606@gmail.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com >
2026-05-27 23:21:20 +08:00
283873411a
glm-image: fix(npu)per-stage runtime env for HCCL ports + GLM-Image NPU stage config ( #3235 )
...
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-27 12:18:47 +12:00
Yueqian Lin and Rein Yang
e996723bc9
[Bugfix] Fix Qwen3-TTS Stage 0 prefix-caching correctness ( #3665 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
the ci failure case is irrelavent to this PR and occurs rarely
2026-05-25 23:09:38 +08:00
BeatSeat and Yueqian Lin
28ce618f0e
[Model] Add GLM-TTS text-to-speech model support ( #3141 )
...
Signed-off-by: BeatSeat <wendavid552@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-24 05:42:26 +08:00
ANHDY and wangmengdi
28467dc19a
[feature]: support Hidream-I1-Full model ( #2572 )
...
Signed-off-by: wangmengdi <160612767@qq.com >
Co-authored-by: wangmengdi <wangmengdi@cmss.chinamobile.com >
2026-05-22 15:23:03 +08:00
bjf-frz and bjf-frz
5cf4605bf9
[Bugfix]Fix distributed stage0 multimodal cache routing ( #3740 )
...
Signed-off-by: bjf-frz <240110042+bjf-frz@users.noreply.github.com >
Co-authored-by: bjf-frz <240110042+bjf-frz@users.noreply.github.com >
2026-05-20 15:31:41 +08:00
Yueqian Lin
fc8486cf20
[Doc][TTS] CosyVoice3 online docs + residual TTS yaml cleanup + remove VoxCPM v1 ( #3748 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-05-20 15:24:06 +08:00
yangsonglin13
1c530d984d
Yuanrong TransferEngine Connector for NPU ( #3180 )
...
Signed-off-by: yangsonglin13 <yangsonglin566@gmail.com >
2026-05-20 14:51:33 +08:00
chickeyton and herotai214
e277feacaf
[large-scale-serving] Integrate OmniCoordinator into stage engine pipeline ( #3569 )
...
Signed-off-by: chickeyton <ngton2014@gmail.com >
Signed-off-by: herotai214 <herotai214@gmail.com >
Co-authored-by: herotai214 <herotai214@gmail.com >
2026-05-19 12:08:20 +08:00
Rustam Khadipash
89f8819525
[Feature] Add support for Pipeline Parallel and integrate it into Wan 2.2 ( #2322 )
...
Signed-off-by: Rustam Khadipash <16683750+hadipash@users.noreply.github.com >
2026-05-19 09:42:17 +08:00
8cfc9179e6
Fix reasoning_parser crash: reconstruct StructuredOutputsConfig from dict ( #2845 )
...
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-17 23:30:24 +08:00
Yuanheng Zhao
a64ebf103b
[Refactor] Migrate and clean up TTS configs: CosyVoice3, OmniVoice, VoxCPM ( #3338 )
...
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
2026-05-17 07:32:34 +08:00
+20
eb4e60ee64
[Rebase] Rebase to vllm v0.21.0 ( #3530 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: NumberWan <wantszkin2003@gmail.com >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: Dnoob <dxpouo@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Signed-off-by: rein yang <ruiruyang2@gmail.com >
Signed-off-by: Nick Cao <ncao@redhat.com >
Signed-off-by: zhumingjue <zhumingjue@huawei.com >
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com >
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: weizhoublue <weizhoublue@github.com >
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io >
Signed-off-by: dengyunyang <584797741@qq.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
Signed-off-by: bjf-frz <frz123db@gmail.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: KexiongYu <yukexiong1@huawei.com >
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: NumberWan <wantszkin2003@gmail.com >
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: Dnoob <dxpouo@gmail.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com >
Co-authored-by: Nick Cao <ncao@redhat.com >
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com >
Co-authored-by: Ricardo Noriega <rnoriega@redhat.com >
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: weizhoublue <45163302+weizhoublue@users.noreply.github.com >
Co-authored-by: weizhoublue <weizhoublue@github.com >
Co-authored-by: dengyunyang <584797741@qq.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: NATURE <wzliu@connect.hku.hk >
Co-authored-by: bjf-frz <frz123db@gmail.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Y. Fisher <yukexiong1@huawei.com >
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-05-16 18:18:43 +08:00
bjf-frz
440c718d2a
[Bugfix]Fix multimodal cache routing for AR replicas ( #3605 )
...
Signed-off-by: bjf-frz <frz123db@gmail.com >
2026-05-15 16:42:18 +08:00
lyj-jjj and Cursor
bbc00f9f86
[BugFix] fix(omni): isolate diffusion KV-cache dtype from vLLM --kv-cache-dtype #3585 ( #3596 )
...
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-05-15 11:22:59 +08:00
wuhang
0d9d57acd9
[Entrypoint][Refactor] Make field type hint more concrete ( #3139 )
...
Signed-off-by: wuhang <wuhang6@huawei.com >
2026-05-14 11:25:55 +08:00
Y. Fisher
4d3eed152a
[Feat][Config] Support additional_config for diffusion worker ( #3020 )
...
Signed-off-by: KexiongYu <yukexiong1@huawei.com >
Signed-off-by: Y. Fisher <yukexiong1@huawei.com >
2026-05-13 14:22:43 +00:00
knlnguyen1802 and Samit
631251a1f8
[Bugfix, rl] Diffusion worker SIGKILL under Ray actor (exitcode -9) ( #3533 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Co-authored-by: Samit <285365963@qq.com >
2026-05-13 15:31:48 +08:00
56ca7dd612
support online FP8 quantization for FA on NPU #2236 ( #2640 )
...
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: gcanlin <canlinguosdu@gmail.com >
2026-05-13 11:46:00 +08:00