Hongsheng Liu
16642dc2e8
[Diffusion] Make DLO host weight cache opt-in
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-19 11:23:36 +08:00
Hongsheng Liu
010b3c1d74
[Diffusion] Make no-AllGather host weight cache automatic
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-19 11:23:36 +08:00
Hongsheng Liu
3bb7547a0f
[Diffusion] Rename runtime cache to host weight cache
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-19 11:23:36 +08:00
Hongsheng Liu
23ab14a15b
[Diffusion] Register runtime cache for direct DLO H2D
...
Avoid recurrent mmap-to-pinned packing in no-AllGather DLO by registering transform-complete runtime-cache mappings under an explicit per-worker budget. Copy mapped tensors directly into the rotating device buffers while preserving bounded-staging fallback and ordered shutdown cleanup.
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-19 11:21:28 +08:00
Hongsheng Liu
d30ccdc315
[Diffusion] Add node-local runtime weight cache for DLO
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-19 11:21:01 +08:00
Yueqian Lin
b283215c28
[Model] Add MiniMax Music 3 text-to-music ( #6186 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-08-18 04:33:13 +00:00
Yukim1 and Hongsheng Liu
4b66f62e10
[Refactor] Remove internal stage_configs_path plumbing ( #5741 )
...
Signed-off-by: zwhzzz0821 <2831474076@qq.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-13 21:29:03 +08:00
Yuekai Zhang and Claude Fable 5
5be7541d01
[Model] Support NVIDIA-NemotronLabs-VoiceChat-11B offline speech-to-speech ( #5842 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-12 16:25:48 +08:00
9b0df3909a
[Feature] timestamps for TTS via a forced-aligner pooling ( #4795 )
...
Signed-off-by: wjinxu <1299461899@qq.com >
Co-authored-by: xurui <xurui1@modelbest.onaliyun.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-12 05:33:34 +00:00
Sy03 and Yueqian Lin
58007c0084
[New Model] Support IndexTTS 2.5 ( #5957 )
...
Signed-off-by: Sysy <1370724210@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-08-11 23:00:35 +00:00
Zhou Taichang and Claude Opus 5
2ce15ab7a0
[Rebase] Rebase to vllm 0.27.0 ( #5976 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-11 15:23:46 +00:00
Jerry Song
89015d5cad
[Feature]: Adds distilled LoRA support for diffusion models ( #2783 )
...
Signed-off-by: Songrui625 <songrui625@gmail.com >
2026-08-11 14:23:43 +00:00
NumberWan
cfa6d66ed6
[BugFix][Nightly CI] Opt in FA deterministic for Qwen-Image accuracy ( #5887 )
...
Signed-off-by: NumberWan <wantszkin2003@gmail.com >
2026-08-07 17:15:22 +08:00
48298030d4
[Diffusion] Add Minimax-H3 modular pipeline support ( #5720 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Signed-off-by: David Chen <530634352@qq.com >
Co-authored-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: David Chen <530634352@qq.com >
2026-08-06 11:43:23 +00:00
SYLAR and lishunyang12
1c2a81f6d8
feat(minimax-h3): enable 2x RTX 4090/5090 with DLO ( #5764 )
...
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
2026-08-06 09:02:29 +08:00
Yueqian Lin and Sy0307
e788ef6e76
[Feature] PersonaPlex (Moshi-based full-duplex S2S): native vLLM port + duplex serving ( #4771 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: Sy0307 <24841857+Sy0307@users.noreply.github.com >
Co-authored-by: Sy0307 <24841857+Sy0307@users.noreply.github.com >
2026-08-04 20:10:33 -07:00
Isotr0py
24741961eb
[Model] Add MiniMax H3 diffusion support ( #5691 )
2026-08-03 14:10:40 +08:00
evanchueng and david6666666
744c65b730
[Feature] Distributed layerwise offload with DP multi-concurrency + mmap weight… ( #5397 )
...
Signed-off-by: evanchueng <evanchueng@outlook.com >
Signed-off-by: david6666666 <530634352@qq.com >
Co-authored-by: david6666666 <530634352@qq.com >
2026-07-30 13:35:38 +00:00
wangyu
7c17111ff5
[Refactor][CI] Reorganize test tree, and move slow e2e to weekly ( #4951 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-07-29 23:53:03 +08:00
Canlin Guo
61324e65d7
[Refactor] Make MOSS-TTS-Local stream session dynamic batch ( #5235 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com >
2026-07-28 21:38:24 +08:00
TaffyOfficial and TaffyOfficial
93786c9cd3
[perf] Add configurable diffusion compile granularity ( #4603 )
...
Signed-off-by: TaffyOfficial <2324465096@qq.com >
Co-authored-by: TaffyOfficial <2324465096@qq.com >
2026-07-24 15:22:06 +00:00
84bbd57e4d
[Feature] Add AllGather-KV sequence-parallel attention str… ( #4968 )
...
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com >
Signed-off-by: dph97 <nenbaying@163.com >
Signed-off-by: zhuhh97 <zhh@unknown>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: dengyunyang <584797741@qq.com >
2026-07-24 14:45:26 +00:00
Bingyu (Spencer) Liu
dcf8b07d39
[Core] Cleanup Omni connector runtime and transfer backends ( #5096 )
...
Signed-off-by: spencerr221 <liubingyu62@gmail.com >
2026-07-22 09:44:15 +08:00
Shi Boao Bowie and bjf-frz
a432b8a325
[Refactor][OutputProcessor 1/8]: clean up multimodal payload accumulation and move payload logic into MultimodalPayload & directory refactor ( #4980 )
...
Signed-off-by: Boao Shi <aoibosh@connect.hku.hk >
Signed-off-by: bjf-frz <frz123db@gmail.com >
Co-authored-by: bjf-frz <frz123db@gmail.com >
2026-07-15 15:50:19 +08:00
ff6f0da271
[Core] Composable parallel: Phase 1 strategy overlay (TP/DP/stage-replica) (RFC #4084 ) ( #4281 )
...
Signed-off-by: Kushan A. <kahmadian@nvidia.com >
Signed-off-by: kushanam <kushanam@users.noreply.github.com >
Signed-off-by: Kushan A <kahmadian@nvidia.com >
Signed-off-by: kahmadian <kahmadian@users.noreply.github.com >
Co-authored-by: kushanam <kushanam@users.noreply.github.com >
Co-authored-by: kahmadian <kahmadian@users.noreply.github.com >
2026-07-10 21:29:24 -06:00
JiangJie Zhang and Samit
c45ac74d43
[Core][Frontend] Support request-level batching for diffusion pipelines ( #4079 )
...
Signed-off-by: jader <yjader@foxmail.com >
Co-authored-by: Samit <285365963@qq.com >
2026-06-29 18:09:19 +02:00
504698dec8
[Core] Performance fix for orchestrator bottleneck: separates inter-stage from client outputs ( #4527 )
...
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: Chenguang ZHENG <645327136@qq.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-06-29 17:43:36 +02:00
Rahul Steiger and Hongsheng Liu
327e9dcaf2
[Feature] Spatially-sharded (SP) decode for the Wan VAE ( #4620 )
...
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-27 09:01:25 +08:00
0b04f2a0d6
[Model] Add Ming-omni-tts MoE 16.8B-A3B + CFM CUDAGraph ( #4341 )
...
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-25 22:11:35 +08:00
Canlin Guo
2e6c83c89c
[Model] Support MOSS-TTS-local-v1.5 ( #4664 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-06-25 15:43:03 +08:00
BeatSeat
044240d8eb
[Model] Add IndexTTS2 text-to-speech support ( #3838 )
...
Signed-off-by: BeatSeat <wendavid552@gmail.com >
Signed-off-by: David <david@podcast.ai >
Signed-off-by: wendavid552 <wendavid552@gmail.com >
Signed-off-by: WenDavid <davidzhangcf@foxmail.com >
2026-06-22 19:24:01 +00:00
57ea047acf
Step audio R1 reasoning parser ( #2846 )
...
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: Mike Qiu <qdy220091330@gmail.com >
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-10 21:03:19 +00:00
akshatvishu and Hongsheng Liu
2fa4c7a4a9
[Model] Add Ming-omni-tts dense 0.5B pipeline ( #2906 )
...
Signed-off-by: akshatvishu <akshatnayak197@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-10 20:11:04 +08:00
amy-why-3459
8949fdc440
[BugFix] Fix the issue of vllm failing to start. ( #4105 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-06-03 15:44:46 +00:00
Alex Brooks
12e75c6618
[Core] Integrate TrackingArgumentParser ( #3369 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-06-02 21:56:24 +08:00
2b7249fbb3
[Model]Support MiniCPM-o 4.5 ( #3642 )
...
Signed-off-by: tc-mb <tianchi_cai@icloud.com >
Co-authored-by: GKangaroo <1095103651@qq.com >
Co-authored-by: GKangaroo <gqx24@mails.tsinghua.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-01 22:12:24 +08:00
Yueqian Lin and Hongsheng Liu
c067485030
[Perf][TTS] Bounded-K active-stream window for Stage 1 (RFC #3535 ) ( #3592 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-01 21:59:10 +08:00
+1
f92d84fcf9
[Rebase] Rebase to vllm releases/v0.22.0 ( #3891 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: david6666666 <530634352@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: nainiu258 <cperfect02@163.com >
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com >
Co-authored-by: Zheng Wengang <zwg0606@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-06-01 11:08:12 +08:00
BeatSeat and Yueqian Lin
28ce618f0e
[Model] Add GLM-TTS text-to-speech model support ( #3141 )
...
Signed-off-by: BeatSeat <wendavid552@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-24 05:42:26 +08:00
Yueqian Lin
fc8486cf20
[Doc][TTS] CosyVoice3 online docs + residual TTS yaml cleanup + remove VoxCPM v1 ( #3748 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-05-20 15:24:06 +08:00
chickeyton and herotai214
e277feacaf
[large-scale-serving] Integrate OmniCoordinator into stage engine pipeline ( #3569 )
...
Signed-off-by: chickeyton <ngton2014@gmail.com >
Signed-off-by: herotai214 <herotai214@gmail.com >
Co-authored-by: herotai214 <herotai214@gmail.com >
2026-05-19 12:08:20 +08:00
Yuanheng Zhao
a64ebf103b
[Refactor] Migrate and clean up TTS configs: CosyVoice3, OmniVoice, VoxCPM ( #3338 )
...
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
2026-05-17 07:32:34 +08:00
baonudesifeizhai and roG0d
c4a0990044
（Phase 1）Add ModelOpt FP8 auto-detect support for diffusion checkpoints #2709 ( #2913 )
...
Signed-off-by: roG0d <rodgarcas98@gmail.com >
Signed-off-by: roG0d <baonudesifeizhai@gmail.com >
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com >
Co-authored-by: roG0d <rodgarcas98@gmail.com >
2026-05-09 16:55:05 +08:00
Chen-Yo Sun
56ca5d9a8f
[BugFix] Forward CLI --tokenizer to per-stage engine configs ( #3120 )
...
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai >
2026-05-07 06:43:01 +08:00
b076006c35
[BugFix] Fix Whitelist optimization CI failure ( #3290 )
...
Signed-off-by: xiaohajiayou <923390377@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-06 14:25:20 +08:00
Gao Han
704b6bb6e6
[CI failed]Revert "[Config Refactor] Derive deploy override fields from stage config" ( #3287 )
...
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk >
2026-05-01 04:39:54 +01:00
Haco
01ebc0cdf5
[Config Refactor] Derive deploy override fields from stage config ( #3162 )
...
Signed-off-by: xiaohajiayou <923390377@qq.com >
2026-05-01 01:04:03 +08:00
Yuanheng Zhao
6b52db9e2b
[Bugfix][Refactor] Migrate Voxtral TTS config and parser registry ( #3065 )
...
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
2026-04-25 18:17:25 +08:00
Haco and lishunyang
22ffb85063
[Config Refactor] sentinel default precedence ( #3078 )
...
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: xiaohajiayou <923390377@qq.com >
Co-authored-by: lishunyang <lishunyang12@163.com >
2026-04-24 20:07:24 +08:00
Chen-Yo Sun and Yueqian Lin
8e1bbc9ab5
[Feature][Voxtral TTS] Support per-model extra sampling params & add cfg_alpha for Voxtral TTS ( #2338 )
...
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
2026-04-22 03:49:50 +00:00
Hongsheng Liu