Taichang Zhou and Claude Opus 4.8
9678ae58cf
[MR V2] Migrate to Model Runner V2
...
Squashed migration rebased onto current main. Collapses the 46-commit
dev/migrate-MR-v2 history (10 merge commits + superseded pre-rebase work)
into a single commit, so the diff reflects only the genuine MR-V2 feature
(worker_v2/ runners + model_states, omni schedulers, stage engine pipeline,
pooling_output serialization bridge). Content already merged to main
separately (step_audio2, tts_adapters, npu/_310p, etc.) drops out.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-06-20 07:19:40 +00:00
Ian Eaves
951ae28eb0
[Frontend] Configurable video storage backends with TTL ( #2531 )
...
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com >
2026-06-19 17:41:22 -06:00
Yueqian Lin
208932210f
[Chore][TTS] Reference recipes.vllm.ai recipes, remove dead code, and fix stale references ( #4567 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-06-19 15:05:26 +08:00
amy-why-3459
f05e03aea8
[Perf][DreamZero]dreamzero supports CUDA graph, torch.compile, and DiT Caching. ( #4213 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-06-19 14:24:37 +08:00
f75c437a60
[Refactor]Base class for dit pipelines with unified parameter declaration[1/N] ( #4225 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-06-18 17:14:18 +08:00
NumberWan and lishunyang12
d744940b18
Refactor: extract OmniStreamingVideoHandler base and QwenOmniStreamingVideoHandler ( #4424 )
...
Signed-off-by: NumberWan <wantszkin2003@gmail.com >
Signed-off-by: lishunyang12 <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: lishunyang12 <125541396+lishunyang12@users.noreply.github.com >
2026-06-17 15:07:45 +00:00
blonde and Claude Opus 4.6
8fa0394d29
[Bugfix] Wire log_stats to AsyncOmni and add missing token metrics for non-text requests ( #4482 )
...
Signed-off-by: blonde <164845202+blondeCS@users.noreply.github.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-06-17 21:31:44 +08:00
64a5bb92de
[Refactor] Omni Stage Runtime and Distributed Replica Control Plane ( #3855 )
...
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Chenguang ZHENG <645327136@qq.com >
2026-06-17 14:13:20 +08:00
f200e988a6
[fix bug] add default limit for image generation ( #3381 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: lishunyang <lishunyang12@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-06-16 22:54:26 +08:00
TJian
5cce797c03
[Bugfix] [CI] [ROCm] [CUDA] Fix voxtral test ( #4380 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-06-16 16:39:23 +08:00
Hongsheng Liu
aff0af9754
[Refactor] Extract diffusion output formatting boundary ( #4407 )
...
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-06-15 22:50:50 +08:00
Yuzhu Dong
e2e79177db
Support DROID policy server for Cosmos3 OpenPI ( #4282 )
...
Signed-off-by: yuzhud <yuzhudong@gmail.com >
2026-06-15 21:22:59 +08:00
Zhou Taichang
7b837944a1
[Rebase] Rebase to vllm 0.23.0 ( #4286 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-06-13 16:43:02 +00:00
Yueqian Lin
d75a6433b4
[Refactor] TTS serving adapter framework + migrate TTS models (excl. ming_flash_omni) ( #4330 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-06-13 04:22:54 +00:00
9bc8705c98
[New Model] Step-Audio2 ( #464 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: wuli666 <djjpro975@gmail.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: wuli666 <djjpro975@gmail.com >
2026-06-13 01:00:33 +00:00
8f869c8ea5
[Refactor] Output Processor Phase 2: separate multimodal output channel ( #1601 ) ( #2744 )
...
Signed-off-by: Megha Agarwal <agarwalmegha1308@gmail.com >
Signed-off-by: meghaagr13 <divyanshsinghvi@gmail.com >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: meghaagr13 <divyanshsinghvi@gmail.com >
Co-authored-by: Boao Shi <aoibosh@connect.hku.hk >
Co-authored-by: Taichang Zhou <tzhouam@connect.ust.hk >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-06-12 15:19:14 +08:00
Chendi.Xue
3fe3aa359b
[XPU][COSMOS3] removing cuda hardcode and make VLLM_VIDEO_SYNC_TIMEOUT a tunable config ( #4360 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-06-12 04:55:07 +00:00
152343d5fb
Feat: non_streaming_mode for Qwen3-TTS Base Models During Online Inference ( #4198 )
...
Signed-off-by: nagisa-kun <1434936049@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Sy03 <1370724210@qq.com >
2026-06-11 22:15:05 +00:00
MaciejBalaNV and Hongsheng Liu
03cc888485
[refactor] Refactor guardrail error handling - add 400 error code ( #4297 )
...
Signed-off-by: Maciej Bala <mbala@nvidia.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-11 21:09:24 +00:00
e27fd46ab9
[Perf] Optimize Higgs Audio v3 serving ( #4204 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-11 13:29:31 -07:00
Chendi.Xue and Hongsheng Liu
5414f78e5f
[WAN2.2-S2V] Add server API for image + audio ( #3394 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-11 18:50:03 +08:00
57ea047acf
Step audio R1 reasoning parser ( #2846 )
...
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: Mike Qiu <qdy220091330@gmail.com >
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-10 21:03:19 +00:00
akshatvishu and Hongsheng Liu
2fa4c7a4a9
[Model] Add Ming-omni-tts dense 0.5B pipeline ( #2906 )
...
Signed-off-by: akshatvishu <akshatnayak197@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-10 20:11:04 +08:00
Optance
d50bf1aa42
[Bugfix] Fix Fish Speech prefix cache collision from missing cache_salt ( #4008 )
...
Signed-off-by: nagelanping <xiao.yi.ming@foxmail.com >
2026-06-10 17:11:01 +08:00
Yuekai Zhang and Claude Opus 4.8
f1bf544208
[Perf][TTS] Optimize cosyvoice TTFP and throughput using TensorRT ( #4168 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-06-10 16:42:34 +08:00
MaciejBalaNV
2f25195750
Cosmos3 video to video generation ( #4266 )
...
Signed-off-by: Maciej Bala <mbala@nvidia.com >
2026-06-09 03:04:05 +08:00
amy-why-3459
fc7393e485
[Refactor]Refactoring audio_in_video implementation ( #3566 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-06-08 14:14:02 +08:00
Lokesh
176b00538c
[Model] Support languages added by fine-tuned Qwen3-TTS checkpoints ( #4210 )
...
Signed-off-by: lokesh <lokesh.v9911@gmail.com >
2026-06-07 10:10:40 -06:00
Yueqian Lin
fce0f51089
[Perf][Higgs-Audio-V3] LRU cache for voice-clone ref-audio encode ( #4200 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-06-07 03:09:16 +00:00
Yuekai Zhang and Yueqian Lin
82e74ce972
[TTS][New Model] support bosonai/higgs-audio-v3-tts-4b ( #4169 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
2026-06-06 15:59:12 +00:00
ChenWenjing
e25239c2ec
[bugfix] fix realtime ci timeout error ( #4187 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
2026-06-06 00:05:07 +08:00
Jianbao Huang
6cefe91370
[Perf] Enable fused RMSNorm for HunyuanImage3 ( #3959 )
2026-06-05 07:41:56 +00:00
Yaegaki1Erika and reandeng
84569280b1
[Bugfix]Fix HunyuanImage3 conditional image and prompt kwargs alignment ( #4145 )
...
Signed-off-by: reandeng <reandeng@tencent.com >
Signed-off-by: Yaegaki1Erika <70182590+Yaegaki1Erika@users.noreply.github.com >
Co-authored-by: reandeng <reandeng@tencent.com >
2026-06-05 12:47:25 +08:00
Zheng Wengang
fccd95b12b
[BugFix] Fix incorrect GPU device mapping in multi-replica stages ( #4132 )
...
Signed-off-by: ZhengWG <zwg0606@gmail.com >
2026-06-04 15:32:58 +00:00
ZacheryAU
bfc7cedcf1
Benchmark data statistics for each stage of omni models ( #3628 )
...
Signed-off-by: ZacheryAU <70869627+ZacheryAU@users.noreply.github.com >
2026-06-04 21:43:35 +08:00
BruceLoveDecimal
e5030c7524
[BugFix] Cast bf16 video frames to float32 before .numpy() in /v1/videos ( #4114 )
...
Signed-off-by: BruceLoveDecimal <39156883+BruceLoveDecimal@users.noreply.github.com >
2026-06-04 17:49:07 +08:00
706bad253d
Add Cosmos3 action modality ( #4102 )
...
Signed-off-by: Maciej Bala <mbala@nvidia.com >
Signed-off-by: Bartosz Stefaniak <bstefaniak@nvidia.com >
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: Maciej Bala <mbala@nvidia.com >
Co-authored-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-06-03 19:30:39 +00:00
07737e6c61
[Test] Add prefix caching + audio output regression test ( #3510 ) ( #3604 )
...
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com >
Signed-off-by: Alex Brooks <albrooks@redhat.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: Alex Brooks <albrooks@redhat.com >
2026-06-03 09:17:53 +00:00
amy-why-3459
9bb4974198
[BugFix] Fix the issue of dataset names not being resolved ( #4094 )
2026-06-03 06:17:04 +00:00
40b2959183
Add Cosmos3 sound generation ( #4073 )
...
Signed-off-by: Maciej Bala <mbala@nvidia.com >
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Signed-off-by: Bartosz Stefaniak <bstefaniak@nvidia.com >
Co-authored-by: Bartosz Stefaniak <bstefaniak@nvidia.com >
Co-authored-by: lishunyang12 <lishunyang12@163.com >
2026-06-03 00:51:44 +00:00
bdfc77147d
[Diffusion] DreamZero world model integration with CFG parallel + OpenPI serving ( #2162 )
...
Signed-off-by: Meng <meng_chen99@163.com >
Signed-off-by: Yangshen Deng <yangshen.d@outlook.com >
Signed-off-by: Suli Wang <50828270+wsl2000@users.noreply.github.com >
Co-authored-by: Meng <meng_chen99@163.com >
Co-authored-by: Suli Wang <50828270+wsl2000@users.noreply.github.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-06-02 19:50:20 +00:00
Alex Brooks
12e75c6618
[Core] Integrate TrackingArgumentParser ( #3369 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-06-02 21:56:24 +08:00
Zhang Jian and Canlin Guo
b550709bba
[TTS][Model] support MOSS-TTS series ( #3420 )
...
Signed-off-by: Zhang <jianmusings@gmail.com >
Signed-off-by: Zhang Jian <jianmusings@gmail.com >
Signed-off-by: jian <jianmusings@gmail.com >
Signed-off-by: Canlin Guo <961750412@qq.com >
Co-authored-by: Canlin Guo <961750412@qq.com >
2026-06-02 15:46:01 +08:00
bc794e625f
Add Cosmos3 model support ( #3454 )
...
Signed-off-by: Maciej Bala <mbala@nvidia.com >
Signed-off-by: MaciejBalaNV <mbala@nvidia.com >
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: lishunyang12 <lishunyang12@163.com >
2026-06-02 04:04:31 +08:00
2b7249fbb3
[Model]Support MiniCPM-o 4.5 ( #3642 )
...
Signed-off-by: tc-mb <tianchi_cai@icloud.com >
Co-authored-by: GKangaroo <1095103651@qq.com >
Co-authored-by: GKangaroo <gqx24@mails.tsinghua.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-01 22:12:24 +08:00
8d83fb59e9
[BugFix] Fix two stop reason for multimodal output ( #3374 )
...
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Signed-off-by: Mike Qiu <qdy220091330@gmail.com >
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-01 21:51:29 +08:00
LHXuuu and vraiti
41a795b5d6
[Metrics] Add audio SLOs + cross-stage transfer families + per-(stage, replica) wrap for upstream vllm:* ( #3576 )
...
Signed-off-by: vraiti <vraiti@redhat.com >
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
Co-authored-by: vraiti <vraiti@redhat.com >
2026-06-01 20:20:48 +08:00
TaffyOfficial and TaffyOfficial
ee33601571
[Perf] Deduplicate AR prefix cache hidden-state CPU staging ( #3734 )
...
Signed-off-by: TaffyOfficial <2324465096@qq.com >
Co-authored-by: TaffyOfficial <2324465096@qq.com >
2026-06-01 09:35:07 +00:00
634af60fba
[Entrypoint] Add realtime OpenPI robot serving API ( #3673 )
...
Signed-off-by: Yangshen Deng <yangshen.d@outlook.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Meng <meng_chen99@163.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-06-01 14:47:23 +08:00
+1
f92d84fcf9
[Rebase] Rebase to vllm releases/v0.22.0 ( #3891 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: david6666666 <530634352@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: nainiu258 <cperfect02@163.com >
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com >
Co-authored-by: Zheng Wengang <zwg0606@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-06-01 11:08:12 +08:00