Zhou Taichang
d9ab4aa1c0
Merge branch 'main' into fix/cfg-companion-output-race
2026-07-30 21:56:54 +08:00
evanchueng and david6666666
744c65b730
[Feature] Distributed layerwise offload with DP multi-concurrency + mmap weight… ( #5397 )
...
Signed-off-by: evanchueng <evanchueng@outlook.com >
Signed-off-by: david6666666 <530634352@qq.com >
Co-authored-by: david6666666 <530634352@qq.com >
2026-07-30 13:35:38 +00:00
Yukim1
b3f4fbf9e2
[Refactor] Migrate Step-Audio2 offline test to deploy config ( #5319 )
...
Signed-off-by: zwhzzz0821 <2831474076@qq.com >
2026-07-30 09:59:05 +00:00
Yukim1
1752e8ee5b
[Refactor] Migrate Qwen3 Omni thinker-only deploy config ( #5285 )
...
Signed-off-by: zwhzzz0821 <2831474076@qq.com >
2026-07-30 09:57:33 +00:00
Alex Brooks
089bb88706
[Model] Remove Text Encoder Output Layer Hardcoding in Flux2 ( #5589 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-07-30 09:55:42 +00:00
tzhouam and Claude Fable 5
e6ad8b67bf
fix(orchestrator): never dispatch diffusion with an incomplete CFG bundle
...
Nightly Bagel failures (builds 2892/2918) traced to a companion-output
race: _handle_kv_ready_raw_outputs marked a companion done on kv_ready
alone, before its processed output was stashed, so the parent could pass
the all_companions_done gate and _forward_to_next_stage dispatched with
0/N companion outputs - silently degraded CFG conditioning, and after an
abort a TP rank desync (mismatched all_reduce sizes, NCCL watchdog kill)
in the Bagel text-embedding path.
Lifecycle now enforces one invariant - diffusion dispatch never happens
with an incomplete companion bundle:
- a companion counts as done only after set_companion_output stashes its
processed output (_route_output is the sole done-marking site);
- the forward gate checks completeness non-destructively and re-defers
the parent instead of popping partial state; a bundle that is done but
gone without ever being consumed fails the request instead of
dispatching degraded CFG (was_bundled distinguishes streaming
re-submissions);
- a companion aborted directly (parent not in the batch) fails its
deferred parent immediately via the existing cleanup plumbing.
Deferred to a follow-up (needs a liveness owner in the orchestrator
loop): a deadline on deferred parents and detection of a companion
replica dying silently.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
Signed-off-by: tzhouam <tzhouam@gmail.com >
2026-07-30 15:22:38 +08:00
TaffyOfficial and TaffyOfficial
0c88f8a0b6
[Refactor] Normalize diffusion request extra arguments in Chat serving ( #5171 )
...
Signed-off-by: TaffyOfficial <2324465096@qq.com >
Co-authored-by: TaffyOfficial <2324465096@qq.com >
2026-07-30 07:08:12 +00:00
TJian
d50ea9fccc
[ROCm] [CI] Fix ROCm CI for v0.26.0 ( #5528 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-07-30 11:27:32 +08:00
Yash Jain and Claude Fable 5
daec3a6a49
[Bugfix] Trim AR sampled logprob rows with new_token_ids on mid-step stop ( #5564 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-30 03:00:53 +00:00
ruirui(rein) yang and Gao Han
0461da345a
[bugfix][MiniCPM-o]delete duplicate files ( #5535 )
...
Signed-off-by: R2-Y <ruiruyang2@gmail.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-07-30 09:59:44 +08:00
aa5364f705
[refactor][MiniCPM-o] Consolidate MiniCPM-o 4.5 2-GPU deploy configs and E2E tests ( #5458 )
...
Signed-off-by: y_null <y_null@qq.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-07-30 09:56:19 +08:00
Mu GuanLin
1b718f7b15
[Feat] [CI] [bugfix] Generalize LTX CFG parallelism to the complete guidance plan ( #5547 )
...
Signed-off-by: mglyn <1203789601@qq.com >
2026-07-29 17:40:40 +00:00
zzh and dengyunyang
dba05c3517
Enable Diffusion Image models asynchronous output ( #4981 )
...
Signed-off-by: zzh <943967662@qq.com >
Co-authored-by: dengyunyang <584797741@qq.com >
2026-07-29 17:21:55 +00:00
whisper
f2d9eb1630
[Bugfix][CI/Build] Avoid refetching HunyuanVideo I2V input during sim… ( #5552 )
...
Signed-off-by: whisper-la <131673492+whisper-la@users.noreply.github.com >
2026-07-29 09:49:07 -07:00
wangyu
7c17111ff5
[Refactor][CI] Reorganize test tree, and move slow e2e to weekly ( #4951 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-07-29 23:53:03 +08:00
zzh
925347332f
Add vae-patch-parallel-size and vae-use-tiling for HunyuanImage Benchmark ( #5502 )
...
Signed-off-by: zzh <943967662@qq.com >
2026-07-29 02:43:45 -07:00
R0CKSTAR
7be7e0d249
Support Python 3.10 runtime compatibility ( #5366 )
...
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com >
2026-07-29 17:11:04 +08:00
BruceLoveDecimal and Gao Han
1c6e731339
fix(diffusion): convert caller sampling params instead of discarding … ( #4115 )
...
Signed-off-by: BruceLoveDecimal <39156883+BruceLoveDecimal@users.noreply.github.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-07-29 13:24:51 +08:00
R0CKSTAR
ab0da662c0
[Hardware] Enable CosyVoice3 on Moore Threads MUSA ( #5164 )
...
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com >
2026-07-29 02:46:27 +00:00
iancarrasco-b10
23bb4395bb
[Feature] Flush the speech WebSocket on input.done without closing it ( #5517 )
...
Signed-off-by: iancarrasco-b10 <ian.carrasco@baseten.co >
2026-07-28 23:46:19 +00:00
Sy03 and Hongsheng Liu
0feb92bdfa
[Bugfix][Minicpm] Support full-duplex three-stage serving ( #5380 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-07-29 01:58:34 +08:00
Inesh Reddy Chappidi
c96fe54bd8
[Test] Add regression test for MiniCPM-o 4.5 audio-stream output kinds ( #5356 )
...
Signed-off-by: Inesh Reddy <ineshreddy249@gmail.com >
2026-07-28 16:27:51 +00:00
c2b55de5b9
[Test] Give the logprobs scheduler doubles the vLLM 0.26 contracts ( #5483 )
...
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: tzhouam <tzhouam@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-07-28 15:25:57 +00:00
herotai214
c96f0bc773
[Perf][GLM-Image] Skip redundant hidden state D2H ( #5346 )
...
Signed-off-by: herotai214 <herotai214@gmail.com >
2026-07-28 20:34:35 +08:00
Mu GuanLin
12d75cf63a
[Feature] Add official LTX multimodal guidance and precision guard ( #5148 )
...
Signed-off-by: mglyn <1203789601@qq.com >
2026-07-28 10:44:25 +00:00
Zhou Taichang and Claude Opus 5
66a9b0c84f
[Bugfix] Select the AR-Diffusion engine in the DreamZero CFG-parallel deploy config ( #5439 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-07-28 17:14:26 +08:00
wangyu
cee2e40c33
[CI] Skip known-failing tests and filter NPU ready TTS by npu mark ( #5489 )
2026-07-28 15:26:36 +08:00
5014fea4a0
[New Model] support nemotron Audex ( #4976 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-07-28 03:58:22 +00:00
Weiming Liao and wangyu
f2c5152df0
[CI][NPU]: Add nightly performance test for NPU - qwen3-omni/wan2.2/qwen-image-edit-2511 ( #5339 )
...
Signed-off-by: Weiming Liao <liaowm5@gmail.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
2026-07-28 11:44:30 +08:00
Wei and Samit
d3e0171cd7
[bugfix] validate AR sampled-token logprob alignment ( #5273 )
...
Signed-off-by: hbhflw2000 <417911774@qq.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-07-28 10:34:19 +08:00
Joshna-Medisetty
807db6efd7
[Refactor] Migrate Voxtral TTS XPU stage config to --deploy-config ( #5467 )
...
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com >
2026-07-28 07:35:35 +08:00
Yueqian Lin and Sy03
d4a5eaac90
[Bugfix][Realtime] Stop disconnected sessions from cycling through stages ( #5388 )
...
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Sy03 <1370724210@qq.com >
2026-07-27 17:04:17 +00:00
Yueqian Lin
8f115164c2
[Bugfix][Connectors] Flush processor tail on terminal async-chunk sends ( #5414 )
...
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-07-27 16:04:08 +00:00
cbab1d4d8f
[Rebase] Rebase to vllm 0.26.0 ( #5443 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: tzhouam <tzhouam@gmail.com >
2026-07-27 23:35:49 +08:00
LHXuuu and Yuanheng Zhao
fb89ab43ea
[Refactor][Ming-TTS] Move speaker extraction and caching to Stage-0 ( #5351 )
...
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com >
2026-07-27 07:55:47 -07:00
SYLAR and lishunyang12
7762d664d8
[Attention] Add trtllm diffusion attention backend with Skip-Softmax ( #5283 )
...
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
2026-07-27 22:48:11 +08:00
Wang fuyin and Hongsheng Liu
4dd1b47f47
[Config] Read stage metadata from VllmOmniConfig ( #5343 )
...
Signed-off-by: Acerak01-fy <wfy2003324@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-07-27 22:44:06 +08:00
e108cbb510
[NPU][HunyuanImage3] Adapt diffusion FusedMoE for vLLM >= 0.24.0 ( #5167 )
...
Signed-off-by: betta18 <jiangmengyu1@huawei.com >
Signed-off-by: jiangmengyu18 <56633611+jiangmengyu18@users.noreply.github.com >
Co-authored-by: betta18 <jiangmengyu1@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-07-27 22:41:34 +08:00
ruirui(rein) yang and amy-why-3459
82172b112d
[BugFix][MiniCPM-o] Restore audio output without async chunk ( #5455 )
...
Signed-off-by: R2-Y <ruiruyang2@gmail.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-07-27 22:19:46 +08:00
zzh
8001bb155d
Fix missing sequence_parallel_size in deploy config output ( #5449 )
...
Signed-off-by: zzh <943967662@qq.com >
2026-07-27 02:15:08 -07:00
汪志鹏
88b7ca2ffa
[Model] Support MammothModa2-Dev ( #5411 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-07-27 17:01:54 +08:00
rein yang
5b230541c9
[Deploy] Reduce MiniCPM-o single-GPU startup memory ( #5447 )
...
Signed-off-by: R2-Y <ruiruyang2@gmail.com >
2026-07-27 16:37:25 +08:00
Zhou Taichang and Claude Opus 4.8
776424f0de
[Refactor][WIP] Add missing test for modelrunner (G1/N) ( #5333 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-27 16:24:47 +08:00
xsmccc and xsmccc
d688aa82fc
[Model] Optimize MiniCPM-o multimodal encoder memory ( #5188 )
...
Signed-off-by: xsmccc <247721718+xsmccc@users.noreply.github.com >
Co-authored-by: xsmccc <247721718+xsmccc@users.noreply.github.com >
2026-07-27 15:24:51 +08:00
Zeyu Huang | 黃澤宇
eaf90d8eff
[Frontend] Add midway prompt update for streaming video generation ( #4652 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
2026-07-27 14:07:43 +08:00
wjsuijlenh
1f01d05603
[Core] Session state manager for AR-diffusion world models (RFC #4480 Phase 0) ( #4487 )
...
Signed-off-by: Wijnand Suijlen <wijnand.suijlen@huawei.com >
2026-07-27 13:17:38 +08:00
Inesh Reddy Chappidi
9f3a73df17
test: make Qwen-Image diffusers-backend latency check fair and stable ( #5108 ) ( #5132 )
...
Signed-off-by: Inesh Reddy <ineshreddy249@gmail.com >
2026-07-26 22:14:26 -07:00
Zheng Wengang
b9b8282cf5
[BugFix] Couple audio+video mm-cache for use_audio_in_video ( #5308 )
...
Signed-off-by: ZhengWG <zwg0606@gmail.com >
2026-07-27 04:29:53 +00:00
Yash Jain
911ea0c586
[Bugfix][Frontend] Parse VLLM_USE_MODELSCOPE via vllm.envs ( #5428 )
...
Signed-off-by: Yash Jain <yashjainjain1704@gmail.com >
2026-07-27 04:19:22 +00:00
Wallbreazzz
e28e389a76
feat(qwen3-tts): add codec_chunk_ramp for gradual chunk size warmup ( #5152 )
...
Signed-off-by: Wallbreazzz <110282866+Wallbreazzz@users.noreply.github.com >
2026-07-27 04:15:43 +00:00