tzhouam and Claude Opus 5
c306445ff2
rebase: migrate diffusion KV cache to the vLLM 0.29 layout model
...
vLLM 0.29 removed AttentionSpec.indexes_kv_by_block_stride and reshaped
KVCacheTensor from (size, shared_by) to an explicit layout: layers in L
order plus layer_stride and block_stride. The block-stride decision now
lives on one physical CacheConfig.kv_cache_layout that the engine core
resolves once, and layout.is_block_outermost is the direct successor of
the old per-spec flag.
Diffusion backends subclass the Omni AttentionBackend ABC, so upstream
get_supported_kv_cache_layouts never sees them and the engine core never
resolves a layout for them. Add diffusion_kv/layout.py to pin it before
get_kv_cache_configs runs, defaulting to LBNHC, which reproduces the
indexes_kv_by_block_stride=False that every dense diffusion backend
declared. The backend hook is kept as an enforced contract rather than
deleted: a backend needing block-stride indexing selects a block-outermost
layout, and a contradiction raises instead of mis-addressing K/V.
Workers receive only a KVCacheConfig, so the resolved name travels on it
(KVCacheConfig.kv_cache_layout) and is adopted into each worker CacheConfig
before allocation. Without that, init_kv_cache raises in the worker because
get_resolved_kv_cache_layout has nothing to return.
init_kv_cache also lost attn_groups and cache_dtype in 0.29; the production
caller in model_runner_backend and the test stubs move to the six-argument
form.
Tensor construction goes through one derivation built on the upstream
compute_layout_strides helper, shared by production and tests, so a wrong
stride cannot be masked by a parallel test-only copy.
Also updates the sizing expectation: 0.29 subtracts the null block that
BlockPool permanently holds back before estimating, so sixteen physical
pages now yield fifteen usable ones.
One review point is addressed only in part, deliberately.
build_native_kv_cache_configs now takes indexes_kv_by_block_stride and
propagates a block-outermost layout end to end when it is set, with a test
covering control plane to worker adoption. Automatic discovery of that
requirement from the workers is not added: 0.29 removed the field that used
to carry it, every diffusion backend in tree declares False, and inventing
an executor RPC for a capability nothing requests would ship untestable
plumbing. Until a backend needs it, the seam is explicit and the mismatch
raises at worker init rather than corrupting addressing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com >
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-09-10 00:41:53 +08:00
tzhouam
f9879239ba
chore: satisfy changed-file policy
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-09-10 00:41:52 +08:00
tzhouam
87a1fd8da2
rebase: adapt to vLLM v0.29 APIs
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-09-10 00:41:52 +08:00
shiyichuan and mershi
06f67695f1
[CI/Build] Avoid duplicate stage CLI deploy config ( #7007 )
...
Signed-off-by: mershi <mershi@tencent.com >
Co-authored-by: mershi <mershi@tencent.com >
2026-09-09 22:05:44 +08:00
NancyFyong and NancyFyong
e74fe3c6e8
[Model] Add Audio8 TTS Preview 0.6B (DualAR, 44.1 kHz codec) ( #6157 )
...
Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com >
Co-authored-by: NancyFyong <NancyFyong@users.noreply.github.com >
2026-09-08 19:00:25 -07:00
boatman
cdf027874c
[Feat][OmniVoice]Support Varlen Attn, Request-Batch and Step-Execution ( #6408 )
...
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com >
2026-09-08 18:58:38 -07:00
fdaf4e31bd
Add vLLM-Omni library info to Hugging Face Hub requests ( #5381 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-08 10:08:38 -06:00
liangmenghuang and Claude Fable 5.1
7a80b79698
[Bugfix][Model] GR00T-N1.7: honor the per-request seed for flow-matching noise ( #7253 )
...
Signed-off-by: liangmengh <liangmengh@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-08 15:53:20 +00:00
Zhou Taichang and Claude Fable 5
142151fef5
Parallel stage initialization (admission + SH/EX device locks) (Worker refactor 1/N) ( #5224 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-09-05 22:22:41 +08:00
Hongsheng Liu
d4096e01fe
[Model] Add native MAGI-2 Preview diffusion support ( #5918 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-09-05 15:32:30 +08:00
akshatvishu and andyluo7
ae57e406fd
[Perf][CI] Use Whisper on GPU when memory permits ( #5675 )
...
Signed-off-by: akshatvishu <akshatnayak197@gmail.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: andyluo7 <andy.luo@amd.com >
2026-09-04 16:35:48 -07:00
Weiming Liao
0f9ee6afd8
[CI][Bugfix] Normalize spoken units in text comparison ( #6947 )
...
Signed-off-by: Weiming Liao <liaowm5@gmail.com >
2026-09-04 15:18:01 +00:00
95a9311696
[CI] Select mirror_hardwares presets via MIRROR_HW (H100/B200) ( #5543 )
...
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Signed-off-by: [Your Name] <your.email@example.com >
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com >
Co-authored-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Bo Li <22713281+bobboli@users.noreply.github.com >
2026-09-02 15:39:49 +00:00
andyluo7 and Hongsheng Liu
477083a103
[CI][ROCm] Fix deterministic AMD L3 blockers ( #6884 )
...
Signed-off-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-09-01 22:53:00 +08:00
AuFlow and Hongsheng Liu
49c6fc1540
[Frontend] Add configurable output sample rate to Speech API ( #6553 )
...
Signed-off-by: 郑倩倩 <2458983186@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-09-01 17:25:09 +08:00
akshatvishu
c41925689d
[BugFix][Higgs-Audio-V3] Fix transcript and concurrent sampling tests ( #6422 )
...
Signed-off-by: akshatvishu <akshatnayak197@gmail.com >
2026-08-30 08:52:15 +00:00
wangyu
e338f0ece3
[CI]Split GPU jobs by cards_* and reject hand-written SKU marks ( #6650 )
...
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: [Your Name] <your.email@example.com >
2026-08-29 18:42:33 +08:00
TJian and root
15c6ab8fa7
[AMD][CI] Fix AMD CI (Partial) ( #6704 )
...
Signed-off-by: root <root@tw020.mia.tensorwave.lan >
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: root <root@tw020.mia.tensorwave.lan >
2026-08-28 14:34:11 +08:00
Deep Shah and amy-why-3459
069e6c5fbc
[CI/Build][MiniCPM-o] Derive duplex admission-probe limit from the deploy config ( #6678 )
...
Signed-off-by: Deep Shah <deep@socratic.co >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-27 19:20:01 +08:00
zhumingjue138 and Gao Han
29e3165bb9
[Test] add marker for stability and add Wan2.2 npu test case ( #6343 )
...
Signed-off-by: zhumingjue <zhumingjue@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-08-25 17:08:58 +08:00
wangyu and amy-why-3459
4af0ddc1c2
[Test] Cleanup for helpers ( #6523 )
...
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: [Your Name] <your.email@example.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-25 09:10:17 +08:00
wangyu
1e2a6446f7
[CI] Remove AudioX and MammothModa2 support ( #6353 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-08-23 04:22:05 +00:00
Nick Cao
b2c9376539
[BugFix] Eliminate distributed test port race by switching to file:// rendezvous ( #6468 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-08-23 07:08:44 +08:00
Ian Eaves
5730af0a95
[Bugfix] Preserve prompt token usage details ( #5181 )
...
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com >
2026-08-20 20:05:05 +00:00
Nick Cao
ebb4abde0a
[Tests] Run diffusion tiny model tests in parallel ( #6339 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-08-20 12:46:32 -04:00
Shenglei Fu and Hongsheng Liu
47b1893676
[Perf][CI] Reuse the Whisper judge worker across a test module ( #6208 )
...
Signed-off-by: Shenglei Fu <sfu@confluent.io >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-20 10:49:44 +08:00
Yichen Wang and Didan Deng
2a9f6aaa8e
Add π0 VLA model support ( #4222 )
...
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
2026-08-17 14:29:17 +08:00
Shenglei Fu and amy-why-3459
5664e69dbf
[CI/Build] Gate one-word pronunciation on a success rate, not per request ( #5681 )
...
Signed-off-by: Shenglei Fu <sfu@confluent.io >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-17 14:02:50 +08:00
SYLAR
ce6ab0ca46
[Model] Add LTX-2.5 support ( #6070 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
2026-08-15 18:27:23 +08:00
Yukim1 and Hongsheng Liu
4b66f62e10
[Refactor] Remove internal stage_configs_path plumbing ( #5741 )
...
Signed-off-by: zwhzzz0821 <2831474076@qq.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-13 21:29:03 +08:00
fcf1085395
[Refactor][OutputProcessor 2/8]: OmniRequestOutput should inherit RequestOutput and no nested wrap-up ( #5146 )
...
Signed-off-by: Boao Shi <aoibosh@connect.hku.hk >
Signed-off-by: bowieshi <aoibosh@connect.hku.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: R2-Y <ruiruyang2@gmail.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: Shi Boao Bowie <aoibosh@connect.hku.hk >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: ruirui(rein) yang <73573651+R2-Y@users.noreply.github.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-12 22:58:23 +08:00
322600bfb2
[CI][MiniCPM-o] Add MiniCPM-o 4.5 accuracy and performance coverage ( #5524 )
...
Signed-off-by: R2-Y <ruiruyang2@gmail.com >
Signed-off-by: psv666 <2693925048@qq.com >
Co-authored-by: psv666 <2693925048@qq.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-11 14:53:49 +08:00
wtz2333 and Claude
36135ef09a
[diffusion][feature] Add LingBot-Video T2I and TI2V generation modes ( #5311 )
...
Signed-off-by: wtz2333 <2955110911@qq.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-10 10:02:31 +08:00
Alex Brooks
b15d4e6a64
[Frontend] Implement Batched Chat Completions ( #5317 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-08-07 16:12:09 -06:00
wangyu and amy-why-3459
4b625d1175
[CI/Build] Resolve hardware-nested perf baselines per concurrency sweep ( #5845 )
...
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: [Your Name] <your.email@example.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-06 21:20:04 +08:00
Yukim1
7a2007cc17
[Refactor] Remove legacy --stage-configs-path from the serve CLI ( #5647 )
...
Signed-off-by: zwhzzz0821 <2831474076@qq.com >
2026-08-03 23:41:52 +08:00
Shenglei Fu
9d67f8e02f
[CI/Build] Let audio assertions pin the transcript language ( #5646 )
...
Signed-off-by: Shenglei Fu <sfu@confluent.io >
2026-07-31 21:44:28 -07:00
NATURE
0a03359224
[Bugfix][MiniCPM-o] Stabilize single-GPU audio generation ( #5637 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-08-01 03:01:06 +00:00
273fc8eb1d
[Tests] Further extend the tiny model testing framework and enable multiple models ( #5353 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-07-31 07:47:26 +08:00
wangyu
7c17111ff5
[Refactor][CI] Reorganize test tree, and move slow e2e to weekly ( #4951 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-07-29 23:53:03 +08:00
Sy03 and Hongsheng Liu
0feb92bdfa
[Bugfix][Minicpm] Support full-duplex three-stage serving ( #5380 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-07-29 01:58:34 +08:00
5014fea4a0
[New Model] support nemotron Audex ( #4976 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-07-28 03:58:22 +00:00
b6c4578dac
[BugFix]Add MiniCPM-o 4.5 end-to-end test suite (offline, online serving) ( #5237 )
...
Signed-off-by: y_null <y_null@qq.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-07-25 17:00:13 -07:00
wangyu
e9a06d5bdc
[CI/Bugfix] Fix empty vllm_omni stub breaking stage CLI imports after Docker cache ( #5338 ) ( #5341 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-07-23 14:15:52 +00:00
Nick Cao
f10320f53f
[Tests] Config driven tiny model builder ( #5090 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-07-22 17:11:53 +00:00
Bingyu (Spencer) Liu
dcf8b07d39
[Core] Cleanup Omni connector runtime and transfer backends ( #5096 )
...
Signed-off-by: spencerr221 <liubingyu62@gmail.com >
2026-07-22 09:44:15 +08:00
Shenglei Fu
7b2edd6650
[Bugfix][Qwen3-TTS] Mode-aware ref_audio artifact readiness ( fixes #5049 ) ( #5157 )
2026-07-20 00:38:07 -07:00
Alex Brooks
9014bb9917
[CI] Fix core_model Level for non-Diffusion Models ( #5008 )
2026-07-12 03:00:50 +00:00
Alex Brooks
690864f277
[Tests] Tiny Diffusion Model Pattern ( #4524 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-07-09 09:03:51 -06:00
leo
267b49b880
Fix Voxtral TTS feedback and short ASR checks ( #4954 )
...
Signed-off-by: leo <yaoliu548926@gmail.com >
2026-07-08 16:52:48 +00:00
