tzhouam and Claude Opus 5
9a399adba8
chore: satisfy changed-file policy
...
Scoped pre-commit over the 27 files this rebase touches: ruff format
reflow, the vLLM-Omni SPDX copyright line on the new modules, and wording
the typos hook rejected.
The Python 3.10 mypy hook still reports 24 errors across five of these
files. All 24 sit on pre-existing lines; none fall on a line this rebase
changed, which the repository documents as the CI-skipped condition.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com >
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-09-10 00:41:53 +08:00
tzhouam and Claude Opus 5
c306445ff2
rebase: migrate diffusion KV cache to the vLLM 0.29 layout model
...
vLLM 0.29 removed AttentionSpec.indexes_kv_by_block_stride and reshaped
KVCacheTensor from (size, shared_by) to an explicit layout: layers in L
order plus layer_stride and block_stride. The block-stride decision now
lives on one physical CacheConfig.kv_cache_layout that the engine core
resolves once, and layout.is_block_outermost is the direct successor of
the old per-spec flag.
Diffusion backends subclass the Omni AttentionBackend ABC, so upstream
get_supported_kv_cache_layouts never sees them and the engine core never
resolves a layout for them. Add diffusion_kv/layout.py to pin it before
get_kv_cache_configs runs, defaulting to LBNHC, which reproduces the
indexes_kv_by_block_stride=False that every dense diffusion backend
declared. The backend hook is kept as an enforced contract rather than
deleted: a backend needing block-stride indexing selects a block-outermost
layout, and a contradiction raises instead of mis-addressing K/V.
Workers receive only a KVCacheConfig, so the resolved name travels on it
(KVCacheConfig.kv_cache_layout) and is adopted into each worker CacheConfig
before allocation. Without that, init_kv_cache raises in the worker because
get_resolved_kv_cache_layout has nothing to return.
init_kv_cache also lost attn_groups and cache_dtype in 0.29; the production
caller in model_runner_backend and the test stubs move to the six-argument
form.
Tensor construction goes through one derivation built on the upstream
compute_layout_strides helper, shared by production and tests, so a wrong
stride cannot be masked by a parallel test-only copy.
Also updates the sizing expectation: 0.29 subtracts the null block that
BlockPool permanently holds back before estimating, so sixteen physical
pages now yield fifteen usable ones.
One review point is addressed only in part, deliberately.
build_native_kv_cache_configs now takes indexes_kv_by_block_stride and
propagates a block-outermost layout end to end when it is set, with a test
covering control plane to worker adoption. Automatic discovery of that
requirement from the workers is not added: 0.29 removed the field that used
to carry it, every diffusion backend in tree declares False, and inventing
an executor RPC for a capability nothing requests would ship untestable
plumbing. Until a backend needs it, the seam is explicit and the mismatch
raises at worker init rather than corrupting addressing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com >
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-09-10 00:41:53 +08:00
tzhouam
f9879239ba
chore: satisfy changed-file policy
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-09-10 00:41:52 +08:00
tzhouam
87a1fd8da2
rebase: adapt to vLLM v0.29 APIs
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-09-10 00:41:52 +08:00
shiyichuan and mershi
06f67695f1
[CI/Build] Avoid duplicate stage CLI deploy config ( #7007 )
...
Signed-off-by: mershi <mershi@tencent.com >
Co-authored-by: mershi <mershi@tencent.com >
2026-09-09 22:05:44 +08:00
NancyFyong and NancyFyong
e74fe3c6e8
[Model] Add Audio8 TTS Preview 0.6B (DualAR, 44.1 kHz codec) ( #6157 )
...
Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com >
Co-authored-by: NancyFyong <NancyFyong@users.noreply.github.com >
2026-09-08 19:00:25 -07:00
boatman
cdf027874c
[Feat][OmniVoice]Support Varlen Attn, Request-Batch and Step-Execution ( #6408 )
...
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com >
2026-09-08 18:58:38 -07:00
fdaf4e31bd
Add vLLM-Omni library info to Hugging Face Hub requests ( #5381 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-08 10:08:38 -06:00
liangmenghuang and Claude Fable 5.1
7a80b79698
[Bugfix][Model] GR00T-N1.7: honor the per-request seed for flow-matching noise ( #7253 )
...
Signed-off-by: liangmengh <liangmengh@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-08 15:53:20 +00:00
Zhou Taichang and Claude Fable 5
142151fef5
Parallel stage initialization (admission + SH/EX device locks) (Worker refactor 1/N) ( #5224 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-09-05 22:22:41 +08:00
Hongsheng Liu
d4096e01fe
[Model] Add native MAGI-2 Preview diffusion support ( #5918 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-09-05 15:32:30 +08:00
akshatvishu and andyluo7
ae57e406fd
[Perf][CI] Use Whisper on GPU when memory permits ( #5675 )
...
Signed-off-by: akshatvishu <akshatnayak197@gmail.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: andyluo7 <andy.luo@amd.com >
2026-09-04 16:35:48 -07:00
Weiming Liao
0f9ee6afd8
[CI][Bugfix] Normalize spoken units in text comparison ( #6947 )
...
Signed-off-by: Weiming Liao <liaowm5@gmail.com >
2026-09-04 15:18:01 +00:00
95a9311696
[CI] Select mirror_hardwares presets via MIRROR_HW (H100/B200) ( #5543 )
...
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Signed-off-by: [Your Name] <your.email@example.com >
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com >
Co-authored-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Bo Li <22713281+bobboli@users.noreply.github.com >
2026-09-02 15:39:49 +00:00
andyluo7 and Hongsheng Liu
477083a103
[CI][ROCm] Fix deterministic AMD L3 blockers ( #6884 )
...
Signed-off-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-09-01 22:53:00 +08:00
AuFlow and Hongsheng Liu
49c6fc1540
[Frontend] Add configurable output sample rate to Speech API ( #6553 )
...
Signed-off-by: 郑倩倩 <2458983186@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-09-01 17:25:09 +08:00
akshatvishu
c41925689d
[BugFix][Higgs-Audio-V3] Fix transcript and concurrent sampling tests ( #6422 )
...
Signed-off-by: akshatvishu <akshatnayak197@gmail.com >
2026-08-30 08:52:15 +00:00
wangyu
e338f0ece3
[CI]Split GPU jobs by cards_* and reject hand-written SKU marks ( #6650 )
...
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: [Your Name] <your.email@example.com >
2026-08-29 18:42:33 +08:00
TJian and root
15c6ab8fa7
[AMD][CI] Fix AMD CI (Partial) ( #6704 )
...
Signed-off-by: root <root@tw020.mia.tensorwave.lan >
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: root <root@tw020.mia.tensorwave.lan >
2026-08-28 14:34:11 +08:00
Deep Shah and amy-why-3459
069e6c5fbc
[CI/Build][MiniCPM-o] Derive duplex admission-probe limit from the deploy config ( #6678 )
...
Signed-off-by: Deep Shah <deep@socratic.co >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-27 19:20:01 +08:00
zhumingjue138 and Gao Han
29e3165bb9
[Test] add marker for stability and add Wan2.2 npu test case ( #6343 )
...
Signed-off-by: zhumingjue <zhumingjue@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-08-25 17:08:58 +08:00
wangyu and amy-why-3459
4af0ddc1c2
[Test] Cleanup for helpers ( #6523 )
...
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: [Your Name] <your.email@example.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-25 09:10:17 +08:00
wangyu
1e2a6446f7
[CI] Remove AudioX and MammothModa2 support ( #6353 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-08-23 04:22:05 +00:00
Nick Cao
b2c9376539
[BugFix] Eliminate distributed test port race by switching to file:// rendezvous ( #6468 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-08-23 07:08:44 +08:00
Ian Eaves
5730af0a95
[Bugfix] Preserve prompt token usage details ( #5181 )
...
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com >
2026-08-20 20:05:05 +00:00
Nick Cao
ebb4abde0a
[Tests] Run diffusion tiny model tests in parallel ( #6339 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-08-20 12:46:32 -04:00
Shenglei Fu and Hongsheng Liu
47b1893676
[Perf][CI] Reuse the Whisper judge worker across a test module ( #6208 )
...
Signed-off-by: Shenglei Fu <sfu@confluent.io >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-20 10:49:44 +08:00
Yichen Wang and Didan Deng
2a9f6aaa8e
Add π0 VLA model support ( #4222 )
...
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
2026-08-17 14:29:17 +08:00
Shenglei Fu and amy-why-3459
5664e69dbf
[CI/Build] Gate one-word pronunciation on a success rate, not per request ( #5681 )
...
Signed-off-by: Shenglei Fu <sfu@confluent.io >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-17 14:02:50 +08:00
SYLAR
ce6ab0ca46
[Model] Add LTX-2.5 support ( #6070 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
2026-08-15 18:27:23 +08:00
Yukim1 and Hongsheng Liu
4b66f62e10
[Refactor] Remove internal stage_configs_path plumbing ( #5741 )
...
Signed-off-by: zwhzzz0821 <2831474076@qq.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-13 21:29:03 +08:00
fcf1085395
[Refactor][OutputProcessor 2/8]: OmniRequestOutput should inherit RequestOutput and no nested wrap-up ( #5146 )
...
Signed-off-by: Boao Shi <aoibosh@connect.hku.hk >
Signed-off-by: bowieshi <aoibosh@connect.hku.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: R2-Y <ruiruyang2@gmail.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: Shi Boao Bowie <aoibosh@connect.hku.hk >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: ruirui(rein) yang <73573651+R2-Y@users.noreply.github.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-12 22:58:23 +08:00
322600bfb2
[CI][MiniCPM-o] Add MiniCPM-o 4.5 accuracy and performance coverage ( #5524 )
...
Signed-off-by: R2-Y <ruiruyang2@gmail.com >
Signed-off-by: psv666 <2693925048@qq.com >
Co-authored-by: psv666 <2693925048@qq.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-11 14:53:49 +08:00
wtz2333 and Claude
36135ef09a
[diffusion][feature] Add LingBot-Video T2I and TI2V generation modes ( #5311 )
...
Signed-off-by: wtz2333 <2955110911@qq.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-10 10:02:31 +08:00
Alex Brooks
b15d4e6a64
[Frontend] Implement Batched Chat Completions ( #5317 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-08-07 16:12:09 -06:00
wangyu and amy-why-3459
4b625d1175
[CI/Build] Resolve hardware-nested perf baselines per concurrency sweep ( #5845 )
...
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: [Your Name] <your.email@example.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-06 21:20:04 +08:00
Yukim1
7a2007cc17
[Refactor] Remove legacy --stage-configs-path from the serve CLI ( #5647 )
...
Signed-off-by: zwhzzz0821 <2831474076@qq.com >
2026-08-03 23:41:52 +08:00
Shenglei Fu
9d67f8e02f
[CI/Build] Let audio assertions pin the transcript language ( #5646 )
...
Signed-off-by: Shenglei Fu <sfu@confluent.io >
2026-07-31 21:44:28 -07:00
NATURE
0a03359224
[Bugfix][MiniCPM-o] Stabilize single-GPU audio generation ( #5637 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-08-01 03:01:06 +00:00
273fc8eb1d
[Tests] Further extend the tiny model testing framework and enable multiple models ( #5353 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-07-31 07:47:26 +08:00
wangyu
7c17111ff5
[Refactor][CI] Reorganize test tree, and move slow e2e to weekly ( #4951 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-07-29 23:53:03 +08:00
Sy03 and Hongsheng Liu
0feb92bdfa
[Bugfix][Minicpm] Support full-duplex three-stage serving ( #5380 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-07-29 01:58:34 +08:00
5014fea4a0
[New Model] support nemotron Audex ( #4976 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-07-28 03:58:22 +00:00
b6c4578dac
[BugFix]Add MiniCPM-o 4.5 end-to-end test suite (offline, online serving) ( #5237 )
...
Signed-off-by: y_null <y_null@qq.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-07-25 17:00:13 -07:00
wangyu
e9a06d5bdc
[CI/Bugfix] Fix empty vllm_omni stub breaking stage CLI imports after Docker cache ( #5338 ) ( #5341 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-07-23 14:15:52 +00:00
Nick Cao
f10320f53f
[Tests] Config driven tiny model builder ( #5090 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-07-22 17:11:53 +00:00
Bingyu (Spencer) Liu
dcf8b07d39
[Core] Cleanup Omni connector runtime and transfer backends ( #5096 )
...
Signed-off-by: spencerr221 <liubingyu62@gmail.com >
2026-07-22 09:44:15 +08:00
Shenglei Fu
7b2edd6650
[Bugfix][Qwen3-TTS] Mode-aware ref_audio artifact readiness ( fixes #5049 ) ( #5157 )
2026-07-20 00:38:07 -07:00
Alex Brooks
9014bb9917
[CI] Fix core_model Level for non-Diffusion Models ( #5008 )
2026-07-12 03:00:50 +00:00
Alex Brooks
690864f277
[Tests] Tiny Diffusion Model Pattern ( #4524 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-07-09 09:03:51 -06:00
