anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,956 Commits 235 Branches 31 Tags
c306445ff2a3cee456a1fbc054b175c868893bda
Commit Graph
2956 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouamandClaude Opus 5 c306445ff2 rebase: migrate diffusion KV cache to the vLLM 0.29 layout model 
vLLM 0.29 removed AttentionSpec.indexes_kv_by_block_stride and reshaped
KVCacheTensor from (size, shared_by) to an explicit layout: layers in L
order plus layer_stride and block_stride. The block-stride decision now
lives on one physical CacheConfig.kv_cache_layout that the engine core
resolves once, and layout.is_block_outermost is the direct successor of
the old per-spec flag.

Diffusion backends subclass the Omni AttentionBackend ABC, so upstream
get_supported_kv_cache_layouts never sees them and the engine core never
resolves a layout for them. Add diffusion_kv/layout.py to pin it before
get_kv_cache_configs runs, defaulting to LBNHC, which reproduces the
indexes_kv_by_block_stride=False that every dense diffusion backend
declared. The backend hook is kept as an enforced contract rather than
deleted: a backend needing block-stride indexing selects a block-outermost
layout, and a contradiction raises instead of mis-addressing K/V.

Workers receive only a KVCacheConfig, so the resolved name travels on it
(KVCacheConfig.kv_cache_layout) and is adopted into each worker CacheConfig
before allocation. Without that, init_kv_cache raises in the worker because
get_resolved_kv_cache_layout has nothing to return.

init_kv_cache also lost attn_groups and cache_dtype in 0.29; the production
caller in model_runner_backend and the test stubs move to the six-argument
form.

Tensor construction goes through one derivation built on the upstream
compute_layout_strides helper, shared by production and tests, so a wrong
stride cannot be masked by a parallel test-only copy.

Also updates the sizing expectation: 0.29 subtracts the null block that
BlockPool permanently holds back before estimating, so sixteen physical
pages now yield fifteen usable ones.

One review point is addressed only in part, deliberately.
build_native_kv_cache_configs now takes indexes_kv_by_block_stride and
propagates a block-outermost layout end to end when it is set, with a test
covering control plane to worker adoption. Automatic discovery of that
requirement from the workers is not added: 0.29 removed the field that used
to carry it, every diffusion backend in tree declares False, and inventing
an executor RPC for a capability nothing requests would ship untestable
plumbing. Until a backend needs it, the seam is explicit and the mismatch
raises at worker init rather than corrupting addressing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:53 +08:00
tzhouam f9879239ba chore: satisfy changed-file policy 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
tzhouam 87a1fd8da2 rebase: adapt to vLLM v0.29 APIs 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
tzhouam 6a63f5cb99 rebase: initialize vLLM v0.29 alignment 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
hyw 6fb7b0a05e [Model] Skip unused frame packing in Wan2.2 S2V (#7155) 
Signed-off-by: hyw <yuweih205@gmail.com>
 2026-09-09 14:34:46 +00:00
andyluo7 b9932aa17b [CI/Build][ROCm] Normalize SenseNova paged-decode hardware markers (#6935) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-09-09 07:32:38 -07:00
shiyichuanandmershi 06f67695f1 [CI/Build] Avoid duplicate stage CLI deploy config (#7007) 
Signed-off-by: mershi <mershi@tencent.com>
Co-authored-by: mershi <mershi@tencent.com>
 2026-09-09 22:05:44 +08:00
andyluo7andHongsheng Liu c94cf4ddde [CI][ROCm] Match CUDA/NPU L2/L3 label routing (#6966) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-09 13:47:15 +00:00
Shaun WalshClaude Opus 4.6Nick Cao
 		0e602509aa [BugFix] Add field validators for /v1/audio/generate request (#4741) 
Signed-off-by: Shaun Walsh <shaunwalsh24@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
 2026-09-09 13:38:42 +00:00
kunkun 2f1845fa4e fix: reserve Qwen3-Omni NVFP4 backend fix (#7200) 
Signed-off-by: kunkunblueberry <1833921874@qq.com>
 2026-09-09 20:35:16 +08:00
Zheng Wengangandamy-why-3459 7c1b1e4189 [Bugfix][Engine] Group overlapping device stages into one sequential init component (#7328) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-09 20:25:41 +08:00
zyz111222 6e79d582e3 [Bugfix][Ascend] fix npu 310/a5 bugs (#6685) 
Signed-off-by: zouyizhou <zouyizhou@huawei.com>
 2026-09-09 19:03:09 +08:00
NumberWan 038c9b948f [Bugfix] Resolve #6931 hub FA3 on torch 2.13 via kernels 0.16.1 (#7185) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-09-09 10:20:34 +00:00
Bo Li cdd4638b17 [Feature][Diffusion] Expose detailed pipeline timings (#6822) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-09-09 09:58:42 +00:00
Anjie HouandClaude Sonnet 5 ecdd762ff5 [2/N] Add a minimal temporal chunk callback for MiniMax-H3 (#7017) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-09 16:40:32 +08:00
Mu GuanLin ffa991b24f [Perf][Model] Add optimized LTX-2.5 DiffVAE operators (#7308) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-09 08:31:16 +00:00
chi030303 c66df06c0f [Doc] Qwen3-TTS: add 0.6B on 1x A100 40GB (#7289) 
Signed-off-by: chi030303 <106855944+chi030303@users.noreply.github.com>
 2026-09-09 07:56:14 +00:00
Sparks 7712701578 [Doc] [Recipe] Add FLUX.1-schnell recipe for RTX 5090 32GB (#7299) 
Signed-off-by: Sparks-M <41097544+Sparks-M@users.noreply.github.com>
 2026-09-09 07:54:46 +00:00
Mu GuanLin ab56148505 [Bugfix] Make LTX vocoder decoding deterministic (#7231) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-09 06:37:01 +00:00
NATURE 5b002727b0 [Core] Split Omni connector model runner mixin (#6903) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-09-09 13:49:43 +08:00
andyluo7 e0d0879794 [Bugfix][Magi2] Allow import without an active Triton driver (#7239) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-09-08 22:48:08 -07:00
wangyu 09b720b0d2 [CI] Skip Qwen3-Omni Server VAD multi-turn realtime test (#7279) (#7314) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-09-09 12:23:29 +08:00
xutianle 968f3e379c [Bugfix][CosyVoice3] Resolve hash snapshot pipeline (#6896) 
Signed-off-by: xutianle <xutianle@fudan.edu.cn>
 2026-09-08 20:33:38 -07:00
Allen WuandTRAE CLI f2e30f46b6 [Bugfix][TTS] Isolate shared Higgs v3 reference encode from request cancellation (#7076) 
Signed-off-by: Allen Wu <allenwu2795@gmail.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
 2026-09-08 20:32:05 -07:00
Jim Ban 358b7d79c1 [Bugfix][Frontend] Enforce image pixel limits for video input references (#6963) 
Signed-off-by: BANANASJIM <bananasjim1@gmail.com>
 2026-09-08 20:31:54 -07:00
Ziming Wang 43b697541a [Bugfix][Frontend] Clear the P0 multimodal cache through the renderer (#7003) 
Signed-off-by: ZenAlexa <zimingwang945@gmail.com>
 2026-09-08 20:31:42 -07:00
Rakesh KariyaCursorClaude Opus 5
 		2bae550c04 [Frontend] Opt-in WebSocket TTS split_granularity and session seed (#7046) 
Signed-off-by: rk9595 <rakesh.kariya@somaiya.edu>
Signed-off-by: Rakesh Kariya <rakesh.kariya@somaiya.edu>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-08 19:15:59 -07:00
zhengjiaandCursor 81c1c460b3 [Bugfix][Frontend] Accept the msgpack-numpy package's numpy markers on the OpenPI endpoint (#6051) 
Signed-off-by: zjli2013 <leezhengjiang@126.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-08 19:15:48 -07:00
NancyFyongandNancyFyong e74fe3c6e8 [Model] Add Audio8 TTS Preview 0.6B (DualAR, 44.1 kHz codec) (#6157) 
Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com>
Co-authored-by: NancyFyong <NancyFyong@users.noreply.github.com>
 2026-09-08 19:00:25 -07:00
boatman cdf027874c [Feat][OmniVoice]Support Varlen Attn, Request-Batch and Step-Execution (#6408) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-08 18:58:38 -07:00
eval d7678e2bb3 [Bugfix][MiniCPM-o] Fix the audio_embeds input path (#5730) 
Signed-off-by: eval-dev <0xe5bca0@gmail.com>
Signed-off-by: eval <74645252+eval-dev@users.noreply.github.com>
 2026-09-08 18:41:42 -07:00
Zhichao Zhang 5078c7ad66 [Bugfix][MiMo-Audio] Align code2wav decode with tokenizer device (#6539) 
Signed-off-by: chaosansui <zzc15560846421@163.com>
Signed-off-by: Zhichao Zhang <60429419+smartDream-chao@users.noreply.github.com>
 2026-09-08 18:38:11 -07:00
Hongsheng Liu f67cc52f74 [Doc] Add AI usage policy for contributions (#7305) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-09 09:17:24 +08:00
Anjie HouandClaude Opus 5 b6f4df320d [Refactor][Diffusion] Resolve offload topology through one plan resolver (#7209) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-09 08:17:38 +08:00
Tianyao Wu b3dd45874a [Bugfix] Build the forced-aligner prompt without a chat template (word timestamps one bin late) (#7240) 
Signed-off-by: Tianyao Wu <rayroy31@gmail.com>
 2026-09-08 19:27:38 +00:00
Lei KeandKrystalRay 5e4baea64d [Bugfix][NPU] Limit MiniMax H3 modulation grid size (#6794) 
Signed-off-by: KrystalRay <keeleiray@gmail.com>
Co-authored-by: KrystalRay <keeleiray@gmail.com>
 2026-09-08 09:08:46 -07:00
Harry MellorHongsheng LiuClaude Opus 5
 		fdaf4e31bd Add vLLM-Omni library info to Hugging Face Hub requests (#5381) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-08 10:08:38 -06:00
liangmenghuangandClaude Fable 5.1 7a80b79698 [Bugfix][Model] GR00T-N1.7: honor the per-request seed for flow-matching noise (#7253) 
Signed-off-by: liangmengh <liangmengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-08 15:53:20 +00:00
andyluo7 de1b0708bd [CI][ROCm] Route LTX2 Ulysses parity to two-GPU lane (#7234) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-09-08 22:59:37 +08:00
Hongsheng Liu 7422a85cc7 [Bugfix] Skip HWR store-size scans when no limit is configured (#7131) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-08 22:54:49 +08:00
YancyandAsthenia a31690041d [Bugfix][Examples] Use --profiler-config flag in offline TTS examples (#6763) 
Signed-off-by: Asthenia <asthenia0412@gmail.com>
Co-authored-by: Asthenia <asthenia0412@gmail.com>
 2026-09-08 13:56:27 +00:00
Honghan Zhuandamy-why-3459 d59d166485 [Bugfix] Preserve additional_config for LLM stages (#7272) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-08 20:44:49 +08:00
Shuolei Wang 5378b77be4 [Model][JoyAI-VL-Interaction] Add native multi-stage pipeline (#5352) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
 2026-09-08 05:07:08 -07:00
liangmenghuangandClaude Fable 5.1 d7740336ff [Bugfix][Model] GR00T-N1.7: honor processor_config image pipeline (use_albumentations / crop_fraction / letter_box_transform) to match Isaac-GR00T (#7083) 
Signed-off-by: liangmengh <liangmengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-08 18:59:01 +08:00
LHXuuu c9c25b33c3 [Frontend] Add server-side VAD for turn-based omni models (#6618) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
 2026-09-08 03:17:00 -07:00
Alicia 031612f250 [Bugfix] Fix video benchmark timeout and failure accounting (#7259) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-09-08 09:40:08 +00:00
andyluo7andwangyu b58ff5cb8b [CI/Build] Wait for duplex reaper recovery in retry test (#7225) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
 2026-09-08 17:10:51 +08:00
tlysanhuo 7c2e30afd3 [Bugfix] Bound omni benchmark per-request timeout to 15 min by default (#7130) 
Signed-off-by: tly <2200895168@qq.com>
 2026-09-08 02:00:25 -07:00
Chenguang Zheng f3fabe4145 [Bugfix] Keep --disable-log-stats out of stage config resolution (#7237) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-09-08 16:19:07 +08:00
Honghan Zhu c086554d1f [Perf]Boogu-Image Skip dense attention masks (#6871) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
 2026-09-08 00:53:20 -07:00