Five further removals/renames, each traced from a failing 3009 job:
- StageEngineCoreProcManager mirrors the parent __init__ rather than calling
super(), and 0.29 added _request_shutdown_timeout there. The inherited
shutdown() read it, raised AttributeError, and left the engine core
subprocesses running, so the test process hung at interpreter exit until
the 15 minute job timeout. That is the whole of the Engine&Model Executor
exit-124: its pytest reported 567 passed, then sat idle for 663s. The
failure surfaced only as a StagePool warning, never as a test failure.
- PromptUpdateDetails.select_text is gone; prompt updates are token oriented
now. MiniCPM-o 4.5 kept three text-based image/video/audio replacements,
which crashed stage-0 startup and account for the whole MiniCPM-o cluster.
They encode the text and select the unk placeholder by token id instead.
- Qwen3-Omni: 0.29 marks audio_feature_lengths keep_on_cpu, while the audio
tower derives device placement from feature_lens, so the mask comparison
mixed cuda and cpu tensors. Moved explicitly, matching upstream
Qwen3OmniMoeThinker._process_audio_input.
- AttentionBackend.get_kv_cache_shape is gone; the per-layer shape comes from
the spec via compute_layer_kv_cache_shape_bytes, whose content dim is in
bytes.
- WeightsMapper.get_unstacked_mapper is renamed to get_rename_mapper.
tests/engine/test_async_omni_engine_abort.py::test_abort fails here both with
and without these changes, so it is environmental rather than a regression.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Scoped pre-commit over the 27 files this rebase touches: ruff format
reflow, the vLLM-Omni SPDX copyright line on the new modules, and wording
the typos hook rejected.
The Python 3.10 mypy hook still reports 24 errors across five of these
files. All 24 sit on pre-existing lines; none fall on a line this rebase
changed, which the repository documents as the CI-skipped condition.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Four unrelated 0.29 surface changes, each reproduced locally against the
pinned f5c3cc24 build:
- PromptUpdateDetails.is_embed now takes only the full token list. The
Bagel img2img replacement still declared (tok, seq), which crashed
stage-0 engine core startup and produced the Distributed and Qwen3-Omni
"no live replica" failures rather than anything KV related.
- dispatch_unquantized_gemm gained a linear_backend argument and is called
with it. The CPU-forcing test stubs took no arguments; they now accept
and ignore whatever upstream passes so a further signature change does
not break them again.
- VocabParallelEmbedding gained a fused kernel that engages whenever
tp_size > 1 on CUDA, and it has no CPU implementation. The encoder TP
fixtures drive CPU tensors under a faked TP group, so they pin the
portable masked path.
- The pyav video decoder backend was removed; valid options are now
opencv, torchcodec, pynvvideocodec and deepstream. The OpenAI video
endpoint hardcoded pyav, which turned every video request into a 400.
It moves to opencv, the upstream default.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
vLLM 0.29 removed AttentionSpec.indexes_kv_by_block_stride and reshaped
KVCacheTensor from (size, shared_by) to an explicit layout: layers in L
order plus layer_stride and block_stride. The block-stride decision now
lives on one physical CacheConfig.kv_cache_layout that the engine core
resolves once, and layout.is_block_outermost is the direct successor of
the old per-spec flag.
Diffusion backends subclass the Omni AttentionBackend ABC, so upstream
get_supported_kv_cache_layouts never sees them and the engine core never
resolves a layout for them. Add diffusion_kv/layout.py to pin it before
get_kv_cache_configs runs, defaulting to LBNHC, which reproduces the
indexes_kv_by_block_stride=False that every dense diffusion backend
declared. The backend hook is kept as an enforced contract rather than
deleted: a backend needing block-stride indexing selects a block-outermost
layout, and a contradiction raises instead of mis-addressing K/V.
Workers receive only a KVCacheConfig, so the resolved name travels on it
(KVCacheConfig.kv_cache_layout) and is adopted into each worker CacheConfig
before allocation. Without that, init_kv_cache raises in the worker because
get_resolved_kv_cache_layout has nothing to return.
init_kv_cache also lost attn_groups and cache_dtype in 0.29; the production
caller in model_runner_backend and the test stubs move to the six-argument
form.
Tensor construction goes through one derivation built on the upstream
compute_layout_strides helper, shared by production and tests, so a wrong
stride cannot be masked by a parallel test-only copy.
Also updates the sizing expectation: 0.29 subtracts the null block that
BlockPool permanently holds back before estimating, so sixteen physical
pages now yield fifteen usable ones.
One review point is addressed only in part, deliberately.
build_native_kv_cache_configs now takes indexes_kv_by_block_stride and
propagates a block-outermost layout end to end when it is set, with a test
covering control plane to worker adoption. Automatic discovery of that
requirement from the workers is not added: 0.29 removed the field that used
to carry it, every diffusion backend in tree declares False, and inventing
an executor RPC for a capability nothing requests would ship untestable
plumbing. Until a backend needs it, the seam is explicit and the mismatch
raises at worker init rather than corrupting addressing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
