apply() already tokenized the prompt to build the engine prompt ids, but the
shared OmniMultiModalProcessor._apply_hf_processor_main then called
_call_hf_processor again, which rebuilt the control-token prompt and ran the
text tokenizer a second time before encoding the reference audio. Upstream's
_cached_apply_hf_processor invokes _apply_hf_processor_main for every
request, so this also happened for text-only requests and for requests whose
reference audio was already cached.
Split the old _call_hf_processor into _encode_prompt_text (used once, in
apply) and _encode_reference_audio, and override _apply_hf_processor_main to
encode the reference audio only, returning just the passthrough data when
there is no audio. _get_mm_fields_config registers only ref_audio_tokens and
ref_audio_len, so dropping input_ids from the media result changes no
consumer. The now-unreferenced _call_hf_processor is deleted; the prompt-text
kwarg transport is left as is.
Validation: ruff and pre-commit on both files; the module imports cleanly;
new tests/model_executor/models/omnivoice/test_processor_prompt_tokenization.py
covers text-only apply (one tokenizer call), the audio path (audio encoded,
no re-tokenization) and the empty-items path (no tokenizer, no encoder);
tests/model_executor/models/omnivoice (106), tests/inputs/test_mm_processor.py
and tests/entrypoints/openai_api/test_tts_{adapter,detection}.py (162) pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpXe2LeRrUQGjZfKFaocMZ
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
The vLLM 0.29 rewrite left Qwen3OmniMoeThinkerMultiModalProcessor with a
_maybe_apply_prompt_updates that is identical to the one on its first base,
Omni's Qwen2_5OmniThinkerMultiModalProcessor (same body modulo docstring).
The MRO is Omni Qwen3 -> Omni Qwen2.5 -> upstream Qwen3 -> upstream Qwen2.5
-> BaseMultiModalProcessor, so deleting the override resolves to the same
code; asserted with
Qwen3OmniMoeThinkerMultiModalProcessor._maybe_apply_prompt_updates is
Qwen2_5OmniThinkerMultiModalProcessor._maybe_apply_prompt_updates.
Validation: ruff and pre-commit on the file; the MRO assertion above;
tests/model_executor/models/{qwen3_omni,qwen2_5_omni} pass (20 + 81 tests).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpXe2LeRrUQGjZfKFaocMZ
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
After the vLLM 0.29 adaptation the override only returned
super()._cached_apply_hf_processor(inputs, timing_ctx). Delete it together
with the imports it alone used (MultiModalProcessingInfo, TimingContext).
The find-only _maybe_apply_prompt_updates override is unchanged.
Validation: ruff and pre-commit on the file; the module imports cleanly;
tests/model_executor/models/voxtral_tts passes (62 tests).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpXe2LeRrUQGjZfKFaocMZ
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
vLLM 0.29 removed BaseMultiModalProcessor._call_hf_processor; the pipeline
now reaches the HF processor through _apply_hf_processor_main and the
_preprocess_hf_mm_data / _postprocess_hf_mm_data hooks. These three
overrides still called super()._call_hf_processor, which would raise
AttributeError if anything reached them:
- bagel: _apply_hf_processor_main already performs the image / img2img
split via self.info.ctx.call_hf_processor, so the override was unreachable.
- mimo_audio_llm: the audios -> audio rename and the sampling_rate kwarg
already live in _preprocess_hf_mm_data.
- step_audio2: the audios -> audio normalisation and the empty
audio_mels / audio_lens defaults already live in _preprocess_hf_mm_data
and _postprocess_hf_mm_data.
Caller audit: the only remaining _call_hf_processor call sites are the live
ones in vllm_omni/inputs/mm_processor.py, omnivoice.py and the GLM-TTS test.
Validation: ruff and pre-commit on the three files; the three modules
import cleanly; CPU tests in tests/inputs, tests/model_executor/models/
{mimo_audio,step_audio2} and test_upstream_prompt_update_api.py pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpXe2LeRrUQGjZfKFaocMZ
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
vLLM 0.29 deleted BaseMultiModalProcessor._hf_processor_applies_updates; no
reference to the hook remains anywhere in vllm/, so these six overrides were
never called. Delete them. No behaviour change.
Validation: pre-commit on the six files; CPU tests in tests/inputs and
tests/model_executor/models/{cosyvoice3,mimo_audio,step_audio2,omnivoice},
test_glm_tts_prompt_len.py and test_upstream_prompt_update_api.py pass;
the six processor modules import cleanly.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpXe2LeRrUQGjZfKFaocMZ
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Reuse upstream weight mapping and multimodal processing, consolidate Omni processor bridges, and trim unused Hunyuan internals. Route no-media kwargs through the active renderer and preserve GLM generation prompt formatting.
Validation: 453 CPU tests passed, 7 skipped; processor imports, Ruff, and diff checks passed. GPU and end-to-end CI not rerun.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
The API server consumes disable_log_stats and passes log_stats to AsyncOmni. Treat it as a frontend field during structured config resolution so existing TTS launch commands still start after rebasing onto the main config refactor. Retain validation for genuinely unowned stage options.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
The response preserves the native sample rate when none is requested. The HNR helper assumed 24 kHz, halving the perceived pitch of dots.tts-soar output and using the wrong autocorrelation window. Let PCM tests declare the native rate, honor requested resampling rates first, and retain the quality thresholds. Cover the failure with a 120 Hz synthetic voice and rate-selection tests.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Every omni worker already forces the v1 model runner because the v2 runner
carries no omni hooks (four worker classes do this). The scheduler was left
reading the config value, and vLLM 0.29 changed that default: 0.28 resolved
this config to v1, 0.29 ends its new fallback chain with v2.
The two then disagreed. The scheduler took its v2 fast path, which
deliberately omits resumed-request token ids ("No resumed reqs in fast
path"), while the worker-side v1 runner still read them, so resuming a
request raised KeyError: <req_id> in _update_states and killed the engine
core. That is the Multi-Replica Startup Test failure: it passes on main
6/6 and on the branch before the rebase (3005), and fails on every build
since.
Forcing it once in OmniSchedulerMixin covers both OmniARScheduler and
OmniGenerationScheduler, mirroring the wording the workers already use.
Adds tests/core/sched/test_omni_scheduler_forces_v1_model_runner.py, since a
default-sensitive regression like this would otherwise return silently: it
pins Omni's side of the contract rather than the upstream default, and was
negative-controlled by removing the fix (2 of 3 cases fail without it).
The 4x H100 multi-replica path itself is not reproducible here, so that part
is verified only by proxy: 165 sched tests, plus a 599-test
core/diffusion/ar_diffusion sweep, and the change is inert whenever the
config already resolves to v1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
vLLM 0.29 renamed the audio attention projection from qkv to qkv_proj, and
its stacked mapping now rewrites checkpoint q_proj/k_proj/v_proj onto that
name. Omni subclasses the upstream audio attention and mirrors its __init__
to add quant_config, so it kept building self.qkv and weight loading died
with "There is no module or parameter named layers.0.self_attn.qkv_proj in
Qwen3OmniMoeAudioEncoder ... available: self_attn.qkv.weight". The inherited
forward calls self.qkv_proj too, so the old name broke both paths.
This is a second, independent cause behind the Omni Accuracy Test failure:
the shutdown-timeout signature also appeared in that job, which is why the
earlier pass attributed it solely to the shutdown fix. Both builds 3009 and
3010 carry this weight-loading error, so it predates that pass.
Not reproducible locally: it needs the 30B checkpoint. The rename mirrors
upstream Qwen3OmniMoeAudioAttention exactly, and the 20 Qwen3-Omni unit tests
still pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
The MiniCPM-o select_text breakage cost eight CI jobs and surfaced only as
"Engine core initialization failed", because the replacement closures that
call these helpers are not driven by any unit test. A static check is what
actually catches the next removal.
Resolves each PromptUpdateDetails attribute referenced under vllm_omni/
against the installed class, so it tracks upstream instead of a hard-coded
list, and names the offending file in the failure. Verified by reintroducing
select_text: the guard fails with that file named, and passes once reverted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Five further removals/renames, each traced from a failing 3009 job:
- StageEngineCoreProcManager mirrors the parent __init__ rather than calling
super(), and 0.29 added _request_shutdown_timeout there. The inherited
shutdown() read it, raised AttributeError, and left the engine core
subprocesses running, so the test process hung at interpreter exit until
the 15 minute job timeout. That is the whole of the Engine&Model Executor
exit-124: its pytest reported 567 passed, then sat idle for 663s. The
failure surfaced only as a StagePool warning, never as a test failure.
- PromptUpdateDetails.select_text is gone; prompt updates are token oriented
now. MiniCPM-o 4.5 kept three text-based image/video/audio replacements,
which crashed stage-0 startup and account for the whole MiniCPM-o cluster.
They encode the text and select the unk placeholder by token id instead.
- Qwen3-Omni: 0.29 marks audio_feature_lengths keep_on_cpu, while the audio
tower derives device placement from feature_lens, so the mask comparison
mixed cuda and cpu tensors. Moved explicitly, matching upstream
Qwen3OmniMoeThinker._process_audio_input.
- AttentionBackend.get_kv_cache_shape is gone; the per-layer shape comes from
the spec via compute_layer_kv_cache_shape_bytes, whose content dim is in
bytes.
- WeightsMapper.get_unstacked_mapper is renamed to get_rename_mapper.
tests/engine/test_async_omni_engine_abort.py::test_abort fails here both with
and without these changes, so it is environmental rather than a regression.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Scoped pre-commit over the 27 files this rebase touches: ruff format
reflow, the vLLM-Omni SPDX copyright line on the new modules, and wording
the typos hook rejected.
The Python 3.10 mypy hook still reports 24 errors across five of these
files. All 24 sit on pre-existing lines; none fall on a line this rebase
changed, which the repository documents as the CI-skipped condition.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Four unrelated 0.29 surface changes, each reproduced locally against the
pinned f5c3cc24 build:
- PromptUpdateDetails.is_embed now takes only the full token list. The
Bagel img2img replacement still declared (tok, seq), which crashed
stage-0 engine core startup and produced the Distributed and Qwen3-Omni
"no live replica" failures rather than anything KV related.
- dispatch_unquantized_gemm gained a linear_backend argument and is called
with it. The CPU-forcing test stubs took no arguments; they now accept
and ignore whatever upstream passes so a further signature change does
not break them again.
- VocabParallelEmbedding gained a fused kernel that engages whenever
tp_size > 1 on CUDA, and it has no CPU implementation. The encoder TP
fixtures drive CPU tensors under a faked TP group, so they pin the
portable masked path.
- The pyav video decoder backend was removed; valid options are now
opencv, torchcodec, pynvvideocodec and deepstream. The OpenAI video
endpoint hardcoded pyav, which turned every video request into a 400.
It moves to opencv, the upstream default.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
vLLM 0.29 removed AttentionSpec.indexes_kv_by_block_stride and reshaped
KVCacheTensor from (size, shared_by) to an explicit layout: layers in L
order plus layer_stride and block_stride. The block-stride decision now
lives on one physical CacheConfig.kv_cache_layout that the engine core
resolves once, and layout.is_block_outermost is the direct successor of
the old per-spec flag.
Diffusion backends subclass the Omni AttentionBackend ABC, so upstream
get_supported_kv_cache_layouts never sees them and the engine core never
resolves a layout for them. Add diffusion_kv/layout.py to pin it before
get_kv_cache_configs runs, defaulting to LBNHC, which reproduces the
indexes_kv_by_block_stride=False that every dense diffusion backend
declared. The backend hook is kept as an enforced contract rather than
deleted: a backend needing block-stride indexing selects a block-outermost
layout, and a contradiction raises instead of mis-addressing K/V.
Workers receive only a KVCacheConfig, so the resolved name travels on it
(KVCacheConfig.kv_cache_layout) and is adopted into each worker CacheConfig
before allocation. Without that, init_kv_cache raises in the worker because
get_resolved_kv_cache_layout has nothing to return.
init_kv_cache also lost attn_groups and cache_dtype in 0.29; the production
caller in model_runner_backend and the test stubs move to the six-argument
form.
Tensor construction goes through one derivation built on the upstream
compute_layout_strides helper, shared by production and tests, so a wrong
stride cannot be masked by a parallel test-only copy.
Also updates the sizing expectation: 0.29 subtracts the null block that
BlockPool permanently holds back before estimating, so sixteen physical
pages now yield fifteen usable ones.
One review point is addressed only in part, deliberately.
build_native_kv_cache_configs now takes indexes_kv_by_block_stride and
propagates a block-outermost layout end to end when it is set, with a test
covering control plane to worker adoption. Automatic discovery of that
requirement from the workers is not added: 0.29 removed the field that used
to carry it, every diffusion backend in tree declares False, and inventing
an executor RPC for a capability nothing requests would ship untestable
plumbing. Until a backend needs it, the seam is explicit and the mismatch
raises at worker init rather than corrupting addressing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
