apply() already tokenized the prompt to build the engine prompt ids, but the
shared OmniMultiModalProcessor._apply_hf_processor_main then called
_call_hf_processor again, which rebuilt the control-token prompt and ran the
text tokenizer a second time before encoding the reference audio. Upstream's
_cached_apply_hf_processor invokes _apply_hf_processor_main for every
request, so this also happened for text-only requests and for requests whose
reference audio was already cached.
Split the old _call_hf_processor into _encode_prompt_text (used once, in
apply) and _encode_reference_audio, and override _apply_hf_processor_main to
encode the reference audio only, returning just the passthrough data when
there is no audio. _get_mm_fields_config registers only ref_audio_tokens and
ref_audio_len, so dropping input_ids from the media result changes no
consumer. The now-unreferenced _call_hf_processor is deleted; the prompt-text
kwarg transport is left as is.
Validation: ruff and pre-commit on both files; the module imports cleanly;
new tests/model_executor/models/omnivoice/test_processor_prompt_tokenization.py
covers text-only apply (one tokenizer call), the audio path (audio encoded,
no re-tokenization) and the empty-items path (no tokenizer, no encoder);
tests/model_executor/models/omnivoice (106), tests/inputs/test_mm_processor.py
and tests/entrypoints/openai_api/test_tts_{adapter,detection}.py (162) pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpXe2LeRrUQGjZfKFaocMZ
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Reuse upstream weight mapping and multimodal processing, consolidate Omni processor bridges, and trim unused Hunyuan internals. Route no-media kwargs through the active renderer and preserve GLM generation prompt formatting.
Validation: 453 CPU tests passed, 7 skipped; processor imports, Ruff, and diff checks passed. GPU and end-to-end CI not rerun.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
The API server consumes disable_log_stats and passes log_stats to AsyncOmni. Treat it as a frontend field during structured config resolution so existing TTS launch commands still start after rebasing onto the main config refactor. Retain validation for genuinely unowned stage options.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
The response preserves the native sample rate when none is requested. The HNR helper assumed 24 kHz, halving the perceived pitch of dots.tts-soar output and using the wrong autocorrelation window. Let PCM tests declare the native rate, honor requested resampling rates first, and retain the quality thresholds. Cover the failure with a 120 Hz synthetic voice and rate-selection tests.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Every omni worker already forces the v1 model runner because the v2 runner
carries no omni hooks (four worker classes do this). The scheduler was left
reading the config value, and vLLM 0.29 changed that default: 0.28 resolved
this config to v1, 0.29 ends its new fallback chain with v2.
The two then disagreed. The scheduler took its v2 fast path, which
deliberately omits resumed-request token ids ("No resumed reqs in fast
path"), while the worker-side v1 runner still read them, so resuming a
request raised KeyError: <req_id> in _update_states and killed the engine
core. That is the Multi-Replica Startup Test failure: it passes on main
6/6 and on the branch before the rebase (3005), and fails on every build
since.
Forcing it once in OmniSchedulerMixin covers both OmniARScheduler and
OmniGenerationScheduler, mirroring the wording the workers already use.
Adds tests/core/sched/test_omni_scheduler_forces_v1_model_runner.py, since a
default-sensitive regression like this would otherwise return silently: it
pins Omni's side of the contract rather than the upstream default, and was
negative-controlled by removing the fix (2 of 3 cases fail without it).
The 4x H100 multi-replica path itself is not reproducible here, so that part
is verified only by proxy: 165 sched tests, plus a 599-test
core/diffusion/ar_diffusion sweep, and the change is inert whenever the
config already resolves to v1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
The MiniCPM-o select_text breakage cost eight CI jobs and surfaced only as
"Engine core initialization failed", because the replacement closures that
call these helpers are not driven by any unit test. A static check is what
actually catches the next removal.
Resolves each PromptUpdateDetails attribute referenced under vllm_omni/
against the installed class, so it tracks upstream instead of a hard-coded
list, and names the offending file in the failure. Verified by reintroducing
select_text: the guard fails with that file named, and passes once reverted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Five further removals/renames, each traced from a failing 3009 job:
- StageEngineCoreProcManager mirrors the parent __init__ rather than calling
super(), and 0.29 added _request_shutdown_timeout there. The inherited
shutdown() read it, raised AttributeError, and left the engine core
subprocesses running, so the test process hung at interpreter exit until
the 15 minute job timeout. That is the whole of the Engine&Model Executor
exit-124: its pytest reported 567 passed, then sat idle for 663s. The
failure surfaced only as a StagePool warning, never as a test failure.
- PromptUpdateDetails.select_text is gone; prompt updates are token oriented
now. MiniCPM-o 4.5 kept three text-based image/video/audio replacements,
which crashed stage-0 startup and account for the whole MiniCPM-o cluster.
They encode the text and select the unk placeholder by token id instead.
- Qwen3-Omni: 0.29 marks audio_feature_lengths keep_on_cpu, while the audio
tower derives device placement from feature_lens, so the mask comparison
mixed cuda and cpu tensors. Moved explicitly, matching upstream
Qwen3OmniMoeThinker._process_audio_input.
- AttentionBackend.get_kv_cache_shape is gone; the per-layer shape comes from
the spec via compute_layer_kv_cache_shape_bytes, whose content dim is in
bytes.
- WeightsMapper.get_unstacked_mapper is renamed to get_rename_mapper.
tests/engine/test_async_omni_engine_abort.py::test_abort fails here both with
and without these changes, so it is environmental rather than a regression.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Scoped pre-commit over the 27 files this rebase touches: ruff format
reflow, the vLLM-Omni SPDX copyright line on the new modules, and wording
the typos hook rejected.
The Python 3.10 mypy hook still reports 24 errors across five of these
files. All 24 sit on pre-existing lines; none fall on a line this rebase
changed, which the repository documents as the CI-skipped condition.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Four unrelated 0.29 surface changes, each reproduced locally against the
pinned f5c3cc24 build:
- PromptUpdateDetails.is_embed now takes only the full token list. The
Bagel img2img replacement still declared (tok, seq), which crashed
stage-0 engine core startup and produced the Distributed and Qwen3-Omni
"no live replica" failures rather than anything KV related.
- dispatch_unquantized_gemm gained a linear_backend argument and is called
with it. The CPU-forcing test stubs took no arguments; they now accept
and ignore whatever upstream passes so a further signature change does
not break them again.
- VocabParallelEmbedding gained a fused kernel that engages whenever
tp_size > 1 on CUDA, and it has no CPU implementation. The encoder TP
fixtures drive CPU tensors under a faked TP group, so they pin the
portable masked path.
- The pyav video decoder backend was removed; valid options are now
opencv, torchcodec, pynvvideocodec and deepstream. The OpenAI video
endpoint hardcoded pyav, which turned every video request into a 400.
It moves to opencv, the upstream default.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
vLLM 0.29 removed AttentionSpec.indexes_kv_by_block_stride and reshaped
KVCacheTensor from (size, shared_by) to an explicit layout: layers in L
order plus layer_stride and block_stride. The block-stride decision now
lives on one physical CacheConfig.kv_cache_layout that the engine core
resolves once, and layout.is_block_outermost is the direct successor of
the old per-spec flag.
Diffusion backends subclass the Omni AttentionBackend ABC, so upstream
get_supported_kv_cache_layouts never sees them and the engine core never
resolves a layout for them. Add diffusion_kv/layout.py to pin it before
get_kv_cache_configs runs, defaulting to LBNHC, which reproduces the
indexes_kv_by_block_stride=False that every dense diffusion backend
declared. The backend hook is kept as an enforced contract rather than
deleted: a backend needing block-stride indexing selects a block-outermost
layout, and a contradiction raises instead of mis-addressing K/V.
Workers receive only a KVCacheConfig, so the resolved name travels on it
(KVCacheConfig.kv_cache_layout) and is adopted into each worker CacheConfig
before allocation. Without that, init_kv_cache raises in the worker because
get_resolved_kv_cache_layout has nothing to return.
init_kv_cache also lost attn_groups and cache_dtype in 0.29; the production
caller in model_runner_backend and the test stubs move to the six-argument
form.
Tensor construction goes through one derivation built on the upstream
compute_layout_strides helper, shared by production and tests, so a wrong
stride cannot be masked by a parallel test-only copy.
Also updates the sizing expectation: 0.29 subtracts the null block that
BlockPool permanently holds back before estimating, so sixteen physical
pages now yield fifteen usable ones.
One review point is addressed only in part, deliberately.
build_native_kv_cache_configs now takes indexes_kv_by_block_stride and
propagates a block-outermost layout end to end when it is set, with a test
covering control plane to worker adoption. Automatic discovery of that
requirement from the workers is not added: 0.29 removed the field that used
to carry it, every diffusion backend in tree declares False, and inventing
an executor RPC for a capability nothing requests would ship untestable
plumbing. Until a backend needs it, the seam is explicit and the mismatch
raises at worker init rather than corrupting addressing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
