main's #5730 added tests/model_executor/models/minicpmo_4_5/test_audio_embeds_input.py
with an identity tokenizer double that only implements encode/decode. On this
branch MiniCPMO45OmniLLMMultiModalProcessor._get_prompt_updates follows the
vLLM 0.29 prompt-update contract (PromptUpdateDetails.select_text is gone),
so it resolves the <unk> placeholder id once via
tokenizer.convert_tokens_to_ids("<unk>"), which the double lacks:
AttributeError: '_IdentityTokenizer' object has no attribute 'convert_tokens_to_ids'
Give the double that method. The tests still read only the placeholder text
from PromptUpdateDetails.full, so the returned id is irrelevant to them.
Branch-only rebase fallout; no production code changed.
Validation: the four failing cases in test_audio_embeds_input.py pass (5 passed);
tests/model_executor/models/minicpmo_4_5 otherwise unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpXe2LeRrUQGjZfKFaocMZ
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
apply() already tokenized the prompt to build the engine prompt ids, but the
shared OmniMultiModalProcessor._apply_hf_processor_main then called
_call_hf_processor again, which rebuilt the control-token prompt and ran the
text tokenizer a second time before encoding the reference audio. Upstream's
_cached_apply_hf_processor invokes _apply_hf_processor_main for every
request, so this also happened for text-only requests and for requests whose
reference audio was already cached.
Split the old _call_hf_processor into _encode_prompt_text (used once, in
apply) and _encode_reference_audio, and override _apply_hf_processor_main to
encode the reference audio only, returning just the passthrough data when
there is no audio. _get_mm_fields_config registers only ref_audio_tokens and
ref_audio_len, so dropping input_ids from the media result changes no
consumer. The now-unreferenced _call_hf_processor is deleted; the prompt-text
kwarg transport is left as is.
Validation: ruff and pre-commit on both files; the module imports cleanly;
new tests/model_executor/models/omnivoice/test_processor_prompt_tokenization.py
covers text-only apply (one tokenizer call), the audio path (audio encoded,
no re-tokenization) and the empty-items path (no tokenizer, no encoder);
tests/model_executor/models/omnivoice (106), tests/inputs/test_mm_processor.py
and tests/entrypoints/openai_api/test_tts_{adapter,detection}.py (162) pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpXe2LeRrUQGjZfKFaocMZ
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Reuse upstream weight mapping and multimodal processing, consolidate Omni processor bridges, and trim unused Hunyuan internals. Route no-media kwargs through the active renderer and preserve GLM generation prompt formatting.
Validation: 453 CPU tests passed, 7 skipped; processor imports, Ruff, and diff checks passed. GPU and end-to-end CI not rerun.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
The MiniCPM-o select_text breakage cost eight CI jobs and surfaced only as
"Engine core initialization failed", because the replacement closures that
call these helpers are not driven by any unit test. A static check is what
actually catches the next removal.
Resolves each PromptUpdateDetails attribute referenced under vllm_omni/
against the installed class, so it tracks upstream instead of a hard-coded
list, and names the offending file in the failure. Verified by reintroducing
select_text: the guard fails with that file named, and passes once reverted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
