apply() already tokenized the prompt to build the engine prompt ids, but the
shared OmniMultiModalProcessor._apply_hf_processor_main then called
_call_hf_processor again, which rebuilt the control-token prompt and ran the
text tokenizer a second time before encoding the reference audio. Upstream's
_cached_apply_hf_processor invokes _apply_hf_processor_main for every
request, so this also happened for text-only requests and for requests whose
reference audio was already cached.
Split the old _call_hf_processor into _encode_prompt_text (used once, in
apply) and _encode_reference_audio, and override _apply_hf_processor_main to
encode the reference audio only, returning just the passthrough data when
there is no audio. _get_mm_fields_config registers only ref_audio_tokens and
ref_audio_len, so dropping input_ids from the media result changes no
consumer. The now-unreferenced _call_hf_processor is deleted; the prompt-text
kwarg transport is left as is.
Validation: ruff and pre-commit on both files; the module imports cleanly;
new tests/model_executor/models/omnivoice/test_processor_prompt_tokenization.py
covers text-only apply (one tokenizer call), the audio path (audio encoded,
no re-tokenization) and the empty-items path (no tokenizer, no encoder);
tests/model_executor/models/omnivoice (106), tests/inputs/test_mm_processor.py
and tests/entrypoints/openai_api/test_tts_{adapter,detection}.py (162) pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpXe2LeRrUQGjZfKFaocMZ
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
