[Bugfix][MiniCPM-o] Build the Stage-0 duplex runtime at load time (#8146)

Signed-off-by: chickeyton <ngton2014@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
chickeytonandClaude Opus 5.5 authored and GitHub committed 2026-09-25 14:57:23 +00:00
1 parent fe9e22df3d
commit 4be6bbe0e4
2 files changed
+71

No files matched your search

@@ -144,6 +144,16 @@ class MiniCPMO45OmniForConditionalGeneration(nn.Module, SupportsMultiModal, Supp
# embeddings and initializes request-local codec generation state.
self.has_preprocess = self.model_stage in {"llm", "tts"}
if self.model_stage == "llm" and getattr(vllm_config.model_config, "session_mode", "turn") == "duplex":
# Build the Stage-0 duplex runtime (remote-code processor and
# tokenizer) with the model. Built lazily, it costs several seconds
# inside the first session's first audio unit, and the session then
# runs that far behind the real-time input stream. The loader
# constructs the model under the target-device context; the
# processor is CPU preprocessing, so keep its tensors on the CPU.
with torch.device("cpu"):
self._duplex_data_plane_helper()
@cached_property
def sampler(self):
if hasattr(self.model, "sampler"):