[Bugfix][MiniCPM-o] Build the Stage-0 duplex runtime at load time (#8146)
Signed-off-by: chickeyton <ngton2014@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
fe9e22df3d
commit
4be6bbe0e4
2 files changed
+71
No files matched your search
@@ -144,6 +144,16 @@ class MiniCPMO45OmniForConditionalGeneration(nn.Module, SupportsMultiModal, Supp
|
||||
# embeddings and initializes request-local codec generation state.
|
||||
self.has_preprocess = self.model_stage in {"llm", "tts"}
|
||||
|
||||
if self.model_stage == "llm" and getattr(vllm_config.model_config, "session_mode", "turn") == "duplex":
|
||||
# Build the Stage-0 duplex runtime (remote-code processor and
|
||||
# tokenizer) with the model. Built lazily, it costs several seconds
|
||||
# inside the first session's first audio unit, and the session then
|
||||
# runs that far behind the real-time input stream. The loader
|
||||
# constructs the model under the target-device context; the
|
||||
# processor is CPU preprocessing, so keep its tensors on the CPU.
|
||||
with torch.device("cpu"):
|
||||
self._duplex_data_plane_helper()
|
||||
|
||||
@cached_property
|
||||
def sampler(self):
|
||||
if hasattr(self.model, "sampler"):
|
||||
|
||||
Reference in new issue
Block a user