[Bugfix] Fix Nemotron VoiceChat duplex nightly handshake (#7884)

Signed-off-by: Sy03 <1370724210@qq.com>
This commit is contained in:
Sy03 authored and GitHub committed 2026-09-25 17:25:15 +00:00
1 parent 5d0963b3dc
commit d31a2a3ba8
26 files changed
+1186 -934

No files matched your search

+13 -4
View File
@@ -72,12 +72,21 @@ does not switch engines. The Python `Omni` / `AsyncOmni` APIs are unchanged.
`session.created.session.capabilities` is present before treating the
connection as full duplex.
**MiniCPM-o 4.5** (`vllm_omni/deploy/minicpmo_4_5.yaml`) is the only model
served over this endpoint today. PersonaPlex and Nemotron VoiceChat still carry
their pre-framework duplex code: their pipelines declare no `duplex_plugin`, so
they run turn-based until the follow-up PRs port them to the plugin contract
**MiniCPM-o 4.5** (`vllm_omni/deploy/minicpmo_4_5.yaml`), **Qwen3-Omni**
(`vllm_omni/deploy/qwen3_omni_duplex.yaml`), and **Nemotron VoiceChat**
(`vllm_omni/deploy/nemotron_labs_voicechat_duplex.yaml`) are served
over this endpoint today. PersonaPlex still carries its pre-framework duplex
code: its pipeline declares no `duplex_plugin`, so it runs turn-based until the
follow-up PR ports it to the plugin contract
(RFC [vllm-omni#7181](https://github.com/vllm-project/vllm-omni/issues/7181)).
For Nemotron VoiceChat Python clients, use
`vllm_omni.clients.nemotron_voicechat.create_duplex_session_config()` with
`DuplexClient`. The preset selects 16 kHz float32 input, 22.05 kHz PCM16 output,
and automatic responses. Append 1,280-sample (80 ms) input frames; when using
`stream_pcm`, set `chunk_ms=80`. Pass `instructions` and optional `tools` to
the preset before opening the session.
JoyVL is a separate HTTP interaction orchestrator and does not use these
WebSocket endpoints. See [Standalone Experimental Servers](standalone_servers.md).
+3 -8
View File
@@ -8,11 +8,9 @@ history stays honest, and a dropped connection can resume the same session.
This page covers how to run a duplex deployment, how to drive it from Python
with `vllm_omni.clients.duplex.DuplexClient`, and the complete wire contract.
The endpoint is served for models that ship a duplex plugin — including
MiniCPM-o 4.5 and Qwen3-Omni — and just for deploy configurations that declare
`session_mode: duplex`. PersonaPlex and Nemotron VoiceChat are ported to the
plugin contract in follow-up PRs and are not served over this endpoint yet. The runtime architecture
is described in [Full-Duplex Runtime (MiniCPM-o 4.5)](../design/fullduplex.md).
The endpoint requires a duplex plugin and a deployment with `session_mode: duplex`.
See [supported models and deployments](full_duplex_api.md#enable-full-duplex)
and the [runtime architecture](../design/fullduplex.md).
## Qwen3-Omni conversation history
@@ -1340,9 +1338,6 @@ them out into typed events before they reach a client.
## Known Limitations
- Only MiniCPM-o 4.5 is served over this endpoint today; PersonaPlex and
Nemotron VoiceChat arrive with the follow-up PRs that port them to the
plugin contract.
- Several surfaces are capability-gated per model (see *Capability
negotiation by model* above): PersonaPlex does not support session resume,
barge-in, or audio truncation; Nemotron VoiceChat does not support barge-in