[Core][Frontend] Unified Full-duplex Framework (#7413)

Signed-off-by: chickeyton <ngton2014@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
chickeytonandClaude Opus 5 authored and GitHub committed 2026-09-16 15:35:36 +08:00
1 parent 9005d78903
commit 99ff4f3070
182 files changed
+26951 -34743

No files matched your search

+12 -5
View File
@@ -145,15 +145,22 @@ interchangeably.
| Incremental text input for speech synthesis | `WS /v1/audio/speech/stream` | Send text events and receive audio | [Streaming Text to Speech](speech_api.md#streaming-text-input-websocket) |
| Live video understanding | `WS /v1/video/chat/stream` | Send video frames and receive text/audio | [Streaming Video Input](video_stream_api.md) |
| Turn-based realtime audio | `WS /v1/realtime` | Stream one audio input and receive transcript/audio events | [Realtime Audio](realtime_api.md) |
| Continuous speech-to-speech interaction | `WS /v1/realtime?duplex=1` or `WS /v1/duplex` | Listen and speak concurrently with session control | [Full Duplex](full_duplex_api.md) |
| Continuous speech-to-speech interaction | `WS /v1/realtime?duplex=1` (alias `WS /v1/duplex`) | Listen and speak concurrently with session control | [Full Duplex](full_duplex_api.md) |
| Generated video chunks | `WS /v1/realtime/video` | Start a diffusion request and receive fragmented MP4 | [Streaming Video Output](streaming_video_output_api.md) |
| Robot policy inference | `WS /v1/realtime/robot/openpi` | Send MessagePack observations and receive action arrays | [OpenPI Robot Policy](openpi_api.md) |
All six routes are model- or configuration-dependent. In particular,
`/v1/realtime` is not full duplex unless the client sets `duplex=1` and the
deployment explicitly enables duplex sessions. Clients should also verify the
duplex capability payload because the query-parameter form falls back to the
ordinary realtime handler when duplex is unavailable.
`/v1/realtime` is full duplex when the model is a duplex model (its pipeline
declares a `duplex_plugin` and the deploy configuration sets
`session_mode: duplex`); a stock Realtime client needs no vendor query
parameter, and `duplex=0` is the explicit opt-out. Such a server serves the
websocket route plus `POST /v1/chat/completions`, and no other turn-based
HTTP route. On a duplex server that chat route is not the turn-based path:
each request runs on a short-lived duplex session, so it holds one of
`duplex_session.max_sessions` for its lifetime and answers at the model's
real-time pace -- see [Full Duplex](full_duplex_api.md). Clients should
also verify the duplex capability payload because the query-parameter form
falls back to the ordinary realtime handler when duplex is unavailable.
## Related Endpoints