[Core][Model] Fit PersonaPlex into the Unified Full-duplex Framework (RFC #7181 PR 2) (#7695)

Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
This commit is contained in:
authored and GitHub committed 2026-09-27 03:02:24 +00:00
1 parent 7ab582ae25
commit 798d52ce55
49 files changed
+3056 -1715

No files matched your search

+36 -20
View File
@@ -2,15 +2,15 @@
Serve [`nvidia/personaplex-7b-v1`](https://huggingface.co/nvidia/personaplex-7b-v1)
(a Moshi-based full-duplex speech-to-speech model) with the native vLLM-Omni engine
through the unified duplex serving stack (`/v1/duplex` and `/v1/realtime?duplex=1`).
through the unified full-duplex framework (`/v1/realtime?duplex=1`, alias `/v1/duplex`).
> Requires a GPU and Hugging Face access to the gated repo
> (`HF_TOKEN` with access to `nvidia/personaplex-7b-v1`).
## Start the server
The default `vllm_omni/deploy/personaplex.yaml` enables the engine-owned
full-duplex control plane (`session_mode: duplex`):
The default `vllm_omni/deploy/personaplex.yaml` is a duplex deployment
(`session_mode: duplex`, two sessions per replica):
```bash
HF_TOKEN=... CUDA_VISIBLE_DEVICES=0 python -m vllm_omni.entrypoints.cli.main serve \
@@ -19,18 +19,32 @@ HF_TOKEN=... CUDA_VISIBLE_DEVICES=0 python -m vllm_omni.entrypoints.cli.main ser
--deploy-config vllm_omni/deploy/personaplex.yaml
```
This exposes:
This exposes `WS /v1/realtime?duplex=1` (alias `WS /v1/duplex`): the OpenAI
Realtime session protocol projected onto vLLM-Omni duplex sessions (client API and
wire protocol: [`docs/serving/realtime_duplex_api.md`](../../../docs/serving/realtime_duplex_api.md)).
There is no `/v1/chat/completions` route: PersonaPlex answers speech only.
- `WS /v1/duplex` — the native duplex session dialect
(`session.create` / `input_audio_buffer.append` / `response.output_audio.delta` ...);
- `WS /v1/realtime?duplex=1` — the same sessions projected onto the
OpenAI Realtime event vocabulary (client API and wire protocol:
[`docs/serving/realtime_duplex_api.md`](../../../docs/serving/realtime_duplex_api.md)).
PersonaPlex is a pure-lockstep model: every session is native duplex, audio flows
continuously in both directions in 80 ms frames, the model decides when to speak,
and there are no client commits or external turn signals
(`supports_client_commit=false`, `supports_external_turn_signal=false`). A session
therefore auto-responds without any vendor flag.
PersonaPlex is a pure-lockstep model: every session on a PersonaPlex deployment is
native duplex (`is_enabled()` is unconditionally true), audio flows continuously in
both directions, and there are no client commits or external turn signals
(`supports_client_commit=false`, `supports_external_turn_signal=false`).
## Talk to it
With the client library and the PersonaPlex preset (24 kHz `pcm_f32le` in, bundled
voice prompt, persona text):
```python
from vllm_omni.clients.duplex import DuplexClient
from vllm_omni.clients.personaplex import create_duplex_session_config
cfg = create_duplex_session_config(voice="NATF2.pt", persona="You are a concise assistant.")
async with DuplexClient("ws://127.0.0.1:8000/v1/realtime?duplex=1", model="/path/to/personaplex-7b-v1", config=cfg) as c:
await c.stream_pcm(pcm_f32le_24k) # keep streaming; the model speaks while it listens
```
Voice and persona are fixed for the session (`session.update` cannot change them).
## Validate the serving path
@@ -45,9 +59,11 @@ python tests/e2e/online_serving/personaplex_realtime_duplex.py \
--output-dir /tmp/personaplex-realtime-duplex
```
The unified endpoint advertises `supports_barge_in=false`: overlapping speech
is native model behavior, but destructive output interruption and model-state
rewind have not been validated for PersonaPlex.
The endpoint advertises `supports_barge_in=false`: overlapping speech is native
model behaviour, but destructive output interruption and model-state rewind have
not been validated for PersonaPlex. `response.cancel` and
`output_audio_buffer.clear` restart the model's conversation context (a fresh
Stage 0 request replays the voice/persona prefill).
## Notes
@@ -56,8 +72,8 @@ rewind have not been validated for PersonaPlex.
regardless of engine speed. On localhost it is smooth.
- The earlier standalone Moshi-web compatibility server (browser client at `/`,
binary WS protocol at `/api/chat`, raw-PCM `/v1/audio/duplex`) was demo-only
and has been removed; use the unified endpoints above.
- Session config (voice / persona / sampling) is passed per session via
`extra_body`; see
`vllm_omni/model_executor/models/personaplex/duplex/serving_adapter.py`.
and has been removed; use the unified endpoint above.
- The model plugin, worker-side lockstep runtime and input framing live in
`vllm_omni/model_executor/models/personaplex/duplex/`; design notes in
[`docs/design/fullduplex-personaplex.md`](../../../docs/design/fullduplex-personaplex.md).
- Full runbook: [`recipes/NVIDIA/PersonaPlex.md`](../../../recipes/NVIDIA/PersonaPlex.md).