Signed-off-by: chickeyton <ngton2014@gmail.com> Signed-off-by: Yueqian Lin <linyueqian@outlook.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
This commit is contained in:
49 files changed
+3056
-1715
No files matched your search
@@ -2,15 +2,15 @@
|
||||
|
||||
Serve [`nvidia/personaplex-7b-v1`](https://huggingface.co/nvidia/personaplex-7b-v1)
|
||||
(a Moshi-based full-duplex speech-to-speech model) with the native vLLM-Omni engine
|
||||
through the unified duplex serving stack (`/v1/duplex` and `/v1/realtime?duplex=1`).
|
||||
through the unified full-duplex framework (`/v1/realtime?duplex=1`, alias `/v1/duplex`).
|
||||
|
||||
> Requires a GPU and Hugging Face access to the gated repo
|
||||
> (`HF_TOKEN` with access to `nvidia/personaplex-7b-v1`).
|
||||
|
||||
## Start the server
|
||||
|
||||
The default `vllm_omni/deploy/personaplex.yaml` enables the engine-owned
|
||||
full-duplex control plane (`session_mode: duplex`):
|
||||
The default `vllm_omni/deploy/personaplex.yaml` is a duplex deployment
|
||||
(`session_mode: duplex`, two sessions per replica):
|
||||
|
||||
```bash
|
||||
HF_TOKEN=... CUDA_VISIBLE_DEVICES=0 python -m vllm_omni.entrypoints.cli.main serve \
|
||||
@@ -19,18 +19,32 @@ HF_TOKEN=... CUDA_VISIBLE_DEVICES=0 python -m vllm_omni.entrypoints.cli.main ser
|
||||
--deploy-config vllm_omni/deploy/personaplex.yaml
|
||||
```
|
||||
|
||||
This exposes:
|
||||
This exposes `WS /v1/realtime?duplex=1` (alias `WS /v1/duplex`): the OpenAI
|
||||
Realtime session protocol projected onto vLLM-Omni duplex sessions (client API and
|
||||
wire protocol: [`docs/serving/realtime_duplex_api.md`](../../../docs/serving/realtime_duplex_api.md)).
|
||||
There is no `/v1/chat/completions` route: PersonaPlex answers speech only.
|
||||
|
||||
- `WS /v1/duplex` — the native duplex session dialect
|
||||
(`session.create` / `input_audio_buffer.append` / `response.output_audio.delta` ...);
|
||||
- `WS /v1/realtime?duplex=1` — the same sessions projected onto the
|
||||
OpenAI Realtime event vocabulary (client API and wire protocol:
|
||||
[`docs/serving/realtime_duplex_api.md`](../../../docs/serving/realtime_duplex_api.md)).
|
||||
PersonaPlex is a pure-lockstep model: every session is native duplex, audio flows
|
||||
continuously in both directions in 80 ms frames, the model decides when to speak,
|
||||
and there are no client commits or external turn signals
|
||||
(`supports_client_commit=false`, `supports_external_turn_signal=false`). A session
|
||||
therefore auto-responds without any vendor flag.
|
||||
|
||||
PersonaPlex is a pure-lockstep model: every session on a PersonaPlex deployment is
|
||||
native duplex (`is_enabled()` is unconditionally true), audio flows continuously in
|
||||
both directions, and there are no client commits or external turn signals
|
||||
(`supports_client_commit=false`, `supports_external_turn_signal=false`).
|
||||
## Talk to it
|
||||
|
||||
With the client library and the PersonaPlex preset (24 kHz `pcm_f32le` in, bundled
|
||||
voice prompt, persona text):
|
||||
|
||||
```python
|
||||
from vllm_omni.clients.duplex import DuplexClient
|
||||
from vllm_omni.clients.personaplex import create_duplex_session_config
|
||||
|
||||
cfg = create_duplex_session_config(voice="NATF2.pt", persona="You are a concise assistant.")
|
||||
async with DuplexClient("ws://127.0.0.1:8000/v1/realtime?duplex=1", model="/path/to/personaplex-7b-v1", config=cfg) as c:
|
||||
await c.stream_pcm(pcm_f32le_24k) # keep streaming; the model speaks while it listens
|
||||
```
|
||||
|
||||
Voice and persona are fixed for the session (`session.update` cannot change them).
|
||||
|
||||
## Validate the serving path
|
||||
|
||||
@@ -45,9 +59,11 @@ python tests/e2e/online_serving/personaplex_realtime_duplex.py \
|
||||
--output-dir /tmp/personaplex-realtime-duplex
|
||||
```
|
||||
|
||||
The unified endpoint advertises `supports_barge_in=false`: overlapping speech
|
||||
is native model behavior, but destructive output interruption and model-state
|
||||
rewind have not been validated for PersonaPlex.
|
||||
The endpoint advertises `supports_barge_in=false`: overlapping speech is native
|
||||
model behaviour, but destructive output interruption and model-state rewind have
|
||||
not been validated for PersonaPlex. `response.cancel` and
|
||||
`output_audio_buffer.clear` restart the model's conversation context (a fresh
|
||||
Stage 0 request replays the voice/persona prefill).
|
||||
|
||||
## Notes
|
||||
|
||||
@@ -56,8 +72,8 @@ rewind have not been validated for PersonaPlex.
|
||||
regardless of engine speed. On localhost it is smooth.
|
||||
- The earlier standalone Moshi-web compatibility server (browser client at `/`,
|
||||
binary WS protocol at `/api/chat`, raw-PCM `/v1/audio/duplex`) was demo-only
|
||||
and has been removed; use the unified endpoints above.
|
||||
- Session config (voice / persona / sampling) is passed per session via
|
||||
`extra_body`; see
|
||||
`vllm_omni/model_executor/models/personaplex/duplex/serving_adapter.py`.
|
||||
and has been removed; use the unified endpoint above.
|
||||
- The model plugin, worker-side lockstep runtime and input framing live in
|
||||
`vllm_omni/model_executor/models/personaplex/duplex/`; design notes in
|
||||
[`docs/design/fullduplex-personaplex.md`](../../../docs/design/fullduplex-personaplex.md).
|
||||
- Full runbook: [`recipes/NVIDIA/PersonaPlex.md`](../../../recipes/NVIDIA/PersonaPlex.md).
|
||||
Reference in new issue
Block a user