vllm-omni/examples/online_serving/personaplex

PersonaPlex full-duplex online serving

Serve nvidia/personaplex-7b-v1 (a Moshi-based full-duplex speech-to-speech model) with the native vLLM-Omni engine.

The server hosts the official PersonaPlex web client at / (auto-downloaded from the model repo) and implements its WebSocket protocol at /api/chat, so you talk to the model live in the browser — mic in, agent speech + inner-monologue text out.

Experimental. Requires a GPU and Hugging Face access to the gated repo (HF_TOKEN with access to nvidia/personaplex-7b-v1).

Start the server

HF_TOKEN=... CUDA_VISIBLE_DEVICES=0 bash run_server.sh
# or directly:
HF_TOKEN=... CUDA_VISIBLE_DEVICES=0 python -m \
    vllm_omni.experimental.fullduplex.personaplex.serving.server --port 8124

Talk to it

• Browser (recommended): open http://localhost:8124/ and allow the microphone. Use headphones so the agent does not hear itself.

• Headless: stream a 24 kHz mono WAV and save the reply:

  python duplex_client.py --url ws://localhost:8124/api/chat --input user.wav --out reply.wav
  

Unified Realtime API

The default vllm_omni/deploy/personaplex.yaml also enables main's engine-owned full-duplex control plane:

CUDA_VISIBLE_DEVICES=0 python -m vllm_omni.entrypoints.cli.main serve \
  /path/to/personaplex-7b-v1 \
  --omni \
  --deploy-config vllm_omni/deploy/personaplex.yaml

Validate the actual /v1/realtime?duplex=1 scheduler path with paced 24 kHz PCM, two concurrent sessions, overflow admission, per-session slot recycling, and non-silent output:

python tests/e2e/online_serving/personaplex_realtime_duplex.py \
  --model /path/to/personaplex-7b-v1 \
  --input-wav /path/to/speech.wav \
  --output-dir /tmp/personaplex-realtime-duplex

This is distinct from the standalone /api/chat compatibility server above. The unified endpoint advertises supports_barge_in=false: overlapping speech is native model behavior, but destructive output interruption and model-state rewind have not been validated for PersonaPlex.

Protocol (/api/chat)

Binary WebSocket messages, first byte is a tag (identical to moshi.server):

Direction	Tag	Payload
server → client	\x00	ready handshake (after the system-prompt prefill)
client → server	\x01	Opus-encoded mic audio (24 kHz)
server → client	\x01	Opus-encoded agent audio
server → client	\x02	UTF-8 inner-monologue text

Connect with query params text_prompt (persona) and voice_prompt (voice file). A raw-PCM endpoint (/v1/audio/duplex, JSON open + float32 frames) is also available for clients that do not want an Opus dependency.

Notes

• Run the client near the server. Real-time 80 ms audio is sensitive to network latency/jitter; over a high-latency remote link playback can stutter regardless of engine speed. On localhost it is smooth.
• Greedy decoding by default. One conversation per server by default (a second concurrent connection is rejected with WS close 1013 until the first ends); pass --batch-size N to host N concurrent conversations on one engine (elastic batching; connections beyond N are rejected until a slot frees).
• Full runbook: recipes/NVIDIA/PersonaPlex.md.