anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196)

Browse Source 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
This commit is contained in:
chickeytonClaude Fable 5Gao Han
 authored and GitHub committed 2026-09-05 08:20:21 +00:00
1 parent d4096e01fe
commit 5d7fdf9350
179 files changed
+6462 -3832
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/serving/standalone_servers.md
+10 -52
View File
@@ -1,61 +1,19 @@
# Standalone Experimental Servers
Most vLLM-Omni endpoints run on the unified server started with
`vllm serve <model> --omni`. The experimental full-duplex package also contains
two separate user-facing server processes. Their routes are not added to the
unified API server.
`vllm serve <model> --omni`. The experimental package also contains one
separate user-facing server process; its routes are not added to the unified
API server.
| Server | Transport | Role |
|--------|-----------|------|
| PersonaPlex compatibility server | WebSocket | Hosts the official PersonaPlex browser protocol and a raw-PCM alternative |
| --- | --- | --- |
| JoyVL interaction server | HTTP | Adds state, memory, proactive decisions, and delegation in front of a separate OpenAI-compatible model backend |
## PersonaPlex Compatibility Server
Use this server when a client requires the official Moshi/PersonaPlex wire
protocol or the bundled browser UI. For the engine-owned unified duplex path,
use [`WS /v1/realtime?duplex=1`](full_duplex_api.md) instead.
Start it with:
```bash
HF_TOKEN=... CUDA_VISIBLE_DEVICES=0 python -m \
vllm_omni.experimental.fullduplex.personaplex.serving.server \
--port 8091
```
| Route | Purpose |
|-------|---------|
| `GET /` | Official PersonaPlex browser client, when its assets are available |
| `GET /health` | Readiness check |
| `WS /api/chat` | Moshi-compatible tagged binary protocol with Opus audio |
| `WS /v1/audio/duplex` | Simpler float32 raw-PCM protocol |
### `/api/chat` Protocol
The first byte of every binary frame is a message tag:
| Direction | Tag | Payload |
|-----------|-----|---------|
| Server to client | `0x00` | Ready handshake |
| Client to server | `0x01` | Opus microphone audio at 24 kHz |
| Server to client | `0x01` | Opus response audio |
| Server to client | `0x02` | UTF-8 inner-monologue text |
The optional `text_prompt` and `voice_prompt` query parameters select a persona
and voice. The default server admits one conversation; use `--batch-size N` to
configure multiple lockstep slots. Connections beyond capacity close with
WebSocket code 1013.
### `/v1/audio/duplex` Protocol
Send `{"type":"open","persona":"...","voice":"..."}` as a JSON text frame,
then exchange binary float32 PCM frames. The server returns generated float32
audio as binary frames and text as `{"type":"text","text":"..."}`. Send
`{"type":"close"}` and wait for `{"type":"done"}` to flush and finish.
See the [PersonaPlex example](https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/personaplex)
for the browser and headless clients.
PersonaPlex no longer ships a standalone compatibility server: it is served
through the unified duplex path (`vllm_omni/deploy/personaplex.yaml`) over
[`WS /v1/realtime?duplex=1`](full_duplex_api.md); see the
[PersonaPlex example](https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/personaplex)
for its clients.
## JoyVL Interaction Server
@@ -80,7 +38,7 @@ python -m vllm_omni.experimental.fullduplex.joyvl.serving.server \
```
| Method and route | Purpose |
|------------------|---------|
| ------------------ | --------- |
| `GET /health` | Readiness check |
| `GET /v1/models` | Reports the configured interaction model |
| `POST /v1/chat/completions` | Processes one frame or interaction tick |