vllm-omni/docs/user_guide/examples/online_serving/aura_omni.md

AURA Omni: Online serving

aura_omni wires ASR, AURA, and Qwen3-TTS into one vLLM-Omni pipeline:

ASR -> AURA -> Qwen3-TTS Talker -> Code2Wav

Supported online path: Realtime duplex at /v1/realtime?duplex=1.

The default deploy profile (vllm_omni/deploy/aura_omni.yaml) sets session_mode: duplex. The aura_omni pipeline declares duplex_plugin, and DuplexOmniEngine requires that mode (same pattern as MiniCPM-o duplex deploys). Turn-based chat/completions is not a supported AURA online mode against this profile.

async_chunk defaults to false in that file (safer if a turn-based OmniOrchestrator ever loads it). Duplex Realtime Stage2→3 still needs async chunks — use the smoke deploy below, or set async_chunk: true yourself.

bash examples/online_serving/aura_omni/run_duplex_smoke_serve.sh
python examples/online_serving/aura_omni/smoke_duplex_realtime_client.py

Configure local checkpoints by editing per-stage model values in the deploy or smoke YAML. The file sets pipeline: aura_omni, so the four-stage topology is used even if the command-line --model points at one component checkpoint.

Silent Stage1 outputs (<|silent|> / id 151669) skip TTS for that turn.

GPU Utilization Recommendation

gpu_memory_utilization in vllm_omni/deploy/aura_omni.yaml controls how much VRAM each stage can reserve. Start with this split for a single GPU:

• Stage 0 (ASR): 0.10
• Stage 1 (AURA): 0.40
• Stage 2 (Qwen3-TTS Talker): 0.20
• Stage 3 (Qwen3-TTS Code2Wav): 0.20

TTS modes (stage extras)

When the duplex session supplies TTS extras, AURA text can feed Qwen3-TTS as:

• Base: voice clone from tts_ref_audio (optional x-vector-only mode)
• CustomVoice: predefined speaker (tts_speaker) with a CustomVoice checkpoint on stages 2 and 3

Optional tts_pass_token_ids passes AURA assistant token ids into Talker instead of detokenized text.

Old chat / Gradio / curl scripts (unsupported)

examples/online_serving/aura_omni/ still contains chat-completions, curl, and Gradio helpers from an older turn-based path. They do not work against the shipped duplex profile. That path was an incomplete stand-in before streaming I/O and session history were available; it is not a supported AURA mode. Use the Realtime duplex smoke client for online verification.