vllm-omni/examples/online_serving/aura_omni

AURA Omni: Online serving

aura_omni serves AURA as a native multi-stage vLLM-Omni pipeline:

Qwen3-ASR -> AURA/Qwen3-VL -> Qwen3-TTS Talker -> Qwen3-TTS Code2Wav

AURA's supported online path is Realtime duplex (/v1/realtime?duplex=1). The default deploy profile sets session_mode: duplex because the pipeline declares duplex_plugin and DuplexOmniEngine requires that mode.

This is AURA v1 / Qwen3-VL (silent / ChatML ids 151669 / 151645), not AURA v2 / Qwen3.5-VL.

Duplex Realtime (only supported online path)

Stage2→3 codec handoff needs async_chunk: true. The default vllm_omni/deploy/aura_omni.yaml keeps async_chunk: false as a safer baseline (turn-based OmniOrchestrator never sees the duplex async-chunk gate). For Realtime, use the smoke deploy (already async_chunk: true) or set that flag yourself:

bash examples/online_serving/aura_omni/run_duplex_smoke_serve.sh
python examples/online_serving/aura_omni/smoke_duplex_realtime_client.py

Smoke deploy file: examples/online_serving/aura_omni/aura_omni_duplex_smoke.yaml.

Equivalent serve with the default topology and async chunks enabled:

# Copy or edit so async_chunk: true, then:
vllm serve aurateam/AURA \
  --omni \
  --port 8091 \
  --deploy-config examples/online_serving/aura_omni/aura_omni_duplex_smoke.yaml \
  --served-model-name aurateam/AURA \
  --trust-remote-code

Connect clients to /v1/realtime?duplex=1. Silent Stage1 outputs gate TTS (no audio for that turn). Overlapped input and vision-follow are AURA duplex capabilities.

Browser UI

AURA has no client VAD: control is push-to-talk on the shared shell from #7585 (examples/online_serving/realtime_web/), profile aura-ptt (hold = is_speech=true PCM + sticky frames; release = commit; silent+frames for vision-follow). Thin wrapper:

python -m examples.online_serving.aura_omni.realtime_web \
    --ws-backend ws://127.0.0.1:8099 --model aurateam/AURA --port 7862

Do not fork a separate aura_omni/realtime_web asset tree; MiniCPM / Qwen profiles are unchanged and hide the PTT control.

Per-stage models

Edit model on each stage in vllm_omni/deploy/aura_omni.yaml (or the smoke YAML) for local checkpoints:

• Stage 0 ASR: Qwen/Qwen3-ASR-1.7B
• Stage 1 AURA: aurateam/AURA
• Stage 2/3 TTS: Qwen/Qwen3-TTS-12Hz-1.7B-Base (or CustomVoice)

GPU utilization

Tune gpu_memory_utilization per stage. Baseline on one large GPU:

• Stage 0 (ASR): 0.10
• Stage 1 (AURA): 0.40
• Stage 2 (Talker): 0.20
• Stage 3 (Code2Wav): 0.20

Old chat / Gradio / curl scripts (unsupported)

openai_chat_completion_client.py, run_curl_multimodal_generation.sh, and run_gradio_demo.sh target the older turn-based Omni path. They do not work against the shipped session_mode: duplex profile.

That turn-based path was an incomplete June-era stand-in: vLLM-Omni did not yet support streaming I/O for this stack, and adding session history then would have required a much larger orchestrator change. It is not a supported AURA mode. Do not run those scripts against the default deploy. Prefer the duplex smoke client above.

Offline

For offline inference, see examples/offline_inference/aura_omni.