`async_chunk` is enabled. The feature path partitions each step's payload into
streaming inter-stage and client handoffs instead.
### Output Builder
The background builder performs the work that previously ran inline:
- resolves which requests need downstream payloads
- converts hidden and multimodal tensors into per-request CPU payloads
- applies model-specific payload processing
- partitions per-step payloads into inter-stage and client streams
- creates the tensor-only `multimodal_outputs` wire payload
- drains live connector readiness signals after building the streaming payload
- constructs the final `OmniModelRunnerOutput`
Input-side connector operations remain synchronous with model execution. In
particular, receiving inputs and flushing previously completed connector
outputs are not deferred because they affect scheduler-visible request state.
### Qwen3-Omni Stage Behavior
| Stage | Async output behavior | Reason |
|---|---|---|
| Thinker | Snapshots hidden states and multimodal outputs; builds the downstream payload in the background | The Talker needs the payload, but the next Thinker decode step only needs sampled-token feedback |
| Talker | Runs lightweight postprocess eagerly; snapshots codec outputs; omits hidden states from the downstream payload | `hidden_states.last` is needed by the next Talker step, while Code2Wav only needs codec codes |
| Code2Wav | Uses the normal generation-stage output path | Code2Wav is not executed by `GPUARModelRunner` |
The Qwen3-TTS Talker follows the same pattern as the Qwen3-Omni Talker:
postprocess runs eagerly because the next decode step needs its updated state,
while codec payload construction runs asynchronously. Its Code2Wav stage also
uses the normal generation-stage path.
## Enabling the Feature
There is no separate `async_omni_output` command-line or YAML option. The
feature is selected automatically for model stages that opt in when the runtime
conditions are safe. On the validated CUDA and ROCm configurations, using the
bundled deployment profile is enough to turn it on for the supported
Qwen3-Omni and Qwen3-TTS pipelines. The selection logic has no CUDA/ROCm-only
guard; see [Compatibility and Fallbacks](#compatibility-and-fallbacks) for the
other platform paths. The same profile does not enable background Omni
materialization on Ascend NPU.
### Example 1: Qwen3-Omni
Start the full Qwen3-Omni pipeline:
```bash
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct \
--omni \
--port 8091
```
The model registry automatically loads
`vllm_omni/deploy/qwen3_omni_moe.yaml`. Its relevant settings are:
```yaml
async_chunk:true
stages:
- stage_id:0# Thinker
enable_prefix_caching:false
# async_scheduling defaults to true for AR stages
- stage_id:1# Talker
enable_prefix_caching:false
# async_scheduling defaults to true for AR stages
- stage_id:2# Code2Wav
enable_prefix_caching:false
async_scheduling:false
```
With this profile, async output materialization activates automatically for:
- **Stage 0, Thinker**: Defers hidden-state and multimodal payload
construction.
- **Stage 1, Talker**: Defers codec payload construction after eagerly
updating its decode state.
Stage 2 does not use this feature because Code2Wav is not an AR stage.
### Example 2: Qwen3-TTS
Start Qwen3-TTS with any checkpoint that uses the Qwen3-TTS Talker pipeline.
For example:
```bash
vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
--omni \
--port 8091
```
The model registry automatically loads `vllm_omni/deploy/qwen3_tts.yaml`. Its
relevant settings are:
```yaml
async_chunk:true
stages:
- stage_id:0# Talker
async_scheduling:true
enable_prefix_caching:false
- stage_id:1# Code2Wav
enable_prefix_caching:false
```
Async output materialization activates only for Stage 0, the AR Talker. The
Talker keeps the state required by its next decode step on the GPU, skips an
unnecessary hidden-state D2H copy, and builds the codec payload for Code2Wav in
the background. Stage 1 uses the generation-stage output path.
The same behavior applies to the Base and VoiceDesign Qwen3-TTS checkpoints
| Postprocess is absent or explicitly runs eagerly | State needed by the next decode step must be updated before the runner returns. Eager postprocess sees the scheduled slice (`None, None` combined tensors), not the prefix-cache-merged full prompt; it must only consume the tail |
