[Edge] Docs: chunk ramp / edge profile section and an edge recipe for Qwen3-TTS
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
580a201631
commit
fa8b1a878e
2 files changed
+112
No files matched your search
@@ -246,6 +246,33 @@ stages:
|
||||
from_stage_0: connector_of_shared_memory
|
||||
```
|
||||
|
||||
### Chunk Ramp and the Edge Profile
|
||||
|
||||
The default schedule emits one 80 ms frame first (`initial_codec_chunk_frames: 1`)
|
||||
and the next 25-frame chunk about 160 ms later, so a player that starts at the
|
||||
first audio stalls for roughly 80 ms (measured on L20X: playback-start latency
|
||||
116 ms vs TTFA 32 ms for the 1.7B model). `codec_chunk_ramp` replaces the
|
||||
initial/steady pair with a growing schedule so the audio buffered ahead always
|
||||
covers the wait for the next chunk:
|
||||
|
||||
```yaml
|
||||
connectors:
|
||||
connector_of_shared_memory:
|
||||
name: SharedMemoryConnector
|
||||
extra:
|
||||
codec_chunk_frames: 25
|
||||
codec_chunk_ramp: [2, 4, 8, 16, 25] # last entry == codec_chunk_frames
|
||||
decode_batch_max_size: 1 # edge: 1-4 concurrent utterances
|
||||
```
|
||||
|
||||
`vllm_omni/deploy/edge/qwen3_tts.yaml` bundles this ramp with single-device,
|
||||
small-batch stage settings and concurrent stage init; select it with
|
||||
`Omni(model=..., deploy_profile="edge")` or `vllm serve --omni --deploy-profile edge`
|
||||
(`deploy_profile="auto"` additionally probes the hardware, see
|
||||
`vllm_omni/edge/`). With the ramp the measured playback-start latency equals
|
||||
TTFA (stall 0 ms, TTFA 41-44 ms) at unchanged RTF; results live under
|
||||
`benchmarks/tts/edge_results/`.
|
||||
|
||||
### Code2Wav Batch Configuration
|
||||
|
||||
For optimal performance with async_chunk, the code2wav stage should be configured with batching:
|
||||
|
||||
Reference in new issue
Block a user