Add speech token usage headers (#4499)

Signed-off-by: JackyLiu <lzwgre@126.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
JackyLiuandHongsheng Liu authored and GitHub committed 2026-08-18 06:53:06 +08:00
1 parent 2fdbf2234a
commit d7216ef514
3 files changed
+86 -5

No files matched your search

+14 -1
View File
@@ -141,7 +141,20 @@ The response shape depends on the streaming parameters:
**Non-streaming (default).** With `stream=false` and no `stream_format`, returns the
complete clip as binary audio data with an appropriate `Content-Type` header (e.g.
`audio/wav`). The raw-bytes body has no JSON carrier, so no `usage` is reported.
`audio/wav`). Because the raw-bytes body has no JSON carrier, successful
non-streaming responses from non-diffusion speech servers report usage through
response headers:
| Header | Description |
| --- | --- |
| `x-vllm-omni-input-tokens` | Total input tokens (`text_tokens` + `audio_tokens`). |
| `x-vllm-omni-output-tokens` | Generated codec/audio tokens. |
| `x-vllm-omni-total-tokens` | `input_tokens` + `output_tokens`. |
| `x-vllm-omni-input-text-tokens` | Tokens from the synthesized text (`input` plus `instructions`). |
| `x-vllm-omni-input-audio-tokens` | Reference-audio codec frames, non-zero only for in-context voice cloning. |
Diffusion-mode speech servers route through a separate response path and do not
emit these headers.
**Raw audio stream** (`stream_format="audio"`). Streams raw audio bytes (PCM or
WAV) as they are decoded.