* qwen3_tts_talker_only pipeline (single terminal LLM_AR stage, final_output_type "latent") + vllm_omni/deploy/qwen3_tts_talker_only.yaml; a terminal talker publishes its per-step codes.audio rows to the client (omni_client_multimodal_output_keys). * /v1/audio/speech/stream: session.config.output_mode="codec_tokens" streams codec.start (JSON) + binary frames (<IHB seq/n_frames/flags + int16 rows, vllm_omni/entrypoints/openai/codec_stream.py) + codec.done; new input.cancel message aborts an in-flight generation for both codec and PCM sessions (single reader task + session inbox replaces direct socket reads); optional session.config.seed; audio.done gains "cancelled" only when it happened. * serving_speech._generate_codec_chunks aligns the cumulative code tensor to output token ids (drops prefill placeholder/EOS rows) and emits deltas. * TTS adapters: supports_codec_stream / codec_stream_spec (Qwen3-TTS: 16 codebooks, 2048 ids, 12.5 Hz, 24 kHz, decoder qwen3_tts_tokenizer_v2_12hz). * examples/online_serving/text_to_speech/qwen3_tts/edge_decoder_client.py decodes locally with Qwen3TTSTokenizerV2Decoder (ramp schedule, 72-frame context, cancel/reset), codec_parity_check.py and benchmarks/tts/codec_stream_e2e.py compare against the two-stage server. * docs/serving/codec_token_streaming.md; CPU unit tests for the wire format, alignment, session config and handler (frames, unsupported adapter, cancel); L3 e2e tests/e2e/online_serving/test_qwen3_tts_codec_stream.py. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
31 lines
961 B
YAML
31 lines
961 B
YAML
# Qwen3-TTS talker-only deployment: stage 0 is terminal and publishes codec
|
|
# frames (codes.audio, 16 codebooks @ 12.5 Hz) to the client instead of PCM.
|
|
# Used by the codec-token streaming mode of /v1/audio/speech/stream
|
|
# (session.config.output_mode: "codec_tokens") with decoding on the client
|
|
# device (see examples/online_serving/text_to_speech/qwen3_tts/edge_decoder_client.py).
|
|
# Stage settings mirror stage 0 of qwen3_tts.yaml.
|
|
pipeline: qwen3_tts_talker_only
|
|
async_chunk: false
|
|
stages:
|
|
- stage_id: 0
|
|
max_num_seqs: 64
|
|
gpu_memory_utilization: 0.3
|
|
trust_remote_code: true
|
|
enable_prefix_caching: false
|
|
async_scheduling: true
|
|
max_num_batched_tokens: 32768
|
|
max_model_len: 4096
|
|
devices: '0'
|
|
default_sampling_params:
|
|
temperature: 0.9
|
|
top_k: 50
|
|
max_tokens: 4096
|
|
min_tokens: 2
|
|
repetition_penalty: 1.05
|
|
silence_ban_frames: 0
|
|
subtalker_sampling_params:
|
|
do_sample: true
|
|
temperature: 0.9
|
|
top_k: 50
|
|
top_p: 1.0
|