tzhouamandClaude Fable 5.1
49a839ecf9
[Edge][WP5] Add codec-token streaming mode with on-device decoding
* qwen3_tts_talker_only pipeline (single terminal LLM_AR stage,
final_output_type "latent") + vllm_omni/deploy/qwen3_tts_talker_only.yaml;
a terminal talker publishes its per-step codes.audio rows to the client
(omni_client_multimodal_output_keys).
* /v1/audio/speech/stream: session.config.output_mode="codec_tokens" streams
codec.start (JSON) + binary frames (<IHB seq/n_frames/flags + int16 rows,
vllm_omni/entrypoints/openai/codec_stream.py) + codec.done; new input.cancel
message aborts an in-flight generation for both codec and PCM sessions
(single reader task + session inbox replaces direct socket reads); optional
session.config.seed; audio.done gains "cancelled" only when it happened.
* serving_speech._generate_codec_chunks aligns the cumulative code tensor to
output token ids (drops prefill placeholder/EOS rows) and emits deltas.
* TTS adapters: supports_codec_stream / codec_stream_spec (Qwen3-TTS: 16
codebooks, 2048 ids, 12.5 Hz, 24 kHz, decoder qwen3_tts_tokenizer_v2_12hz).
* examples/online_serving/text_to_speech/qwen3_tts/edge_decoder_client.py
decodes locally with Qwen3TTSTokenizerV2Decoder (ramp schedule, 72-frame
context, cancel/reset), codec_parity_check.py and
benchmarks/tts/codec_stream_e2e.py compare against the two-stage server.
* docs/serving/codec_token_streaming.md; CPU unit tests for the wire format,
alignment, session config and handler (frames, unsupported adapter, cancel);
L3 e2e tests/e2e/online_serving/test_qwen3_tts_codec_stream.py.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-09 11:10:13 +08:00
..
2026-09-09 11:01:40 +08:00
2026-07-28 03:58:22 +00:00
2026-07-28 03:58:22 +00:00
2026-07-28 03:58:22 +00:00
2026-07-28 03:58:22 +00:00
2026-07-28 03:58:22 +00:00
2026-07-28 03:58:22 +00:00
2026-08-03 23:41:52 +08:00
2026-07-28 03:58:22 +00:00
2026-07-22 09:44:15 +08:00
2026-08-20 07:09:12 +08:00
2026-04-24 21:21:30 +08:00
2026-07-22 09:44:15 +08:00
2026-07-29 02:46:27 +00:00
2026-05-12 14:39:58 +08:00
2026-08-13 17:58:33 +00:00
2026-07-28 17:14:26 +08:00
2026-07-05 07:17:58 +08:00
2026-06-02 21:28:12 +00:00
2026-07-22 09:44:15 +08:00
2026-06-02 21:28:12 +00:00
2026-08-20 07:09:12 +08:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-08-14 22:06:05 -07:00
2026-05-27 12:18:47 +12:00
2026-05-24 05:42:26 +08:00
2026-06-20 11:45:26 +08:00
2026-07-22 09:44:15 +08:00
2026-08-13 10:00:50 -07:00
2026-08-13 10:00:50 -07:00
2026-08-30 08:52:15 +00:00
2026-05-11 22:49:19 +08:00
2026-07-24 14:45:26 +00:00
2026-07-22 09:44:15 +08:00
2026-07-18 09:34:33 +00:00
2026-08-11 23:00:35 +00:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-06-01 16:01:25 +08:00
2026-09-07 13:38:23 +08:00
2026-08-28 08:32:32 +08:00
2026-08-28 08:32:32 +08:00
2026-08-26 16:26:24 +08:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-05-03 14:43:49 +12:00
2026-05-03 14:43:49 +12:00
2026-05-03 14:43:49 +12:00
2026-07-12 19:50:51 -07:00
2026-07-13 06:45:28 +08:00
2026-08-29 23:27:47 +08:00
2026-08-26 10:56:51 +08:00
2026-08-29 23:27:47 +08:00
2026-08-26 10:56:51 +08:00
2026-08-26 10:56:51 +08:00
2026-08-26 10:56:51 +08:00
2026-09-06 22:25:35 +08:00
2026-08-28 14:27:05 +08:00
2026-08-28 14:27:05 +08:00
2026-08-18 04:33:13 +00:00
2026-08-18 04:33:13 +00:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-08-13 08:56:26 +00:00
2026-08-20 07:09:12 +08:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-09-03 18:49:19 -07:00
2026-09-03 18:49:19 -07:00
2026-09-03 18:49:19 -07:00
2026-07-08 00:44:33 +00:00
2026-08-04 20:10:33 -07:00
2026-08-17 14:29:17 +08:00
2026-08-11 23:00:35 +00:00
2026-06-10 08:37:33 +08:00
2026-08-27 08:21:03 +00:00
2026-09-04 21:16:58 +08:00
2026-08-12 05:33:34 +00:00
2026-08-28 02:11:05 -07:00
2026-09-09 11:10:13 +08:00
2026-09-09 11:02:14 +08:00
2026-08-13 10:00:50 -07:00
2026-07-22 09:44:27 +00:00
2026-07-18 09:34:33 +00:00
2026-07-18 09:34:33 +00:00
2026-07-18 09:34:33 +00:00
2026-07-04 17:47:11 +00:00
2026-07-28 07:35:35 +08:00
2026-09-01 01:02:54 +00:00