tzhouamandClaude Fable 5.1 49a839ecf9 [Edge][WP5] Add codec-token streaming mode with on-device decoding ...

* qwen3_tts_talker_only pipeline (single terminal LLM_AR stage,
  final_output_type "latent") + vllm_omni/deploy/qwen3_tts_talker_only.yaml;
  a terminal talker publishes its per-step codes.audio rows to the client
  (omni_client_multimodal_output_keys).
* /v1/audio/speech/stream: session.config.output_mode="codec_tokens" streams
  codec.start (JSON) + binary frames (<IHB seq/n_frames/flags + int16 rows,
  vllm_omni/entrypoints/openai/codec_stream.py) + codec.done; new input.cancel
  message aborts an in-flight generation for both codec and PCM sessions
  (single reader task + session inbox replaces direct socket reads); optional
  session.config.seed; audio.done gains "cancelled" only when it happened.
* serving_speech._generate_codec_chunks aligns the cumulative code tensor to
  output token ids (drops prefill placeholder/EOS rows) and emits deltas.
* TTS adapters: supports_codec_stream / codec_stream_spec (Qwen3-TTS: 16
  codebooks, 2048 ids, 12.5 Hz, 24 kHz, decoder qwen3_tts_tokenizer_v2_12hz).
* examples/online_serving/text_to_speech/qwen3_tts/edge_decoder_client.py
  decodes locally with Qwen3TTSTokenizerV2Decoder (ramp schedule, 72-frame
  context, cancel/reset), codec_parity_check.py and
  benchmarks/tts/codec_stream_e2e.py compare against the two-stage server.
* docs/serving/codec_token_streaming.md; CPU unit tests for the wire format,
  alignment, session config and handler (frames, unsupported adapter, cancel);
  L3 e2e tests/e2e/online_serving/test_qwen3_tts_codec_stream.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-09-09 11:10:13 +08:00

..

protocol

[Edge][WP5] Add codec-token streaming mode with on-device decoding

2026-09-09 11:10:13 +08:00

tts_adapters

[Edge][WP5] Add codec-token streaming mode with on-device decoding

2026-09-09 11:10:13 +08:00

__init__.py

[Core] Supports stage abstraction in the diffusion model (#391)

2025-12-27 01:59:33 +13:00

api_server.py

[Frontend][Model] Add Cosmos3 control uploads to video API (#7027)

2026-09-07 01:38:56 -07:00

audio_utils_mixin.py

[Frontend] Add configurable output sample rate to Speech API (#6553)

2026-09-01 17:25:09 +08:00

batch_serving.py

[Frontend] Implement Batched Chat Completions (#5317)

2026-08-07 16:12:09 -06:00

codec_stream.py

[Edge][WP5] Add codec-token streaming mode with on-device decoding

2026-09-09 11:10:13 +08:00

diffusion_request_utils.py

[Model][Feat] Support Minimax-H3 quality grading requests through dynamic loading/unloading with Cache-DiT (#5853)

2026-08-07 09:07:23 +00:00

errors.py

[Frontend] Rewrite video API for async job lifecycle (#1665)

2026-03-13 17:11:06 +08:00

image_api_utils.py

[Bugfix] Match image file metadata to output format (#5619)

2026-08-01 15:09:27 +00:00

realtime_connection.py

[Bugfix][Realtime] Stop disconnected sessions from cycling through stages (#5388)

2026-07-27 17:04:17 +00:00

serving_audio_generate.py

[BugFix][TTS] Move user input from INFO to DEBUG in TTS/audio log lines (#6329)

2026-08-18 16:22:11 -06:00

serving_chat.py

[Bugfix] Use Seed-TTS reference audio with MiniCPM-o 4.5 (#6628)

2026-08-31 10:27:34 +08:00

serving_speech_stream.py

[Edge][WP5] Add codec-token streaming mode with on-device decoding

2026-09-09 11:10:13 +08:00

serving_speech.py

[Edge][WP5] Add codec-token streaming mode with on-device decoding

2026-09-09 11:10:13 +08:00

serving_video_output_stream.py

[Feature] Support text encoder Disaggregation for H3 (#5885)

2026-08-28 14:27:05 +08:00

serving_video_stream.py

Refactor: extract OmniStreamingVideoHandler base and QwenOmniStreamingVideoHandler (#4424)

2026-06-17 15:07:45 +00:00

serving_video.py

[Frontend][Model] Add Cosmos3 control uploads to video API (#7027)

2026-09-07 01:38:56 -07:00

speech_usage.py

[Bugfix][Qwen3-TTS] Handle codec generations that exhaust their token budget (#6728)

2026-08-29 08:59:05 -07:00

stage_params.py

[Refactor] Centralize stage sampling params resolution (#3153)

2026-04-26 23:30:11 +08:00

storage.py

[Frontend] Configurable video storage backends with TTL (#2531)

2026-06-19 17:41:22 -06:00

stores.py

[Frontend] Rewrite video API for async job lifecycle (#1665)

2026-03-13 17:11:06 +08:00

utils.py

[Feature] Support text encoder Disaggregation for H3 (#5885)

2026-08-28 14:27:05 +08:00

video_api_utils.py

[Model] Add native MAGI-2 Preview diffusion support (#5918)

2026-09-05 15:32:30 +08:00

video_frame_filter.py

[Feature] Streaming video input with EVS frame filtering (RFC #2201 Phase 2-4) (#2342)

2026-04-24 22:54:35 +08:00

video_stream_base.py

[Refactor][OutputProcessor 2/8]: OmniRequestOutput should inherit RequestOutput and no nested wrap-up (#5146)

2026-08-12 22:58:23 +08:00

video_stream_context.py

[Feature] Streaming video input with EVS frame filtering (RFC #2201 Phase 2-4) (#2342)

2026-04-24 22:54:35 +08:00

video_stream_envs.py

[Feature] Streaming video input with EVS frame filtering (RFC #2201 Phase 2-4) (#2342)

2026-04-24 22:54:35 +08:00

video_stream_session.py

[Feature] Streaming video input with EVS frame filtering (RFC #2201 Phase 2-4) (#2342)

2026-04-24 22:54:35 +08:00