anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,942 Commits 235 Branches 31 Tags
c4e61133fbb5587e0cf8972ce32ffac4e37fa739
Commit Graph
117 Commits
Author SHA1 Message Date
tzhouamandClaude Fable 5.1 6ce8de1b28 [Edge][CPU] KV-cached code predictor, int8-dynamic CPU fast path, model-side CPU sampler 
Measured on a Xeon 8480C (12+4 threads, 0.6B): the 15 residual-codebook sub-steps
re-ran the 5-layer predictor over the whole prefix each time (97 ms per frame of
the 134 ms talker step) and vLLM's stock sampler spent 9 ms per step in its
torch.compile'd random-sample kernel on CPU.

- code predictor: forward_cached (KV cache per layer, 2-token prefill then one
  token per sub-step) and a shape-static forward_static_step (tensor position +
  boolean mask) that torch.compile handles as one graph; incremental decoding
  is the default on CPU (VLLM_OMNI_PREDICTOR_INCREMENTAL), optional compile
  (VLLM_OMNI_PREDICTOR_COMPILE=step), optional thread cap.
- CPU fast path: dynamic int8 Linear layers (fp32 activations) for the predictor
  and its lm heads (VLLM_OMNI_PREDICTOR_INT8, default on; micro-benchmark 70 ->
  36 ms per frame for the 15 sub-steps); VLLM_OMNI_PREDICTOR_PARITY_CHECK=1
  records top-1 agreement / max logit diff against the bf16 model in situ.
- Qwen3-TTS talker: prefer_model_sampler on CPU with a sample() that mirrors
  vLLM's Sampler (logits processors + penalties, temperature, top-k/top-p,
  exponential-noise sampling with per-request generators) without the compiled
  kernel (VLLM_OMNI_CPU_FAST_SAMPLER, default on).
- step_stats: sampler phase timers and sample-metadata timers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-10 01:56:00 +08:00
tzhouamandClaude Fable 5.1 49a839ecf9 [Edge][WP5] Add codec-token streaming mode with on-device decoding 
* qwen3_tts_talker_only pipeline (single terminal LLM_AR stage,
  final_output_type "latent") + vllm_omni/deploy/qwen3_tts_talker_only.yaml;
  a terminal talker publishes its per-step codes.audio rows to the client
  (omni_client_multimodal_output_keys).
* /v1/audio/speech/stream: session.config.output_mode="codec_tokens" streams
  codec.start (JSON) + binary frames (<IHB seq/n_frames/flags + int16 rows,
  vllm_omni/entrypoints/openai/codec_stream.py) + codec.done; new input.cancel
  message aborts an in-flight generation for both codec and PCM sessions
  (single reader task + session inbox replaces direct socket reads); optional
  session.config.seed; audio.done gains "cancelled" only when it happened.
* serving_speech._generate_codec_chunks aligns the cumulative code tensor to
  output token ids (drops prefill placeholder/EOS rows) and emits deltas.
* TTS adapters: supports_codec_stream / codec_stream_spec (Qwen3-TTS: 16
  codebooks, 2048 ids, 12.5 Hz, 24 kHz, decoder qwen3_tts_tokenizer_v2_12hz).
* examples/online_serving/text_to_speech/qwen3_tts/edge_decoder_client.py
  decodes locally with Qwen3TTSTokenizerV2Decoder (ramp schedule, 72-frame
  context, cancel/reset), codec_parity_check.py and
  benchmarks/tts/codec_stream_e2e.py compare against the two-stage server.
* docs/serving/codec_token_streaming.md; CPU unit tests for the wire format,
  alignment, session config and handler (frames, unsupported adapter, cancel);
  L3 e2e tests/e2e/online_serving/test_qwen3_tts_codec_stream.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 11:10:13 +08:00
tzhouamandClaude Fable 5.1 35e43b9983 [Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend) 
* vllm_omni/platforms/cpu/platform.py: CPUOmniPlatform(OmniPlatform, CpuPlatform)
  selected when VLLM_TARGET_DEVICE=cpu or a +cpu vLLM wheel is installed
  (builtin plugin "cpu"; explicit CPU target short-circuits accelerator probes
  like vLLM does). NUMA-based memory queries, one logical device, no graph
  wrapper (sentinel type for isinstance checks), inductor opt-in via
  VLLM_OMNI_CPU_INDUCTOR=1, diffusion stages unsupported.
* vllm_omni/platforms/cpu/worker: CPUARWorker / CPUGenerationWorker compose
  vLLM's CPUWorker with the Omni mixin (XPU pattern); CPUARModelRunner /
  CPUGenerationModelRunner compose the Omni runners with CPUModelRunner
  (no graphs, no async omni output; sleep/wake return error ACKs).
* patch.py skips the CuMem patch on CPU-only torch; Qwen3-TTS speaker encoder
  and prompt-embeds dtype follow the model dtype (float32/float16 on CPU).
* vllm_omni/deploy/qwen3_tts.yaml: `platforms: cpu:` overlay (eager, no async
  scheduling, absolute kv_cache_memory_bytes, disjoint OMP thread ranges).
* requirements/cpu.txt (+psutil, intel-openmp), docs/getting_started/installation/cpu.md.

Measured (this x86 Xeon 8480C host, vllm 0.28.0+cpu, bf16, 12+4 threads):
Qwen3-TTS 0.6B CustomVoice end-to-end on CPU, 12 prompts: TTFA p50 292 ms,
RTF ~1.7 (25-frame chunk ~4.6 s), init 185 s. Result JSON + init timeline
committed under benchmarks/tts/edge_results/.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 11:02:14 +08:00
tzhouamandClaude Fable 5.1 073d31b9cd [Edge][WP2] Add initialization timeline and cold-start profiler 
* vllm_omni/engine/init_timeline.py: InitTimeline phase collector; engine
  phases (stages_init, g3_device_lock, g2_spawn/ready) and worker phases
  (proc_entry, engine_core_init, init_device, load_model/load_weights,
  talker_mtp_init, attn_prewarm, cudagraph_capture, talker_mtp_capture,
  code2wav_cudagraph, predictor_setup_compile/warmup/cudagraph) recorded via
  worker_phase/timed_phase; workers append JSONL under
  VLLM_OMNI_INIT_TIMELINE_DIR (inherited at spawn) and the engine merges them.
  Enabled by VLLM_OMNI_INIT_TIMELINE=<json> or log_stats; table logged at INFO,
  partial table on orchestrator startup timeout, suggested timeouts logged.
* omni_base.py: named deploy profiles default to stage_init_timeout=900 /
  init_timeout=1500 unless given explicitly.
* gpu_model_runner._init_talker_mtp tolerates max_cudagraph_capture_size=None;
  gpu_ar_model_runner.shutdown skips CUDA graph teardown on CPU.
* benchmarks/tts/init_profile.py: runs stream_latency_bench --init-only per
  configuration (default, cold_cache, eager_stage1, eager_both,
  parallel_stage_init, edge, ...) and tabulates phase durations.
* stream_latency_bench: --parallel-stage-init / --deploy-profile flags; the
  resource poller no longer shadows Thread._stop and skips GPU sampling on CPU.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 11:01:39 +08:00
GXINandHongsheng Liu 3ab582c993 [Bugfix][Qwen3-TTS] Reject task mismatches before EngineCore dispatch (#6113) 
Signed-off-by: XIN GAO <1037396230@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-02 11:16:55 -07:00
wy17003andHongsheng Liu b454e34966 [Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. (#6149) 
Signed-off-by: wy17003 <yaow17003@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-25 15:24:59 +08:00
boatmanandHongsheng Liu 3781214675 [Refactor] Move model-specific capability metadata into TTS adapters (#6138) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 14:26:49 +08:00
林鑫sheenlinyurainGao Han
 		37c11e6e4f [Perf][Qwen3-TTS] Fuse QKV and gate_up projections in code predictor (#5791) 
Signed-off-by: sheenlin <sheenlin@tencent.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: sheenlin <sheenlin@tencent.com>
Co-authored-by: yurain <yurain26@users.noreply.github.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-20 19:22:40 +00:00
Inesh Reddy Chappidi cd3867c373 [Model] Suppress silence codec tokens for the first N Qwen3-TTS decod… (#5048) 
Signed-off-by: Inesh Reddy <ineshreddy249@gmail.com>
 2026-08-20 09:22:37 -07:00
boatmanandYueqian Lin 4f9b65074d [Perf][Qwen3-TTS]Cached Incremental Decode (#5202) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-08-11 05:24:04 +00:00
Gao Han 78c144f3a8 [CI failed] Revert "[Perf][Qwen3-TTS] Fuse QKV and gate_up projections in code predictor" (#5777) 2026-08-04 15:21:38 +00:00
林鑫sheenlinyurain
 		6fdc4ca62d [Perf][Qwen3-TTS] Fuse QKV and gate_up projections in code predictor (#4958) 
Signed-off-by: sheenlin <sheenlin@tencent.com>
Co-authored-by: sheenlin <sheenlin@tencent.com>
Co-authored-by: yurain <yurain26@users.noreply.github.com>
 2026-08-04 11:59:32 +00:00
Ma1oneZhangSy03Hongsheng Liu
 		af2f84582b perf(qwen3-tts): cache decoder masks and compile pre-transformer (#5107) 
Signed-off-by: Ziyang Zhang <hafuhafu@qq.com>
Co-authored-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-25 08:09:19 +00:00
Yukim1andHongsheng Liu 793d3e9ab7 [Config] Return VllmOmniConfig from create_from_model (#4818) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-18 09:21:07 +00:00
amy-why-3459 5dc787d5e8 [Perf][Qwen3-TTS] Align talker MTP CUDA graph capture with Qwen3-Omni & Adapt to async output (#4923) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-07-10 11:11:29 +08:00
Yueqian Lin 6d50348ef1 [Perf][Qwen3-TTS] Skip per-step hidden-state D2H via hidden pooler payload opt-out (#4879) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-07-08 11:59:41 -07:00
Yueqian Lin d7b365df64 [Bugfix][Qwen3-TTS] Keep seeded residual MTP sampling batched (#4889) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-07-05 21:47:06 +08:00
Sy03Claude Sonnet 4.6Yueqian Lin
 		f11b32a42e [Bugfix] Qwen3-TTS talker handles multi-token replay on resume (#4559) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-07-02 04:46:05 +00:00
rein yang afbcc51672 [Model] Aura support - Non async chunk path (#4257) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
 2026-06-21 22:08:26 +08:00
Yueqian LinSy03Hongsheng Liu
 		cead615288 [Bugfix][Qwen3-Omni] Fix Code2Wav CUDA-graph output length surplus (#4466) (#4525) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-19 23:58:45 -07:00
Yueqian Lin 208932210f [Chore][TTS] Reference recipes.vllm.ai recipes, remove dead code, and fix stale references (#4567) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-06-19 15:05:26 +08:00
Zhou Taichang 7b837944a1 [Rebase] Rebase to vllm 0.23.0 (#4286) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-06-13 16:43:02 +00:00
Henry JjjwClaude Fable 5Yueqian Lin
 		1733831f78 [Bugfix] Fix cross-request codes.ref leak in Qwen3-TTS make_omni_output (#4373) 
Signed-off-by: jjw <m@jjw.in>
Co-authored-by: jjw <m@jjw.in>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-06-13 00:41:49 +00:00
zyz111222andCanlin Guo 6de0ca36bb [Hardware][Ascend] Adapt Qwen3 TTS for 310P (#4283) 
Signed-off-by: zouyizhou <zouyizhou@huawei.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-06-12 09:39:21 +00:00
Juan Pablo ZuluagaJuanPZuluagaGao Han
 		57227dc7f3 [Perf][Bugfix] qwen3-tts hot path: prefix-cache OOM guards + talker/orchestrator micro-opts (#3689) 
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com>
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-06-09 10:36:57 -07:00
Sy03Dan250124Yueqian LinYueqian Lin
 		d4c139508f [Feature] Add precomputed custom voices and Qwen3-TTS ref-context cache (#3492) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Dan250124 <416947747@qq.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-05-31 17:51:24 +00:00
ChenWenjing 5dfdf584dd [Feat]audio streaming input for async chunk (#3614) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-05-31 17:39:25 +08:00
Viacheslav KlimkovandYueqian Lin 914e92cf80 [Refactor][Qwen3-TTS] Extract reusable prompt-embeds builder and make tts_pad_embed a persistent buffer (#3992) 
Signed-off-by: Viacheslav Klimkov <vklimkov@nvidia.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-05-30 16:11:00 +08:00
nperraud 9fdb3bcb89 [Bugfix][Model] Qwen3-TTS: don't collapse 2D ref_code list when estimating prompt length (#3940) 
Signed-off-by: nperraud <nathanael.perraudin@gmail.com>
 2026-05-29 09:18:25 -06:00
Nick CaoandClaude c20afebce4 [Refactor][Qwen3-TTS] Construct speech tokenizer encoder natively (#3360) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-28 16:20:49 +08:00
NATURE 3a347b10b0 [3/5][core]refactor communication layer: PR 3 of 5, all other models in non async mode (#3719) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-05-28 14:06:41 +08:00
Juan Pablo ZuluagaJuanPZuluagaYueqian Lin
 		0f3264a9c0 [Perf][Bugfix] cache hot buffers in qwen3_tts talker; fall back on evicted state (#3688) 
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com>
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-05-28 05:45:54 +00:00
BeatSeat dab596e468 [Refactor] Unify Snake/SnakeBeta and alias-free activation into common modules (#3886) 
Signed-off-by: BeatSeat <wendavid552@gmail.com>
 2026-05-27 23:41:35 -05:00
Dan 1f1c82ecdd [BugFix] Fix Qwen3-TTS Code2Wav compatibility with transformers >= 5.9.0 (#3880) 
Signed-off-by: Dan250124 <416947747@qq.com>
 2026-05-26 16:24:43 +00:00
Yueqian LinandRein Yang e996723bc9 [Bugfix] Fix Qwen3-TTS Stage 0 prefix-caching correctness (#3665) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>

the ci failure case is irrelavent to this PR and occurs rarely
 2026-05-25 23:09:38 +08:00
amy-why-3459andHongsheng Liu 748470c1fb [BugFix] code2wav supports disabling CUDA graph. (#3732) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-24 00:05:55 +08:00
Sy03andamy-why-3459 ab59673f21 [Bugfix][Qwen3-Omni] Handle short Code2Wav chunk outputs (#3687) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-05-18 18:59:23 +00:00
Sy03Hongsheng LiuSYLAR
 		c99df1ebd9 [TTS][Perf] Optimize Qwen3-TTS high-concurrency serving (#3662) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-05-17 16:58:59 -04:00
+20
Zhou TaichangClaude Opus 4.7NumberWandsinghviDnoobYueqian LinHongsheng Liuknlnguyen1802Samitrein yangNick Caozhumingjue138Ricardo Noriegalyj-jjjCursorgcanlinwangyuweizhoublueweizhoubluedengyunyang汪志鹏WeiQing ChenJie LiuYueqian LinNATUREbjf-frzamy-why-3459Y. FisherZeyu Huang | 黃澤宇Copilot Autofix powered by AI
 		eb4e60ee64 [Rebase] Rebase to vllm v0.21.0 (#3530) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Dnoob <dxpouo@gmail.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: rein yang <ruiruyang2@gmail.com>
Signed-off-by: Nick Cao <ncao@redhat.com>
Signed-off-by: zhumingjue <zhumingjue@huawei.com>
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com>
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: weizhoublue <weizhoublue@github.com>
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io>
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: NumberWan <wantszkin2003@gmail.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Dnoob <dxpouo@gmail.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com>
Co-authored-by: Ricardo Noriega <rnoriega@redhat.com>
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: weizhoublue <45163302+weizhoublue@users.noreply.github.com>
Co-authored-by: weizhoublue <weizhoublue@github.com>
Co-authored-by: dengyunyang <584797741@qq.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: NATURE <wzliu@connect.hku.hk>
Co-authored-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Y. Fisher <yukexiong1@huawei.com>
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-05-16 18:18:43 +08:00
dsinghvi e40076e872 [Refactor] msgspec standardisation for data entry key names and improved type checks (#3149) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
 2026-05-12 14:30:13 +08:00
Sy03 2c73cf3522 [Perf] Fix Qwen3-TTS latency regression (#3485) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-05-10 23:07:38 +00:00
Nick Cao e6466cf0e6 [Refactor] Replace and ban a few torch.cuda functions in favor of torch.accelerator replacements. (#3365) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-05-07 23:23:59 +08:00
Juan Pablo ZuluagaandYueqian Lin 19f8f42822 [Feat][Qwen3-Omni] Add CUDA graph support for Code2Wav decoder (#2376) 
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-05-07 06:16:03 +08:00
Sy03 2b85474c23 [Bugfix] Propagate seed to Qwen3-TTS Fast AR sampler (#3350) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-05-05 14:57:55 -04:00
Nick CaoandClaude fde38ca54f [Refactor][Qwen3-TTS] Construct Code2Wav decoder natively (#2341) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-05 15:26:30 +00:00
Juan Pablo ZuluagaandJuanPZuluaga b64ab05d09 [TTS][SpeakerCacheManager] A global speaker cache manager for Voice Cloning (#2630) 
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com>
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com>
 2026-05-05 10:53:45 -04:00
Canlin Guo c007d40b14 [NPU] Upgrade to v0.20.0 & align with GPU model runner (#3325) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-05-04 22:38:43 +08:00
Nick CaoandClaude 9de3806624 [Cleanup] Extract shared SinusPositionEmbedding/DiTTimestepEmbedding/timestep_embedding layer (#3285) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-01 20:19:42 +08:00
Zhou TaichangandYuanheng Zhao 7f2b1e9386 [Rebase] Rebase to vllm 0.20.0 (#3232) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com>
 2026-04-30 09:56:59 +12:00
Guanzhe (Leo) Huang d34093b360 [BugFix] qwen3_tts: build speaker_encoder in __init__ so load_format: dummy works (#3117) 
Signed-off-by: leohuang257 <masugchds@gmail.com>
 2026-04-27 13:56:15 +08:00