tzhouam and Claude Fable 5.1
073d31b9cd
[Edge][WP2] Add initialization timeline and cold-start profiler
...
* vllm_omni/engine/init_timeline.py: InitTimeline phase collector; engine
phases (stages_init, g3_device_lock, g2_spawn/ready) and worker phases
(proc_entry, engine_core_init, init_device, load_model/load_weights,
talker_mtp_init, attn_prewarm, cudagraph_capture, talker_mtp_capture,
code2wav_cudagraph, predictor_setup_compile/warmup/cudagraph) recorded via
worker_phase/timed_phase; workers append JSONL under
VLLM_OMNI_INIT_TIMELINE_DIR (inherited at spawn) and the engine merges them.
Enabled by VLLM_OMNI_INIT_TIMELINE=<json> or log_stats; table logged at INFO,
partial table on orchestrator startup timeout, suggested timeouts logged.
* omni_base.py: named deploy profiles default to stage_init_timeout=900 /
init_timeout=1500 unless given explicitly.
* gpu_model_runner._init_talker_mtp tolerates max_cudagraph_capture_size=None;
gpu_ar_model_runner.shutdown skips CUDA graph teardown on CPU.
* benchmarks/tts/init_profile.py: runs stream_latency_bench --init-only per
configuration (default, cold_cache, eager_stage1, eager_both,
parallel_stage_init, edge, ...) and tabulates phase durations.
* stream_latency_bench: --parallel-stage-init / --deploy-profile flags; the
resource poller no longer shadows Thread._stop and skips GPU sampling on CPU.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 11:01:39 +08:00
tzhouam and Claude Fable 5.1
f804bbd679
[Edge][WP0] Record Qwen3-TTS streaming baselines on 1x L20X
...
Default deploy config (1 -> 25 frame schedule), 12 prompts x 3 repeats,
warm-up excluded, one GPU via `gpu run`:
1.7B CustomVoice: TTFA p50 32.0 ms, playback-start p50 116 ms (stall at
TTFA 85 ms), RTF_total 0.086, chunk gap p50 163 ms, init 271 s (cold caches)
0.6B CustomVoice: TTFA p50 27.8 ms, playback-start p50 95 ms (stall 67 ms),
RTF_total 0.077, chunk gap p50 146 ms, init 152 s
Per-request chunk arrivals, memory and the full summaries are in the JSON
(force-added: *.json is gitignored repo-wide).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 10:49:37 +08:00
tzhouam and Claude Fable 5.1
b70207f57f
[Edge][WP0] Add offline streaming-latency benchmark for chunked TTS
...
Adds benchmarks/tts/stream_latency_bench.py (AsyncOmni driver reusing the
Qwen3-TTS example's prompt builder; --model override, warm-up/repeat,
--init-only, per-chunk arrivals, peak RSS and per-process GPU memory) and
benchmarks/tts/stream_latency_metrics.py (TTFA, underrun-free playback start,
stall at TTFA, RTF streamed/total, chunk gaps, percentiles) with CPU unit
tests. Results go under benchmarks/tts/edge_results/stream_latency/.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 10:24:53 +08:00
Weiming Liao and Gao Han
1cd92553cf
[CI][Bugfix] Use the matching checkpoint for Qwen3-TTS VoiceDesign tests ( #7054 )
...
Signed-off-by: Weiming Liao <liaowm5@gmail.com >
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-09-06 04:27:21 +00:00
DanaerLee
ca6b4e2112
[Perf][Bugfix][OmniVoice] Restore float16 serving, and fuse the generator hot loop ( #6317 )
...
Signed-off-by: MrlixiangWE <mrdanaer@gmail.com >
2026-09-02 19:54:55 +00:00
f90e992dc1
[Model] Optimize CosyVoice3 TensorRT stream handoff ( #5673 )
...
Signed-off-by: Allen Wu <allenwu2795@gmail.com >
Co-authored-by: TRAE CLI <noreply@bytedance.com >
Co-authored-by: Sy03 <1370724210@qq.com >
Co-authored-by: TRAE CLI <traecli@bytedance.com >
2026-08-25 22:51:46 -07:00
Yukim1
6f55a06fab
[Docs] Remove stale stage config references ( #6270 )
...
Signed-off-by: zwhzzz0821 <2831474076@qq.com >
2026-08-20 07:09:12 +08:00
Sy03 and Yueqian Lin
58007c0084
[New Model] Support IndexTTS 2.5 ( #5957 )
...
Signed-off-by: Sysy <1370724210@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-08-11 23:00:35 +00:00
Canlin Guo
2e6c83c89c
[Model] Support MOSS-TTS-local-v1.5 ( #4664 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-06-25 15:43:03 +08:00
e27fd46ab9
[Perf] Optimize Higgs Audio v3 serving ( #4204 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-11 13:29:31 -07:00
Zhang Jian and Canlin Guo
b550709bba
[TTS][Model] support MOSS-TTS series ( #3420 )
...
Signed-off-by: Zhang <jianmusings@gmail.com >
Signed-off-by: Zhang Jian <jianmusings@gmail.com >
Signed-off-by: jian <jianmusings@gmail.com >
Signed-off-by: Canlin Guo <961750412@qq.com >
Co-authored-by: Canlin Guo <961750412@qq.com >
2026-06-02 15:46:01 +08:00
+1
f92d84fcf9
[Rebase] Rebase to vllm releases/v0.22.0 ( #3891 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: david6666666 <530634352@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: nainiu258 <cperfect02@163.com >
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com >
Co-authored-by: Zheng Wengang <zwg0606@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-06-01 11:08:12 +08:00
ChenWenjing and Gao Han
834abdaf8b
[ci][nightly] Voxcpm2 performance benchmark ( #3864 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-05-29 09:44:29 +08:00
Yueqian Lin
fc8486cf20
[Doc][TTS] CosyVoice3 online docs + residual TTS yaml cleanup + remove VoxCPM v1 ( #3748 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-05-20 15:24:06 +08:00
Yueqian Lin and Hongsheng Liu
f99e8b2d3c
[Bench] Add audio-streaming continuity metric for TTS ( #3618 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-20 05:08:56 +00:00
wangyu and Huang, Zeyu
6f2ad7b403
[Test] Unify L2/L3 test layout, Buildkite steps, and test helpers ( #2556 )
...
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
2026-05-08 14:12:23 +08:00
Yueqian Lin
39e3fd3d12
[Benchmark] Universal TTS benchmark: Qwen3-TTS + VoxCPM2 with 3 task types (voice-clone/default/design) ( #2835 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-04-24 23:35:33 +08:00
