tzhouam and Claude Fable 5.1
f804bbd679
[Edge][WP0] Record Qwen3-TTS streaming baselines on 1x L20X
...
Default deploy config (1 -> 25 frame schedule), 12 prompts x 3 repeats,
warm-up excluded, one GPU via `gpu run`:
1.7B CustomVoice: TTFA p50 32.0 ms, playback-start p50 116 ms (stall at
TTFA 85 ms), RTF_total 0.086, chunk gap p50 163 ms, init 271 s (cold caches)
0.6B CustomVoice: TTFA p50 27.8 ms, playback-start p50 95 ms (stall 67 ms),
RTF_total 0.077, chunk gap p50 146 ms, init 152 s
Per-request chunk arrivals, memory and the full summaries are in the JSON
(force-added: *.json is gitignored repo-wide).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 10:49:37 +08:00
tzhouam and Claude Fable 5.1
b22dcc59f7
[Edge][WP1] Add edge deploy profile and deploy_profile selector
...
* vllm_omni/deploy/edge/qwen3_tts.yaml: single-device Qwen3-TTS profile with a
ramped codec chunk schedule (2->4->8->16->25 frames) that removes the ~80 ms
playback stall of the default 1->25 schedule, decode batch 1, max_num_seqs 4.
* deploy_profile="edge" (Omni/AsyncOmni kwarg, --deploy-profile CLI, headless
serve) resolves vllm_omni/deploy/<profile>/<model_type>.yaml; mutually
exclusive with deploy_config; explicit profiles never fall back silently.
* tests/helpers/stage_config.py: modify_stage_config anchors a relative
base_config before writing the temp YAML so overlay-derived configs resolve.
* async_omni_engine.py also carries the init-timeline hooks used by WP2.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 10:27:33 +08:00
tzhouam and Claude Fable 5.1
b70207f57f
[Edge][WP0] Add offline streaming-latency benchmark for chunked TTS
...
Adds benchmarks/tts/stream_latency_bench.py (AsyncOmni driver reusing the
Qwen3-TTS example's prompt builder; --model override, warm-up/repeat,
--init-only, per-chunk arrivals, peak RSS and per-process GPU memory) and
benchmarks/tts/stream_latency_metrics.py (TTFA, underrun-free playback start,
stall at TTFA, RTF streamed/total, chunk gaps, percentiles) with CPU unit
tests. Results go under benchmarks/tts/edge_results/stream_latency/.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 10:24:53 +08:00
2bae550c04
[Frontend] Opt-in WebSocket TTS split_granularity and session seed ( #7046 )
...
Signed-off-by: rk9595 <rakesh.kariya@somaiya.edu >
Signed-off-by: Rakesh Kariya <rakesh.kariya@somaiya.edu >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-08 19:15:59 -07:00
zhengjia and Cursor
81c1c460b3
[Bugfix][Frontend] Accept the msgpack-numpy package's numpy markers on the OpenPI endpoint ( #6051 )
...
Signed-off-by: zjli2013 <leezhengjiang@126.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-08 19:15:48 -07:00
NancyFyong and NancyFyong
e74fe3c6e8
[Model] Add Audio8 TTS Preview 0.6B (DualAR, 44.1 kHz codec) ( #6157 )
...
Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com >
Co-authored-by: NancyFyong <NancyFyong@users.noreply.github.com >
2026-09-08 19:00:25 -07:00
boatman
cdf027874c
[Feat][OmniVoice]Support Varlen Attn, Request-Batch and Step-Execution ( #6408 )
...
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com >
2026-09-08 18:58:38 -07:00
eval
d7678e2bb3
[Bugfix][MiniCPM-o] Fix the audio_embeds input path ( #5730 )
...
Signed-off-by: eval-dev <0xe5bca0@gmail.com >
Signed-off-by: eval <74645252+eval-dev@users.noreply.github.com >
2026-09-08 18:41:42 -07:00
Zhichao Zhang
5078c7ad66
[Bugfix][MiMo-Audio] Align code2wav decode with tokenizer device ( #6539 )
...
Signed-off-by: chaosansui <zzc15560846421@163.com >
Signed-off-by: Zhichao Zhang <60429419+smartDream-chao@users.noreply.github.com >
2026-09-08 18:38:11 -07:00
Hongsheng Liu
f67cc52f74
[Doc] Add AI usage policy for contributions ( #7305 )
...
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-09-09 09:17:24 +08:00
Anjie Hou and Claude Opus 5
b6f4df320d
[Refactor][Diffusion] Resolve offload topology through one plan resolver ( #7209 )
...
Signed-off-by: specture724 <specture724@gmail.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-09 08:17:38 +08:00
Tianyao Wu
b3dd45874a
[Bugfix] Build the forced-aligner prompt without a chat template (word timestamps one bin late) ( #7240 )
...
Signed-off-by: Tianyao Wu <rayroy31@gmail.com >
2026-09-08 19:27:38 +00:00
Lei Ke and KrystalRay
5e4baea64d
[Bugfix][NPU] Limit MiniMax H3 modulation grid size ( #6794 )
...
Signed-off-by: KrystalRay <keeleiray@gmail.com >
Co-authored-by: KrystalRay <keeleiray@gmail.com >
2026-09-08 09:08:46 -07:00
fdaf4e31bd
Add vLLM-Omni library info to Hugging Face Hub requests ( #5381 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-08 10:08:38 -06:00
liangmenghuang and Claude Fable 5.1
7a80b79698
[Bugfix][Model] GR00T-N1.7: honor the per-request seed for flow-matching noise ( #7253 )
...
Signed-off-by: liangmengh <liangmengh@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-08 15:53:20 +00:00
andyluo7
de1b0708bd
[CI][ROCm] Route LTX2 Ulysses parity to two-GPU lane ( #7234 )
...
Signed-off-by: andyluo7 <andy.luo@amd.com >
2026-09-08 22:59:37 +08:00
Hongsheng Liu
7422a85cc7
[Bugfix] Skip HWR store-size scans when no limit is configured ( #7131 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-09-08 22:54:49 +08:00
Yancy and Asthenia
a31690041d
[Bugfix][Examples] Use --profiler-config flag in offline TTS examples ( #6763 )
...
Signed-off-by: Asthenia <asthenia0412@gmail.com >
Co-authored-by: Asthenia <asthenia0412@gmail.com >
2026-09-08 13:56:27 +00:00
Honghan Zhu and amy-why-3459
d59d166485
[Bugfix] Preserve additional_config for LLM stages ( #7272 )
...
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-09-08 20:44:49 +08:00
Shuolei Wang
5378b77be4
[Model][JoyAI-VL-Interaction] Add native multi-stage pipeline ( #5352 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
2026-09-08 05:07:08 -07:00
liangmenghuang and Claude Fable 5.1
d7740336ff
[Bugfix][Model] GR00T-N1.7: honor processor_config image pipeline (use_albumentations / crop_fraction / letter_box_transform) to match Isaac-GR00T ( #7083 )
...
Signed-off-by: liangmengh <liangmengh@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-08 18:59:01 +08:00
LHXuuu
c9c25b33c3
[Frontend] Add server-side VAD for turn-based omni models ( #6618 )
...
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
2026-09-08 03:17:00 -07:00
Alicia
031612f250
[Bugfix] Fix video benchmark timeout and failure accounting ( #7259 )
...
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com >
2026-09-08 09:40:08 +00:00
andyluo7 and wangyu
b58ff5cb8b
[CI/Build] Wait for duplex reaper recovery in retry test ( #7225 )
...
Signed-off-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
2026-09-08 17:10:51 +08:00
tlysanhuo
7c2e30afd3
[Bugfix] Bound omni benchmark per-request timeout to 15 min by default ( #7130 )
...
Signed-off-by: tly <2200895168@qq.com >
2026-09-08 02:00:25 -07:00
Chenguang Zheng
f3fabe4145
[Bugfix] Keep --disable-log-stats out of stage config resolution ( #7237 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
2026-09-08 16:19:07 +08:00
Honghan Zhu
c086554d1f
[Perf]Boogu-Image Skip dense attention masks ( #6871 )
...
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com >
2026-09-08 00:53:20 -07:00
wuli666
c704aeccae
[Bugfix] Fix Step-Audio2 L4 CI KV cache allocation ( #6839 )
...
Signed-off-by: wuli666 <421774554@qq.com >
2026-09-08 14:17:33 +08:00
Chenguang Zheng
087ac46318
[Refactor 1/N] Centralize Omni config resolution ( #5140 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
2026-09-08 10:30:26 +08:00
Deep Shah and amy-why-3459
1413eaab7c
[Bugfix][Qwen3-Omni] Read code-predictor RoPE theta from rope_parameters ( #7228 )
...
Signed-off-by: Deep Shah <deep@socratic.co >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-09-08 10:25:27 +08:00
ZacheryAU
b1ec136bd7
[Frontend] Add omni benchmark support for image and video endpoints ( #4728 )
2026-09-08 09:28:21 +08:00
Yueqian Lin
e1c59edec6
[Refactor] fish_speech: vendor DAC codec modules, drop external fish-speech dependency ( #4871 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-09-08 09:22:36 +08:00
boatman
043f03d576
[Migrate]Move model-specific helpers out of serving_speech ( #6697 )
...
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com >
2026-09-08 08:59:45 +08:00
Jierui (Jerry) Xu
c3498fb869
[Model] Add Boogu-Image Turbo pipeline resolution ( #6961 )
...
Signed-off-by: Jerry Xu <xjr2423@gmail.com >
2026-09-08 08:59:23 +08:00
6942a5b600
[1/N] Stream Wan VAE chunks to the media consumer ( #7016 )
...
Signed-off-by: specture724 <specture724@gmail.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-09-08 07:41:13 +08:00
Rahul Steiger and Nils Hoffmann
3b335814cc
[Bugfix][Cosmos3] Fix distributed Transfer output envelope ( #7205 )
...
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com >
Co-authored-by: Nils Hoffmann <nhoffmann@nvidia.com >
2026-09-08 07:34:08 +08:00
cf36f5e474
[Bugfix][MiniCPM-o] Allow ragged audio_feature_lens across a batch ( #7071 )
...
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com >
Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-09-07 23:51:13 +08:00
Lei Ke and KrystalRay
7be014bce6
[Bugfix][NPU] Fix MiniMax-H3 INT8 quantization dispatch ( #6876 )
...
Signed-off-by: KrystalRay <keeleiray@gmail.com >
Co-authored-by: KrystalRay <keeleiray@gmail.com >
2026-09-07 14:50:44 +00:00
汪志鹏
9b6a308914
[Perf][Diffusion] Keep LoRA weights resident across activation cycles ( #7195 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-09-07 14:27:24 +00:00
WeiQing Chen
30d6a0b4e6
[Bugfix][MiniMax-H3] Stabilize keyframe VAE encoding ( #7191 )
...
Signed-off-by: david6666666 <530634352@qq.com >
Signed-off-by: David Chen <530634352@qq.com >
2026-09-07 21:52:54 +08:00
Zeyu Huang | 黃澤宇 and Alicia
4b96c57730
[skip ci][Doc] Module design doc for diffusion runtime ( #6440 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
2026-09-07 21:52:24 +08:00
Mu GuanLin
3204a0b2ad
[Model][Perf] Optimize LTX Ulysses sequence parallelism ( #7079 )
...
Signed-off-by: mglyn <1203789601@qq.com >
2026-09-07 10:48:26 +00:00
Rahul Steiger and Rahul Steiger
f080f41e03
[Bugfix][Cosmos3] Auto-pad non-divisible Ulysses GEN sequences ( #6918 )
...
Signed-off-by: Rahul Steiger <rsteiger@nvl72d188-T17.cm.cluster >
Co-authored-by: Rahul Steiger <rsteiger@nvl72d188-T17.cm.cluster >
2026-09-07 08:58:28 +00:00
FredHuangNV
85b0bad2d3
[Frontend][Model] Add Cosmos3 control uploads to video API ( #7027 )
...
Signed-off-by: Fred Huang <frhuang@nvidia.com >
2026-09-07 01:38:56 -07:00
677265b64b
[Bugfix][Diffusion] Make Cosmos3 transfer image arrays writable ( #6915 )
...
Signed-off-by: Rahul Steiger <rsteiger@nvl72d179-T17.cm.cluster >
Co-authored-by: Rahul Steiger <rsteiger@nvl72d179-T17.cm.cluster >
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster >
2026-09-07 01:06:16 -07:00
Shenglei Fu and Claude
91dfafcaab
[Bugfix][Diffusion] Include guidance_scale_2_provided in the request-batch key ( #7078 )
...
Signed-off-by: Shenglei Fu <sfu@confluent.io >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-07 15:58:19 +08:00
summer and Alicia
4b81f692d1
[Bugfix][Diffusion] Canonicalize cache_backend=None to "none" ( #7041 )
...
Signed-off-by: summer <128961079+zhang-keliang@users.noreply.github.com >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
2026-09-07 15:56:51 +08:00
WeiQing Chen
b78c31eda6
[Doc] Update vLLM-Omni WeChat QR code ( #7178 )
...
Signed-off-by: David Chen <530634352@qq.com >
2026-09-07 06:28:30 +00:00
Lucas Ruan
34aa1d2afa
[Doc] Add A100 recipe for Wan2.2-TI2V-5B ( #7189 )
...
Signed-off-by: lucasruan1618 <huy40025071@gmail.com >
2026-09-07 14:09:32 +08:00
BruceLoveDecimal
5cd925631c
[Feature][LingBot World]stepwise execution ( #6844 )
...
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com >
Signed-off-by: liuqihao <liuqihao970610@gmail.com >
2026-09-07 13:38:23 +08:00