 tzhouamandClaude Fable 5.1
|
073d31b9cd
|
[Edge][WP2] Add initialization timeline and cold-start profiler
* vllm_omni/engine/init_timeline.py: InitTimeline phase collector; engine
phases (stages_init, g3_device_lock, g2_spawn/ready) and worker phases
(proc_entry, engine_core_init, init_device, load_model/load_weights,
talker_mtp_init, attn_prewarm, cudagraph_capture, talker_mtp_capture,
code2wav_cudagraph, predictor_setup_compile/warmup/cudagraph) recorded via
worker_phase/timed_phase; workers append JSONL under
VLLM_OMNI_INIT_TIMELINE_DIR (inherited at spawn) and the engine merges them.
Enabled by VLLM_OMNI_INIT_TIMELINE=<json> or log_stats; table logged at INFO,
partial table on orchestrator startup timeout, suggested timeouts logged.
* omni_base.py: named deploy profiles default to stage_init_timeout=900 /
init_timeout=1500 unless given explicitly.
* gpu_model_runner._init_talker_mtp tolerates max_cudagraph_capture_size=None;
gpu_ar_model_runner.shutdown skips CUDA graph teardown on CPU.
* benchmarks/tts/init_profile.py: runs stream_latency_bench --init-only per
configuration (default, cold_cache, eager_stage1, eager_both,
parallel_stage_init, edge, ...) and tabulates phase durations.
* stream_latency_bench: --parallel-stage-init / --deploy-profile flags; the
resource poller no longer shadows Thread._stop and skips GPU sampling on CPU.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-09 11:01:39 +08:00 |
|
|
BruceLoveDecimal
|
5cd925631c
|
[Feature][LingBot World]stepwise execution (#6844)
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
|
2026-09-07 13:38:23 +08:00 |
|
|
汪志鹏andClaude Opus 5
|
ae70479619
|
[Bugfix] Drop the removed diffusion_batch_size kwarg from the stage-init test (#7101)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-09-05 17:05:33 +00:00 |
|
|
Zhou TaichangandClaude Fable 5
|
142151fef5
|
Parallel stage initialization (admission + SH/EX device locks) (Worker refactor 1/N) (#5224)
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-05 22:22:41 +08:00 |
|
|
|
5d7fdf9350
|
[Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196)
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
|
2026-09-05 08:20:21 +00:00 |
|
|
SYLARandhsliuustc0106
|
7493cbd3c2
|
[Bugfix] Fix Qwen3-Omni MoE backend selection (#7019)
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
|
2026-09-04 21:16:58 +08:00 |
|
|
Arifuzzaman Joy
|
3cddda9781
|
[Bugfix] Prevent build_engine_args_dict from mutating stage_config.engine_args (#6783)
Signed-off-by: Arifuzzamanjoy <s1710374103@ru.ac.bd>
|
2026-09-01 16:45:28 +00:00 |
|
|
Andy Zhou
|
2298359f67
|
[Bugfix][Diffusion] Preserve stage attention backend (#6645)
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
|
2026-09-01 20:18:44 +08:00 |
|
|
Yuxuan RhoandHongsheng Liu
|
7992c2213e
|
[Config] Remove diffusion_batch_size and use max_num_seqs instead. (#6484)
Signed-off-by: LyxWxj <yuxuanlou621@foxmail.com>
Signed-off-by: Yuxuan Rho <104440472+LyxWxj@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-09-01 01:02:54 +00:00 |
|
|
TJian
|
ded8934626
|
[AMD] [CI] Fix tests for v0.28.0 (#6830)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
|
2026-08-31 09:10:27 +08:00 |
|
|
Gao Han
|
f7f48929db
|
[Bugfix][Diffusion] Restore inline single-stage execution (#6813)
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
|
2026-08-30 21:40:47 +08:00 |
|
|
Yueqian Lin
|
a6b559cf54
|
[Bugfix][MiniMax-Music3] Resolve stage subdirs against a real snapshot (#6640)
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
|
2026-08-30 06:51:24 +00:00 |
|
|
|
21dd4b33ac
|
[Bugfix][MiniCPM-o] Stabilize native duplex streaming and Stage-1 handoffs (#6529)
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Nathan Price <nathan@abridge.com>
Co-authored-by: Eric(Qun) <186020953+0xtoward@users.noreply.github.com>
|
2026-08-29 13:05:24 -07:00 |
|
|
Sy03
|
3adee400a0
|
[Bugfix][Qwen3-TTS] Handle codec generations that exhaust their token budget (#6728)
Signed-off-by: Sy03 <1370724210@qq.com>
|
2026-08-29 08:59:05 -07:00 |
|
|
|
bb5bf8fd0a
|
[Bugfix] Fix colocate-async sleep admission race and deliver AR abort tokens (#6367)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Samit <285365963@qq.com>
|
2026-08-29 09:11:39 +00:00 |
|
|
|
d94061201d
|
[Bugfix][Engine] Make abort idempotent during shutdown (#6327)
Signed-off-by: Allen Wu <allenwu2795@gmail.com>
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
|
2026-08-28 19:51:25 -07:00 |
|
|
Sy03
|
009b992c5b
|
[Bugfix][MiniCPM-o] Isolate duplex handoff failures and barge-in (#6170)
Signed-off-by: Sy03 <1370724210@qq.com>
|
2026-08-28 15:46:41 +08:00 |
|
|
|
b6a4baa659
|
[Feature] Support text encoder Disaggregation for H3 (#5885)
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: yuanwu <yuan.wu@intel.com>
|
2026-08-28 14:27:05 +08:00 |
|
|
NATURE
|
54c1fc64b7
|
[Bugfix] Gate async-chunk segment resume by session mode (#6680)
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
|
2026-08-27 18:52:09 +08:00 |
|
|
Zhou TaichangandClaude Opus 5
|
651b03225b
|
[Rebase] Rebase to vllm 0.28.0 (#6606)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-27 06:12:55 +00:00 |
|
|
|
c2723979c0
|
[Bugfix] Validate per-stage device layout before spawning workers (#5003) (#5742)
Signed-off-by: ChoHee15 <cc5281@126.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
|
2026-08-26 19:25:22 +00:00 |
|
|
Yueqian LinandYueqian Lin
|
8cfede4309
|
[Perf][Engine] Event-driven orchestration loop (opt-in) — S1 of #4855 (#5221)
Signed-off-by: Yueqian Lin <gao@yueqian.us>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <gao@yueqian.us>
|
2026-08-26 22:15:25 +08:00 |
|
|
psv666andamy-why-3459
|
5ec576ab4c
|
[Bugfix] Fix missing Realtime audio completion (#6564)
Signed-off-by: psv666 <2693925048@qq.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
|
2026-08-26 18:27:25 +08:00 |
|
|
|
7ae15c74c8
|
text and image metric (#6150)
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
|
2026-08-26 02:54:45 +00:00 |
|
|
Alex Brooks
|
0eb47b76d0
|
[Bugfix] Handle DoS Overflow Cases (#6598)
Signed-off-by: Alex Brooks <albrooks@redhat.com>
|
2026-08-25 16:09:03 +00:00 |
|
|
NumberWan
|
bc8ea489ef
|
[BugFix] Qwen-image performance regressed - Avoid mapping diffusion_batch_size onto scheduler max_num_seqs (#6525)
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
|
2026-08-25 03:36:11 -07:00 |
|
|
wy17003andHongsheng Liu
|
b454e34966
|
[Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. (#6149)
Signed-off-by: wy17003 <yaow17003@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-08-25 15:24:59 +08:00 |
|
|
|
f0808c9793
|
[Bugfix] Fix remote replica membership lifecycle races (#5277)
Signed-off-by: hbhflw2000 <417911774@qq.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Samit <285365963@qq.com>
|
2026-08-25 10:34:59 +08:00 |
|
|
wangyuandamy-why-3459
|
4af0ddc1c2
|
[Test] Cleanup for helpers (#6523)
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
|
2026-08-25 09:10:17 +08:00 |
|
|
Nguyen Kha Nhat LongandCursor
|
d15b7ca3e1
|
[Feature]: Support pause / resume and sleep / wake for AR stages in AsyncOmni (#6084)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-08-24 16:21:32 +08:00 |
|
|
akshatvishu
|
39c16d75a9
|
[tts][bugfix] Fix CosyVoice3 sampling and stage handoff (#6424)
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
|
2026-08-22 21:20:57 -07:00 |
|
|
Yash Jain
|
c6e38860e9
|
[Bugfix][Core] Fix replica device split when a stage omits devices (#5445)
Signed-off-by: Yash Jain <yashjainjain1704@gmail.com>
|
2026-08-21 00:44:03 +08:00 |
|
|
Inesh Reddy Chappidi
|
cd3867c373
|
[Model] Suppress silence codec tokens for the first N Qwen3-TTS decod… (#5048)
Signed-off-by: Inesh Reddy <ineshreddy249@gmail.com>
|
2026-08-20 09:22:37 -07:00 |
|
|
amy-why-3459andCursor
|
9c61d5b67b
|
[Bugfix][MiniCPM-o] Stop Talker mid-utterance truncation and load cod… (#6346)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-08-20 11:50:07 +08:00 |
|
|
Wang fuyinandHongsheng Liu
|
e1c5f59aab
|
[Config] Reuse vLLM configs in VllmOmniConfig (#6050)
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-08-18 21:27:58 +08:00 |
|
|
Yueqian Lin
|
b283215c28
|
[Model] Add MiniMax Music 3 text-to-music (#6186)
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
|
2026-08-18 04:33:13 +00:00 |
|
|
Yueqian LinandHongsheng Liu
|
611c083587
|
[BugFix][TTS] R1.1-R1.4: turn silent async-chunk failures into visible ones (#6033)
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-08-17 10:51:17 +08:00 |
|
|
hurukawa
|
2a315e1a2d
|
[Perf] Support request-level batching for Wan2.2 pipelines (#5676)
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: nagisa <1434936049@qq.com>
|
2026-08-15 23:58:36 +08:00 |
|
|
m0g3r
|
09b6a1604a
|
[Bugfix] Restore per-stage runtime env during launch (#6214)
Signed-off-by: m0g3r <87276771+m0g3r@users.noreply.github.com>
|
2026-08-15 13:00:27 +00:00 |
|
|
MrlixiangWE
|
e57574ae29
|
[Bugfix] Carry ec_transfer_params and num_cache_creation_tokens on OmniRequestOutput (#6152)
Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>
|
2026-08-15 08:37:27 +00:00 |
|
|
TJianandHongsheng Liu
|
bc73546c06
|
[ROCm] [CI] Migrate CI to mi300x for v0.27.x (#5886)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-08-14 15:38:47 +08:00 |
|
|
Sy03
|
dbc0dd6d85
|
[Bugfix][Higgs-Audio-V3] Disable XQA decode (#6068)
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Sysy <1370724210@qq.com>
|
2026-08-13 10:00:50 -07:00 |
|
|
Yukim1andHongsheng Liu
|
4b66f62e10
|
[Refactor] Remove internal stage_configs_path plumbing (#5741)
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-08-13 21:29:03 +08:00 |
|
|
|
fcf1085395
|
[Refactor][OutputProcessor 2/8]: OmniRequestOutput should inherit RequestOutput and no nested wrap-up (#5146)
Signed-off-by: Boao Shi <aoibosh@connect.hku.hk>
Signed-off-by: bowieshi <aoibosh@connect.hku.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Shi Boao Bowie <aoibosh@connect.hku.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: ruirui(rein) yang <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-08-12 22:58:23 +08:00 |
|
|
Sy03andYueqian Lin
|
58007c0084
|
[New Model] Support IndexTTS 2.5 (#5957)
Signed-off-by: Sysy <1370724210@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
|
2026-08-11 23:00:35 +00:00 |
|
|
Zhou TaichangandClaude Opus 5
|
2ce15ab7a0
|
[Rebase] Rebase to vllm 0.27.0 (#5976)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-11 15:23:46 +00:00 |
|
|
|
322600bfb2
|
[CI][MiniCPM-o] Add MiniCPM-o 4.5 accuracy and performance coverage (#5524)
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
Signed-off-by: psv666 <2693925048@qq.com>
Co-authored-by: psv666 <2693925048@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
|
2026-08-11 14:53:49 +08:00 |
|
|
|
ee118ab492
|
[Bugfix] Per-replica fault isolation: keep API server alive on single-stage death (#4285) (#4583)
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
|
2026-08-10 23:00:18 -04:00 |
|
|
Caspian ChenandSYLAR
|
593b404539
|
[Quantization][Qwen2.5-Omni] Support thinker-only ModelOpt NVFP4 W4A4 checkpoints (#5073)
Signed-off-by: Caspian443 <scrisis843@gmail.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
|
2026-08-09 02:45:57 +08:00 |
|
|
Wang fuyin
|
3d7fc3b9ba
|
[Config] Read engine args from VllmOmniConfig (#5678)
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Signed-off-by: Wang fuyin <wfy2003324@163.com>
|
2026-08-07 11:17:33 +08:00 |
|