Gao Han
e8193787cc
Merge branch 'main' into fix/pr4709-review-followup
2026-07-18 11:28:29 +08:00
Zhou Taichang and Claude Opus 4.8
d3c47efc4f
[Rebase] Rebase to vllm v0.25.0 ( #5042 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-07-12 23:39:53 +08:00
Sy03 and Yueqian Lin
9eaca72382
[Perf][VoxCPM2] Add unified decode graph optimization ( #4606 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-07-04 05:30:48 +00:00
tzhouam and Claude Fable 5
35a423c9bd
[BugFix][CI] Address review follow-ups from v0.24.0 rebase ( #4709 )
...
Addresses Gaohan123's post-approval review comments on #4709 :
- pipeline_dreamzero: restore platform-dispatched synchronize() instead of
torch.cuda so DreamZero keeps working on non-CUDA hardware (regression
of #4399 introduced during the rebase).
- omni_scheduler_mixin: drop the VLLM_OMNI_INPUT_WAIT_TIMEOUT_S env-var
override; the full-payload input-wait timeout is now a plain constant.
- diffusion_worker: implement _DiffusionVllmModelConfig.is_diffusion by
mirroring upstream ModelConfig.is_diffusion (dLLM canvas_length check)
and document why omni DiT pipelines must not enable dLLM paths.
- qwen3_tts: make _embedding_dtype a class attribute on the talker and
prompt-embeds builder so __new__-based test stubs inherit it; remove
the (duplicated) manual assignments from the preprocess test.
- .buildkite: document why the rebase raised per-step timeouts (steps
were killed by the old limits, masking real failures; e.g. CustomVoice
passes in ~9-15 min but cold runs were killed at 20m).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-07-02 07:36:14 +00:00
Zhou Taichang
a560ed184d
[Rebase] Rebase to vllm v0.24.0 ( #4709 )
2026-06-30 19:41:06 +08:00
504698dec8
[Core] Performance fix for orchestrator bottleneck: separates inter-stage from client outputs ( #4527 )
...
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: Chenguang ZHENG <645327136@qq.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-06-29 17:43:36 +02:00
BeatSeat
044240d8eb
[Model] Add IndexTTS2 text-to-speech support ( #3838 )
...
Signed-off-by: BeatSeat <wendavid552@gmail.com >
Signed-off-by: David <david@podcast.ai >
Signed-off-by: wendavid552 <wendavid552@gmail.com >
Signed-off-by: WenDavid <davidzhangcf@foxmail.com >
2026-06-22 19:24:01 +00:00
LHXuuu and Hongsheng Liu
dff757da9c
[Perf][PrefixCache] Avoid per-step blocking write in OmniTensorPrefix… ( #4106 )
...
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-17 16:53:47 +08:00
Zhou Taichang
7b837944a1
[Rebase] Rebase to vllm 0.23.0 ( #4286 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-06-13 16:43:02 +00:00
8f869c8ea5
[Refactor] Output Processor Phase 2: separate multimodal output channel ( #1601 ) ( #2744 )
...
Signed-off-by: Megha Agarwal <agarwalmegha1308@gmail.com >
Signed-off-by: meghaagr13 <divyanshsinghvi@gmail.com >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: meghaagr13 <divyanshsinghvi@gmail.com >
Co-authored-by: Boao Shi <aoibosh@connect.hku.hk >
Co-authored-by: Taichang Zhou <tzhouam@connect.ust.hk >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-06-12 15:19:14 +08:00
Juan Pablo Zuluaga and JuanPZuluaga
0342827dd7
[Bugfix] qwen3-tts prefix cache: drop per-key size cap that corrupted… ( #4317 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com >
2026-06-10 21:20:08 +08:00
longguo and npuichigo
73df8326d7
[Bugfix] purge chunk-transfer zombies on every schedule tick to keep engine-core alive on aborts ( fixes #3736 ) ( #3774 )
...
Signed-off-by: abinggo <107740309+abinggo@users.noreply.github.com >
Co-authored-by: npuichigo <11533479+npuichigo@users.noreply.github.com >
2026-06-10 09:00:23 +08:00
57227dc7f3
[Perf][Bugfix] qwen3-tts hot path: prefix-cache OOM guards + talker/orchestrator micro-opts ( #3689 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-06-09 10:36:57 -07:00
Yueqian Lin
82cce633a3
[Perf][Higgs-Audio-V3] Turn on Stage-0 prefix caching by default ( #4199 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-06-07 03:09:47 +00:00
ChenWenjing
e25239c2ec
[bugfix] fix realtime ci timeout error ( #4187 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
2026-06-06 00:05:07 +08:00
ChenWenjing
3842970a82
[bugfix] fix streaming input issue after rebase 0.22.0 ( #4085 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
2026-06-03 16:07:41 +08:00
LHXuuu and vraiti
41a795b5d6
[Metrics] Add audio SLOs + cross-stage transfer families + per-(stage, replica) wrap for upstream vllm:* ( #3576 )
...
Signed-off-by: vraiti <vraiti@redhat.com >
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
Co-authored-by: vraiti <vraiti@redhat.com >
2026-06-01 20:20:48 +08:00
TaffyOfficial and TaffyOfficial
ee33601571
[Perf] Deduplicate AR prefix cache hidden-state CPU staging ( #3734 )
...
Signed-off-by: TaffyOfficial <2324465096@qq.com >
Co-authored-by: TaffyOfficial <2324465096@qq.com >
2026-06-01 09:35:07 +00:00
+1
f92d84fcf9
[Rebase] Rebase to vllm releases/v0.22.0 ( #3891 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: david6666666 <530634352@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: nainiu258 <cperfect02@163.com >
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com >
Co-authored-by: Zheng Wengang <zwg0606@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-06-01 11:08:12 +08:00
ChenWenjing
5dfdf584dd
[Feat]audio streaming input for async chunk ( #3614 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
2026-05-31 17:39:25 +08:00
NATURE
3a347b10b0
[3/5][core]refactor communication layer: PR 3 of 5, all other models in non async mode ( #3719 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-05-28 14:06:41 +08:00
Ziming Huang and tzhouam
220ce0b4ea
[Test] Add long text output correctness test for Qwen3-Omni ( #3539 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: tzhouam <tzhouam@connect.ust.hk >
2026-05-28 08:27:29 +08:00
Lancer and Alicia
5c3c81920b
[Bugfix] Ensure stage and diffusion subprocesses exit when parent dies unexpectedly ( #3751 )
...
Signed-off-by: Lancer <maruixiang6688@gmail.com >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
2026-05-28 08:22:04 +08:00
Yueqian Lin and Rein Yang
e996723bc9
[Bugfix] Fix Qwen3-TTS Stage 0 prefix-caching correctness ( #3665 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
the ci failure case is irrelavent to this PR and occurs rarely
2026-05-25 23:09:38 +08:00
33220b1e39
[BugFix] Finish async_chunk requests without pad-token injection ( #3613 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-16 23:34:04 +12:00
+20
eb4e60ee64
[Rebase] Rebase to vllm v0.21.0 ( #3530 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: NumberWan <wantszkin2003@gmail.com >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: Dnoob <dxpouo@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Signed-off-by: rein yang <ruiruyang2@gmail.com >
Signed-off-by: Nick Cao <ncao@redhat.com >
Signed-off-by: zhumingjue <zhumingjue@huawei.com >
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com >
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: weizhoublue <weizhoublue@github.com >
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io >
Signed-off-by: dengyunyang <584797741@qq.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
Signed-off-by: bjf-frz <frz123db@gmail.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: KexiongYu <yukexiong1@huawei.com >
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: NumberWan <wantszkin2003@gmail.com >
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: Dnoob <dxpouo@gmail.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com >
Co-authored-by: Nick Cao <ncao@redhat.com >
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com >
Co-authored-by: Ricardo Noriega <rnoriega@redhat.com >
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: weizhoublue <45163302+weizhoublue@users.noreply.github.com >
Co-authored-by: weizhoublue <weizhoublue@github.com >
Co-authored-by: dengyunyang <584797741@qq.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: NATURE <wzliu@connect.hku.hk >
Co-authored-by: bjf-frz <frz123db@gmail.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Y. Fisher <yukexiong1@huawei.com >
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-05-16 18:18:43 +08:00
NATURE and Hongsheng Liu
82a0b3a467
[2/5] [core]refactor communication layer: PR 2 of 5 Qwen3 Omni non async ( #2677 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-15 14:14:08 +08:00
Alex Brooks
bb239fa949
[Core] Support Async & Sync AutoRegressive Scheduling ( #3306 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-05-05 12:09:45 +08:00
汪志鹏 and Hongsheng Liu
e50a066548
[BugFix]: Fix async scheduer transfer exceed KV cache ( #3318 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-03 20:02:22 +08:00
amy-why-3459 and Hongsheng Liu
01f500a5bf
[perf]Qwen3-Omni performance optimization ( #3203 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-01 22:47:24 +08:00
Zhou Taichang and Yuanheng Zhao
7f2b1e9386
[Rebase] Rebase to vllm 0.20.0 ( #3232 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com >
2026-04-30 09:56:59 +12:00
Hongsheng Liu
9de131e6b0
Revert "[perf]Qwen3-Omni performance optimization" ( #3202 )
2026-04-28 13:23:29 +08:00
amy-why-3459
335e2eb9e4
[perf]Qwen3-Omni performance optimization ( #3164 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-04-28 11:59:26 +08:00
2390350533
[Refactor] Standardize data entry key names to {type}.{qualifier} format ( #1829 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-25 18:15:38 +08:00
ChenWenjing
4124a1f858
[Feature]Support audio streaming input and output-phase2 ( #2581 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
2026-04-18 15:42:41 +08:00
amy-why-3459
b88d3ce75e
[BugFix] Fixing occasional engine crashes caused by abort requests ( #2871 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-04-17 09:15:35 +00:00
Alex Brooks
f1e3f03726
[feature] Hidden State Prefix Caching ( #2164 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-04-15 17:01:45 +00:00
NATURE and Hongsheng Liu
0d4e975e1b
[core]refactor communication layer: PR1(Added Refactor Infra Only) ( #1555 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-13 16:01:14 +08:00
NATURE
e12250119b
[Bugfix] Fix Bagel online mode for 1. Hang after several requests 2. Non-deterministic image quality regression. ( #2458 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-04-13 03:36:20 +00:00
汪志鹏
8dd66ceb00
[BugFix] Continue decode if don't need transfer kv cache between two … ( #2502 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-04-07 13:30:01 +12:00
indevn and linyueqian
6fc38e0467
[Model][Core] Enable async_chunk streaming pipeline for CosyVoice3 ( #1703 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: indevn <indevn@outlook.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
2026-04-05 06:11:33 +00:00
Zhou Taichang
2804a85e58
[Rebase] Rebase to vllm v0.19.0 ( #2475 )
2026-04-04 15:03:32 +08:00
NATURE
bbae904f1d
[Bugfix] Fix delayed decoding bug for Bagel AR/DIT workflow (L3 test_bagel_img2img error) ( #2422 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-04-01 15:01:39 +00:00
NATURE
d40840b114
[Perf] Bagel KV-ready early forwarding and time step consistency for /v1/chat/completions ( #2398 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-04-01 08:50:45 +00:00
Du Bin
69652a2983
[Bugfix] Fix memory leak: missing chunk_transfer_adapter.cleanup() in OmniARScheduler ( #2028 )
...
Signed-off-by: dubin555 <dubin555@gmail.com >
Signed-off-by: OSS Scout <scout@oss-scout.local >
2026-03-24 10:52:31 -04:00
Nick Cao
47ba887b93
[Feat] Support scalar types in AdditionalInformationEntry ( #2105 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-03-24 10:29:38 -04:00
Du Bin
b4a96b0db4
[Bugfix] Restore chunk-waiting requests on OmniNewRequestData rewrap failure ( #1691 )
...
Signed-off-by: dubin555 <dubin555@gmail.com >
Signed-off-by: OSS Scout <scout@oss-scout.local >
2026-03-22 14:32:51 +08:00
Zhou Taichang and gcanlin
a90a7690b8
[Rebase] Rebase to vllm v0.18.0 ( #2037 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: gcanlin <canlinguosdu@gmail.com >
2026-03-21 20:37:38 +08:00
89b527e533
Dev/rebase v0170 ( #1639 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-03-09 17:29:38 +08:00
Yueqian Lin
a3240fd484
[Bugfix] Fix transformers 5.x compat issues in online TTS serving ( #1536 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-04 01:28:32 +08:00