Commit Graph
78 Commits
Author SHA1 Message Date
NATURE 3a347b10b0 [3/5][core]refactor communication layer: PR 3 of 5, all other models in non async mode (#3719)
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
2026-05-28 14:06:41 +08:00
Ziming Huangandtzhouam 220ce0b4ea [Test] Add long text output correctness test for Qwen3-Omni (#3539)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: tzhouam <tzhouam@connect.ust.hk>
2026-05-28 08:27:29 +08:00
LancerandAlicia 5c3c81920b [Bugfix] Ensure stage and diffusion subprocesses exit when parent dies unexpectedly (#3751)
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
2026-05-28 08:22:04 +08:00
Yueqian LinandRein Yang e996723bc9 [Bugfix] Fix Qwen3-TTS Stage 0 prefix-caching correctness (#3665)
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>

the ci failure case is irrelavent to this PR and occurs rarely
2026-05-25 23:09:38 +08:00
33220b1e39 [BugFix] Finish async_chunk requests without pad-token injection (#3613)
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-05-16 23:34:04 +12:00
+20 eb4e60ee64 [Rebase] Rebase to vllm v0.21.0 (#3530)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Dnoob <dxpouo@gmail.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: rein yang <ruiruyang2@gmail.com>
Signed-off-by: Nick Cao <ncao@redhat.com>
Signed-off-by: zhumingjue <zhumingjue@huawei.com>
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com>
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: weizhoublue <weizhoublue@github.com>
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io>
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: NumberWan <wantszkin2003@gmail.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Dnoob <dxpouo@gmail.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com>
Co-authored-by: Ricardo Noriega <rnoriega@redhat.com>
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: weizhoublue <45163302+weizhoublue@users.noreply.github.com>
Co-authored-by: weizhoublue <weizhoublue@github.com>
Co-authored-by: dengyunyang <584797741@qq.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: NATURE <wzliu@connect.hku.hk>
Co-authored-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Y. Fisher <yukexiong1@huawei.com>
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-16 18:18:43 +08:00
NATUREandHongsheng Liu 82a0b3a467 [2/5] [core]refactor communication layer: PR 2 of 5 Qwen3 Omni non async (#2677)
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-05-15 14:14:08 +08:00
Alex Brooks bb239fa949 [Core] Support Async & Sync AutoRegressive Scheduling (#3306)
Signed-off-by: Alex Brooks <albrooks@redhat.com>
2026-05-05 12:09:45 +08:00
汪志鹏andHongsheng Liu e50a066548 [BugFix]: Fix async scheduer transfer exceed KV cache (#3318)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-05-03 20:02:22 +08:00
amy-why-3459andHongsheng Liu 01f500a5bf [perf]Qwen3-Omni performance optimization (#3203)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-05-01 22:47:24 +08:00
Zhou TaichangandYuanheng Zhao 7f2b1e9386 [Rebase] Rebase to vllm 0.20.0 (#3232)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com>
2026-04-30 09:56:59 +12:00
Hongsheng Liu 9de131e6b0 Revert "[perf]Qwen3-Omni performance optimization" (#3202) 2026-04-28 13:23:29 +08:00
amy-why-3459 335e2eb9e4 [perf]Qwen3-Omni performance optimization (#3164)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
2026-04-28 11:59:26 +08:00
2390350533 [Refactor] Standardize data entry key names to {type}.{qualifier} format (#1829)
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-04-25 18:15:38 +08:00
ChenWenjing 4124a1f858 [Feature]Support audio streaming input and output-phase2 (#2581)
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
2026-04-18 15:42:41 +08:00
amy-why-3459 b88d3ce75e [BugFix] Fixing occasional engine crashes caused by abort requests (#2871)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
2026-04-17 09:15:35 +00:00
Alex Brooks f1e3f03726 [feature] Hidden State Prefix Caching (#2164)
Signed-off-by: Alex Brooks <albrooks@redhat.com>
2026-04-15 17:01:45 +00:00
NATUREandHongsheng Liu 0d4e975e1b [core]refactor communication layer: PR1(Added Refactor Infra Only) (#1555)
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-04-13 16:01:14 +08:00
NATURE e12250119b [Bugfix] Fix Bagel online mode for 1. Hang after several requests 2. Non-deterministic image quality regression. (#2458)
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
2026-04-13 03:36:20 +00:00
汪志鹏 8dd66ceb00 [BugFix] Continue decode if don't need transfer kv cache between two … (#2502)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
2026-04-07 13:30:01 +12:00
indevnandlinyueqian 6fc38e0467 [Model][Core] Enable async_chunk streaming pipeline for CosyVoice3 (#1703)
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: indevn <indevn@outlook.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
2026-04-05 06:11:33 +00:00
Zhou Taichang 2804a85e58 [Rebase] Rebase to vllm v0.19.0 (#2475) 2026-04-04 15:03:32 +08:00
NATURE bbae904f1d [Bugfix] Fix delayed decoding bug for Bagel AR/DIT workflow (L3 test_bagel_img2img error) (#2422)
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
2026-04-01 15:01:39 +00:00
NATURE d40840b114 [Perf] Bagel KV-ready early forwarding and time step consistency for /v1/chat/completions (#2398)
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
2026-04-01 08:50:45 +00:00
Du Bin 69652a2983 [Bugfix] Fix memory leak: missing chunk_transfer_adapter.cleanup() in OmniARScheduler (#2028)
Signed-off-by: dubin555 <dubin555@gmail.com>
Signed-off-by: OSS Scout <scout@oss-scout.local>
2026-03-24 10:52:31 -04:00
Nick Cao 47ba887b93 [Feat] Support scalar types in AdditionalInformationEntry (#2105)
Signed-off-by: Nick Cao <ncao@redhat.com>
2026-03-24 10:29:38 -04:00
Du Bin b4a96b0db4 [Bugfix] Restore chunk-waiting requests on OmniNewRequestData rewrap failure (#1691)
Signed-off-by: dubin555 <dubin555@gmail.com>
Signed-off-by: OSS Scout <scout@oss-scout.local>
2026-03-22 14:32:51 +08:00
Zhou Taichangandgcanlin a90a7690b8 [Rebase] Rebase to vllm v0.18.0 (#2037)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
2026-03-21 20:37:38 +08:00
89b527e533 Dev/rebase v0170 (#1639)
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
2026-03-09 17:29:38 +08:00
Yueqian Lin a3240fd484 [Bugfix] Fix transformers 5.x compat issues in online TTS serving (#1536)
Signed-off-by: linyueqian <linyueqian@outlook.com>
2026-03-04 01:28:32 +08:00
Junhong Liu 742d845b8e Make chunk_size and left_context_size configurable via YAML for async chunking (#1423)
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
2026-03-03 22:06:52 +08:00
Sy03 0cc3481acf [Feat] : Support Async chunk cleanup (#1087)
Signed-off-by: Sy03 <1370724210@qq.com>
2026-02-27 14:39:33 +08:00
Junhong LiuandGao Han 919ad28da5 [BugFix] process request.num_cached_tokens if it equals to the initial value (#1468)
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
2026-02-26 10:59:18 +08:00
Sy03andHongsheng Liu efbe4116ff [Feature] : Support disaggregated inference pipeline for Qwen3_TTS (#1161)
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-02-21 07:36:44 +08:00
75770c9309 [Rebase] Rebase to vllm v0.16.0 (#1357)
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
2026-02-13 18:05:07 +08:00
ab1acb27ff [Feature]async scheduling to overlap chunk IO and compute (#951)
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Bhanu068 <voutharoja.bhanu06@gmail.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
2026-02-12 16:05:02 +08:00
amy-why-3459 d1cf4c4c0f [Bugfix] async_chunk rebase v0.15.0 (#1195)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
2026-02-04 20:50:09 +08:00
Zhou Taichangand汪志鹏 70ef548448 Dev/rebase v0.15.0 (#1159)
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
2026-02-03 12:24:45 +08:00
amy-why-3459 d2b3a24af3 [BugFix] Modify the method of obtaining external_request_id (#961)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
2026-01-27 20:56:02 +08:00
52bf1e2f40 [Feature]Support async computation and communication across stages by chunks (#727)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-01-26 11:21:24 +08:00
汪志鹏andwzliu 7f821be57b Support Bagel Model (#726)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: wzliu <wzliu@connect.hku.hk>
2026-01-22 23:09:55 +13:00
Gao Han a54c323952 [Model] Support Qwen3-TTS model series (#895)
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
2026-01-22 16:57:37 +08:00
+13 776c3a794e Dev/rebase 0.14.0 (#847)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
2026-01-21 00:25:25 +08:00
8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2025-12-19 16:42:02 +08:00
Cyrus Leung c091a03d82 [Chore] Bump Python version and apply formatting changes for v0.12 release (#126)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2025-12-16 15:19:40 +08:00
Lancer 31fedcfa2b [Misc] fix yaml config (#320)
Signed-off-by: Lancer <maruixiang6688@gmail.com>
2025-12-16 02:40:30 +00:00
汪志鹏 6c333161a4 [BugFix]Remove duplicate code (#309)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
2025-12-15 01:42:55 +00:00
Hongsheng LiuandWeiQing Chen f22944eea4 [Doc] fix docs Feature Design and Module Design (#283)
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
2025-12-12 14:52:53 +08:00
Zhou TaichangandGao Han c70b167f70 [Engine] Remove Diffusion_XX which duplicates with Generation_XX (#163)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Gao Han <gaohan19@huawei.com>
2025-12-05 03:04:23 +00:00
Samit 04987ffa7a [Bugfix] Fix Qwen-omni Online Inference Bug caused by check_stop and long sequence (#112)
Signed-off-by: samithuang <285365963@qq.com>
2025-11-30 23:23:45 +08:00