Commit Graph
198 Commits
Author SHA1 Message Date
BeatSeat 044240d8eb [Model] Add IndexTTS2 text-to-speech support (#3838)
Signed-off-by: BeatSeat <wendavid552@gmail.com>
Signed-off-by: David <david@podcast.ai>
Signed-off-by: wendavid552 <wendavid552@gmail.com>
Signed-off-by: WenDavid <davidzhangcf@foxmail.com>
2026-06-22 19:24:01 +00:00
rein yang afbcc51672 [Model] Aura support - Non async chunk path (#4257)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
2026-06-21 22:08:26 +08:00
amy-why-3459 f05e03aea8 [Perf][DreamZero]dreamzero supports CUDA graph, torch.compile, and DiT Caching. (#4213)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
2026-06-19 14:24:37 +08:00
f75c437a60 [Refactor]Base class for dit pipelines with unified parameter declaration[1/N] (#4225)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 17:14:18 +08:00
64a5bb92de [Refactor] Omni Stage Runtime and Distributed Replica Control Plane (#3855)
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Chenguang ZHENG <645327136@qq.com>
2026-06-17 14:13:20 +08:00
Zheng WengangandHongsheng Liu ba4932c364 [Bugfix] Fix multi-replica stage identity and clean up redundant deploy config (#4410)
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-06-16 14:08:11 +08:00
Hongsheng Liu aff0af9754 [Refactor] Extract diffusion output formatting boundary (#4407)
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
2026-06-15 22:50:50 +08:00
Zeng ChuangandHongsheng Liu 10404e4c5b [Perf] Restore parallel stage initialization for AR+DiT pipelines (#3641)
Signed-off-by: zengchuang <zengchuang3@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-06-15 09:37:11 +08:00
Zhou Taichang 7b837944a1 [Rebase] Rebase to vllm 0.23.0 (#4286)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-06-13 16:43:02 +00:00
8f869c8ea5 [Refactor] Output Processor Phase 2: separate multimodal output channel (#1601) (#2744)
Signed-off-by: Megha Agarwal <agarwalmegha1308@gmail.com>
Signed-off-by: meghaagr13 <divyanshsinghvi@gmail.com>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: meghaagr13 <divyanshsinghvi@gmail.com>
Co-authored-by: Boao Shi <aoibosh@connect.hku.hk>
Co-authored-by: Taichang Zhou <tzhouam@connect.ust.hk>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 15:19:14 +08:00
MaciejBalaNVandHongsheng Liu 03cc888485 [refactor] Refactor guardrail error handling - add 400 error code (#4297)
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-06-11 21:09:24 +00:00
dengyunyangandHongsheng Liu 14879d6619 [Bugfix][HunyuanImage] fix accuracy in stream mode (#4265)
Signed-off-by: dengyunyang <584797741@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-06-11 10:39:00 +08:00
57ea047acf Step audio R1 reasoning parser (#2846)
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: Mike Qiu <qdy220091330@gmail.com>
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-06-10 21:03:19 +00:00
akshatvishuandHongsheng Liu 2fa4c7a4a9 [Model] Add Ming-omni-tts dense 0.5B pipeline (#2906)
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-06-10 20:11:04 +08:00
e1b0b83238 add moriio transfer engine (#1742)
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
Signed-off-by: zejwang  <zejwang@amd.com>
Signed-off-by: junkang1991 <junkangchow@gmail.com>
Signed-off-by: Zejian Wang <zejianwang@sjtu.edu.cn>
Co-authored-by: zejwang <zejwang@amd.com>
Co-authored-by: junkang1991 <junkangchow@gmail.com>
Co-authored-by: Zejian Wang <zejianwang@sjtu.edu.cn>
Co-authored-by: Zejian Wang <Zejian.Wang@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-10 08:37:33 +08:00
57227dc7f3 [Perf][Bugfix] qwen3-tts hot path: prefix-cache OOM guards + talker/orchestrator micro-opts (#3689)
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com>
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
2026-06-09 10:36:57 -07:00
ChenWenjing e25239c2ec [bugfix] fix realtime ci timeout error (#4187)
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
2026-06-06 00:05:07 +08:00
ZacheryAU bfc7cedcf1 Benchmark data statistics for each stage of omni models (#3628)
Signed-off-by: ZacheryAU <70869627+ZacheryAU@users.noreply.github.com>
2026-06-04 21:43:35 +08:00
amy-why-3459 8949fdc440 [BugFix] Fix the issue of vllm failing to start. (#4105)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
2026-06-03 15:44:46 +00:00
rein yangandHongsheng Liu 3a7c7f1422 [Bugfix] fix qwen3-omni performance regression (#3575)
Signed-off-by: rein yang <ruiruyang2@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-06-02 23:02:07 +08:00
Alex Brooks 12e75c6618 [Core] Integrate TrackingArgumentParser (#3369)
Signed-off-by: Alex Brooks <albrooks@redhat.com>
2026-06-02 21:56:24 +08:00
bc794e625f Add Cosmos3 model support (#3454)
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Signed-off-by: MaciejBalaNV <mbala@nvidia.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
2026-06-02 04:04:31 +08:00
2b7249fbb3 [Model]Support MiniCPM-o 4.5 (#3642)
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: GKangaroo <1095103651@qq.com>
Co-authored-by: GKangaroo <gqx24@mails.tsinghua.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-06-01 22:12:24 +08:00
Yueqian LinandHongsheng Liu c067485030 [Perf][TTS] Bounded-K active-stream window for Stage 1 (RFC #3535) (#3592)
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-06-01 21:59:10 +08:00
LHXuuuandvraiti 41a795b5d6 [Metrics] Add audio SLOs + cross-stage transfer families + per-(stage, replica) wrap for upstream vllm:* (#3576)
Signed-off-by: vraiti <vraiti@redhat.com>
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: vraiti <vraiti@redhat.com>
2026-06-01 20:20:48 +08:00
+1 f92d84fcf9 [Rebase] Rebase to vllm releases/v0.22.0 (#3891)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: nainiu258 <cperfect02@163.com>
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com>
Co-authored-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
2026-06-01 11:08:12 +08:00
d4c139508f [Feature] Add precomputed custom voices and Qwen3-TTS ref-context cache (#3492)
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Dan250124 <416947747@qq.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
2026-05-31 17:51:24 +00:00
ChenWenjing 5dfdf584dd [Feat]audio streaming input for async chunk (#3614)
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
2026-05-31 17:39:25 +08:00
b690a10e3a [Feat] Support MagCache (#1287)
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
2026-05-29 22:26:23 +08:00
c69ed4037d [feat] Support prompt embedding caching for diffusion model (#2962)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-05-29 15:36:03 +08:00
LancerandAlicia 5c3c81920b [Bugfix] Ensure stage and diffusion subprocesses exit when parent dies unexpectedly (#3751)
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
2026-05-28 08:22:04 +08:00
Hacoandzzhuoxin1508 37eebff674 [BugFix] Fix diffusion parallel_config YAML override and add deploy config field allowlist (#3483)
Signed-off-by: xiaohajiayou <923390377@qq.com>
Co-authored-by: zzhuoxin1508 <zzhuoxin1508@users.noreply.github.com>
2026-05-28 01:00:32 +08:00
a698269e47 [Model] Add Ming-flash-omni-2.0 Image Generation (Diffusion) Stage (#2875)
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com>
2026-05-27 23:21:20 +08:00
283873411a glm-image: fix(npu)per-stage runtime env for HCCL ports + GLM-Image NPU stage config (#3235)
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-05-27 12:18:47 +12:00
Yueqian LinandRein Yang e996723bc9 [Bugfix] Fix Qwen3-TTS Stage 0 prefix-caching correctness (#3665)
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>

the ci failure case is irrelavent to this PR and occurs rarely
2026-05-25 23:09:38 +08:00
BeatSeatandYueqian Lin 28ce618f0e [Model] Add GLM-TTS text-to-speech model support (#3141)
Signed-off-by: BeatSeat <wendavid552@gmail.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
2026-05-24 05:42:26 +08:00
ANHDYandwangmengdi 28467dc19a [feature]: support Hidream-I1-Full model (#2572)
Signed-off-by: wangmengdi <160612767@qq.com>
Co-authored-by: wangmengdi <wangmengdi@cmss.chinamobile.com>
2026-05-22 15:23:03 +08:00
bjf-frzandbjf-frz 5cf4605bf9 [Bugfix]Fix distributed stage0 multimodal cache routing (#3740)
Signed-off-by: bjf-frz <240110042+bjf-frz@users.noreply.github.com>
Co-authored-by: bjf-frz <240110042+bjf-frz@users.noreply.github.com>
2026-05-20 15:31:41 +08:00
Yueqian Lin fc8486cf20 [Doc][TTS] CosyVoice3 online docs + residual TTS yaml cleanup + remove VoxCPM v1 (#3748)
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
2026-05-20 15:24:06 +08:00
yangsonglin13 1c530d984d Yuanrong TransferEngine Connector for NPU (#3180)
Signed-off-by: yangsonglin13 <yangsonglin566@gmail.com>
2026-05-20 14:51:33 +08:00
chickeytonandherotai214 e277feacaf [large-scale-serving] Integrate OmniCoordinator into stage engine pipeline (#3569)
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: herotai214 <herotai214@gmail.com>
Co-authored-by: herotai214 <herotai214@gmail.com>
2026-05-19 12:08:20 +08:00
Rustam Khadipash 89f8819525 [Feature] Add support for Pipeline Parallel and integrate it into Wan 2.2 (#2322)
Signed-off-by: Rustam Khadipash <16683750+hadipash@users.noreply.github.com>
2026-05-19 09:42:17 +08:00
8cfc9179e6 Fix reasoning_parser crash: reconstruct StructuredOutputsConfig from dict (#2845)
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-05-17 23:30:24 +08:00
Yuanheng Zhao a64ebf103b [Refactor] Migrate and clean up TTS configs: CosyVoice3, OmniVoice, VoxCPM (#3338)
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
2026-05-17 07:32:34 +08:00
+20 eb4e60ee64 [Rebase] Rebase to vllm v0.21.0 (#3530)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Dnoob <dxpouo@gmail.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: rein yang <ruiruyang2@gmail.com>
Signed-off-by: Nick Cao <ncao@redhat.com>
Signed-off-by: zhumingjue <zhumingjue@huawei.com>
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com>
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: weizhoublue <weizhoublue@github.com>
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io>
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: NumberWan <wantszkin2003@gmail.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Dnoob <dxpouo@gmail.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com>
Co-authored-by: Ricardo Noriega <rnoriega@redhat.com>
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: weizhoublue <45163302+weizhoublue@users.noreply.github.com>
Co-authored-by: weizhoublue <weizhoublue@github.com>
Co-authored-by: dengyunyang <584797741@qq.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: NATURE <wzliu@connect.hku.hk>
Co-authored-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Y. Fisher <yukexiong1@huawei.com>
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-16 18:18:43 +08:00
bjf-frz 440c718d2a [Bugfix]Fix multimodal cache routing for AR replicas (#3605)
Signed-off-by: bjf-frz <frz123db@gmail.com>
2026-05-15 16:42:18 +08:00
lyj-jjjandCursor bbc00f9f86 [BugFix] fix(omni): isolate diffusion KV-cache dtype from vLLM --kv-cache-dtype #3585 (#3596)
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-15 11:22:59 +08:00
wuhang 0d9d57acd9 [Entrypoint][Refactor] Make field type hint more concrete (#3139)
Signed-off-by: wuhang <wuhang6@huawei.com>
2026-05-14 11:25:55 +08:00
Y. Fisher 4d3eed152a [Feat][Config] Support additional_config for diffusion worker (#3020)
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Signed-off-by: Y. Fisher <yukexiong1@huawei.com>
2026-05-13 14:22:43 +00:00
knlnguyen1802andSamit 631251a1f8 [Bugfix, rl] Diffusion worker SIGKILL under Ray actor (exitcode -9) (#3533)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
2026-05-13 15:31:48 +08:00