anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,144 Commits 235 Branches 31 Tags
df8137e858233685407ec266b3a6c8b030d99f30
Commit Graph
216 Commits
Author SHA1 Message Date
Yueqian Lin ede3dddfdf fix(serving): scope no-async-chunk FINAL_ONLY streaming to qwen3_tts 
async_chunk defaults to False in OmniModelConfig, so coercing every
async_chunk=False deploy to FINAL_ONLY for streaming requests would
silently disable incremental delivery for non-Qwen3 models that stream
DELTA-style today. Gate the coercion on _tts_model_type == "qwen3_tts"
(the model this PR validated) and pin the non-qwen3 behavior with a test.

Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-07-03 22:57:27 -07:00
Gao Han fc40c1b236 Merge branch 'main' into bugfix/qwen3-tts-customvoice-asr-escalation 2026-07-04 13:26:03 +08:00
Sy03 c523026a8e Fix Qwen3-TTS no-async-chunk payload splitting 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-07-04 12:35:08 +08:00
Kotaro Kikuchi 46f10478dc [Bugfix] Qwen-Image-Layered: allow layers=2 to match upstream demo (#4777) 
Signed-off-by: Kotaro Kikuchi <ktrk115@gmail.com>
 2026-07-04 03:06:24 +00:00
Alex Brooks 5c51782c6b [Bugfix] Realign Serving Tokenization Name to Fix Error Handling Routes (#4816) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-07-02 15:23:35 +08:00
Zhou Taichang a560ed184d [Rebase] Rebase to vllm v0.24.0 (#4709) 2026-06-30 19:41:06 +08:00
faceless void 11f6a0b72e Make speech streaming default to SSE (#4679) 
Signed-off-by: syd520zy <529477025@qq.com>
 2026-06-30 02:17:24 +00:00
JiangJie ZhangandSamit c45ac74d43 [Core][Frontend] Support request-level batching for diffusion pipelines (#4079) 
Signed-off-by: jader <yjader@foxmail.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-06-29 18:09:19 +02:00
wangyuandHongsheng Liu 7619ed6b82 [CI] Reducing mithril-h100-pool in ready&merge CI, optimize ready CI duration (#4354) 
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: [Your Name] <[Your Email]>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-29 05:17:57 +00:00
Yueqian Lin 05a86ed10e [Test] Un-skip Qwen3-TTS batch E2E; match documented omit-null response shape (#4757) (#4759) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-06-26 23:31:44 -07:00
wangyu c6ee3f7c18 [Test] Skip Qwen3-TTS batch E2E tests (#4758) 2026-06-27 10:45:40 +08:00
Yueqian LinandSy03 b66cd024b9 [Bugfix] Fix /v1/audio/speech usage token accounting for Qwen3-TTS (#4646) (#4673) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Sy03 <1370724210@qq.com>
 2026-06-27 06:13:31 +08:00
Gao Han c582805e7a Revert "[Bugfix] Use MediaConnector for image/video URL fetching to prevent SSRF" (#4751) 2026-06-27 01:18:48 +08:00
Sy03 c39ac9fffd [Bugfix] Stream Qwen3-TTS WebSocket input as one request (#4731) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-06-26 07:48:30 +00:00
WeiQing ChenandIanniu d94108693a [BugFix] Fix Wan2.2-VACE-Fun cache and serving lifecycle (#4667) 
Signed-off-by: Ianniu <985683179yzy@gmail.com>
Signed-off-by: david6666666 <530634352@qq.com>
Co-authored-by: Ianniu <985683179yzy@gmail.com>
 2026-06-25 11:20:58 +08:00
Nick CaoandHongsheng Liu d8edf0c9d5 [Bugfix] Use MediaConnector for image/video URL fetching to prevent SSRF (#2565) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-25 06:28:26 +08:00
Alex Brooks 465f1464d2 [Feat] Support Pipeline Resolution Based on HF Config & Make Pipeline Registry Eager (#3760) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-06-24 11:02:29 +08:00
rongfu.lengandGao Han a2d1dafc65 [Feature] add a new response_format is file type when image generate (#1673) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-06-24 09:04:13 +08:00
Zeyu Huang | 黃澤宇Copilot Autofix powered by AICursor
 		3901ad8165 [feat] Streaming diffusion video generation output (#3737) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-23 09:57:53 +00:00
Ian EavesYueqian LinYueqian Lin
 		50bc51ffdf fix: return proper HTTP status codes for audio voice endpoints (#3969) 
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-06-22 22:57:07 +00:00
+2
faceless voidNick CaoChendi.XueHongsheng LiuKristofferJunhong LiulishunyangCopilot Autofix powered by AIHyoseop SongZhou TaichangAliciaamy-why-3459
 		ec4ec0f604 Add SSE stream_format support for audio speech streaming (#4490) 
Signed-off-by: syd520zy <529477025@qq.com>
Signed-off-by: faceless void <100186390+syd520zy@users.noreply.github.com>
Signed-off-by: Nick Cao <ncao@redhat.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: kTorp <Kristoffer.Torp@amd.com>
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hyoseop Song <crad_on25@naver.com>
Signed-off-by: Hyoseop Song  <crad_on25@naver.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Kristoffer <kristoffer.torp@amd.com>
Co-authored-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Hyoseop Song <crad_on25@naver.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-06-22 06:42:25 +00:00
王金旭CursorClaude Opus 4.8
 		ad07f98b73 [Feat] Streaming TTS word-level timestamps via forced aligner (#4034) 
Signed-off-by: wjinxu <1299461899@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-22 04:51:45 +00:00
rein yang afbcc51672 [Model] Aura support - Non async chunk path (#4257) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
 2026-06-21 22:08:26 +08:00
MaciejBalaNV a1e1ac0155 Cosmos3 transfer (#4379) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Signed-off-by: MaciejBalaNV <mbala@nvidia.com>
 2026-06-21 09:59:18 +08:00
Ian Eaves 951ae28eb0 [Frontend] Configurable video storage backends with TTL (#2531) 
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com>
 2026-06-19 17:41:22 -06:00
amy-why-3459 f05e03aea8 [Perf][DreamZero]dreamzero supports CUDA graph, torch.compile, and DiT Caching. (#4213) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-06-19 14:24:37 +08:00
NumberWanandlishunyang12 d744940b18 Refactor: extract OmniStreamingVideoHandler base and QwenOmniStreamingVideoHandler (#4424) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Signed-off-by: lishunyang12 <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: lishunyang12 <125541396+lishunyang12@users.noreply.github.com>
 2026-06-17 15:07:45 +00:00
Peiqi YinHongsheng LiuChenguang ZHENG
 		64a5bb92de [Refactor] Omni Stage Runtime and Distributed Replica Control Plane (#3855) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Chenguang ZHENG <645327136@qq.com>
 2026-06-17 14:13:20 +08:00
Junhong LiulishunyangHongsheng LiuCopilot Autofix powered by AI
 		f200e988a6 [fix bug] add default limit for image generation (#3381) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-06-16 22:54:26 +08:00
Yuzhu Dong e2e79177db Support DROID policy server for Cosmos3 OpenPI (#4282) 
Signed-off-by: yuzhud <yuzhudong@gmail.com>
 2026-06-15 21:22:59 +08:00
Yueqian Lin d75a6433b4 [Refactor] TTS serving adapter framework + migrate TTS models (excl. ming_flash_omni) (#4330) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-06-13 04:22:54 +00:00
hurukawaYueqian LinSy03
 		152343d5fb Feat: non_streaming_mode for Qwen3-TTS Base Models During Online Inference (#4198) 
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Sy03 <1370724210@qq.com>
 2026-06-11 22:15:05 +00:00
MaciejBalaNVandHongsheng Liu 03cc888485 [refactor] Refactor guardrail error handling - add 400 error code (#4297) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-11 21:09:24 +00:00
Chendi.XueandHongsheng Liu 5414f78e5f [WAN2.2-S2V] Add server API for image + audio (#3394) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-11 18:50:03 +08:00
Honghan ZhuClaude Opus 4.8zhuhonghan
 		b8f68174ad [Refactor] Remove dead legacy pipeline.yaml loader and duplicate diff… (#4023) 
Signed-off-by: dph97 <nenbaying@163.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: zhuhonghan <z00806815@china.huawei.com>
 2026-06-09 07:34:35 +08:00
MaciejBalaNV 2f25195750 Cosmos3 video to video generation (#4266) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
 2026-06-09 03:04:05 +08:00
Lokesh 176b00538c [Model] Support languages added by fine-tuned Qwen3-TTS checkpoints (#4210) 
Signed-off-by: lokesh <lokesh.v9911@gmail.com>
 2026-06-07 10:10:40 -06:00
Yaegaki1Erikaandreandeng 84569280b1 [Bugfix]Fix HunyuanImage3 conditional image and prompt kwargs alignment (#4145) 
Signed-off-by: reandeng <reandeng@tencent.com>
Signed-off-by: Yaegaki1Erika <70182590+Yaegaki1Erika@users.noreply.github.com>
Co-authored-by: reandeng <reandeng@tencent.com>
 2026-06-05 12:47:25 +08:00
Zheng Wengang fccd95b12b [BugFix] Fix incorrect GPU device mapping in multi-replica stages (#4132) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
 2026-06-04 15:32:58 +00:00
ZacheryAU bfc7cedcf1 Benchmark data statistics for each stage of omni models (#3628) 
Signed-off-by: ZacheryAU <70869627+ZacheryAU@users.noreply.github.com>
 2026-06-04 21:43:35 +08:00
bastefaniakMaciej Balalishunyang12SYLAR
 		706bad253d Add Cosmos3 action modality (#4102) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Signed-off-by: Bartosz Stefaniak <bstefaniak@nvidia.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Maciej Bala <mbala@nvidia.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-06-03 19:30:39 +00:00
ChenWenjing 2e108a4109 [ci] add merge/ready ci for audio realtime api (#4069) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-06-03 18:57:16 +08:00
MaciejBalaNVBartosz Stefaniaklishunyang12
 		40b2959183 Add Cosmos3 sound generation (#4073) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: Bartosz Stefaniak <bstefaniak@nvidia.com>
Co-authored-by: Bartosz Stefaniak <bstefaniak@nvidia.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
 2026-06-03 00:51:44 +00:00
Yangshen DengMengSuli Wangamy-why-3459
 		bdfc77147d [Diffusion] DreamZero world model integration with CFG parallel + OpenPI serving (#2162) 
Signed-off-by: Meng <meng_chen99@163.com>
Signed-off-by: Yangshen Deng <yangshen.d@outlook.com>
Signed-off-by: Suli Wang <50828270+wsl2000@users.noreply.github.com>
Co-authored-by: Meng <meng_chen99@163.com>
Co-authored-by: Suli Wang <50828270+wsl2000@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-06-02 19:50:20 +00:00
Alex Brooks 12e75c6618 [Core] Integrate TrackingArgumentParser (#3369) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-06-02 21:56:24 +08:00
MaciejBalaNVSYLARlishunyang12
 		bc794e625f Add Cosmos3 model support (#3454) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Signed-off-by: MaciejBalaNV <mbala@nvidia.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
 2026-06-02 04:04:31 +08:00
Mike QiuMike_QiuClaude Opus 4.7Hongsheng Liu
 		8d83fb59e9 [BugFix] Fix two stop reason for multimodal output (#3374) 
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
Signed-off-by: Mike Qiu <qdy220091330@gmail.com>
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-01 21:51:29 +08:00
Yangshen DengMengamy-why-3459
 		634af60fba [Entrypoint] Add realtime OpenPI robot serving API (#3673) 
Signed-off-by: Yangshen Deng <yangshen.d@outlook.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Meng <meng_chen99@163.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-06-01 14:47:23 +08:00
+1
Zhou TaichangClaude Opus 4.7skfZheng WengangHongsheng LiuWeiQing ChenSYLARYueqian Linnainiu258JasonYueqian Lin
 		f92d84fcf9 [Rebase] Rebase to vllm releases/v0.22.0 (#3891) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: nainiu258 <cperfect02@163.com>
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com>
Co-authored-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-06-01 11:08:12 +08:00
Sy03Dan250124Yueqian LinYueqian Lin
 		d4c139508f [Feature] Add precomputed custom voices and Qwen3-TTS ref-context cache (#3492) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Dan250124 <416947747@qq.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-05-31 17:51:24 +00:00