anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,829 Commits 235 Branches 31 Tags
69fd466fba56b9dcdcd1e76d9b7986ebf37fb3fa
Commit Graph
1829 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Gao Han 69fd466fba Merge branch 'main' into feat/diffusion-prefetch-protection 2026-06-02 23:22:08 +08:00
NumberWan 35ee3c7543 [BugFix]Qwen-Image performance regression by using torch RMSNorm(RMSNorm backend) (#4074) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-06-02 23:19:17 +08:00
rein yangandHongsheng Liu 3a7c7f1422 [Bugfix] fix qwen3-omni performance regression (#3575) 
Signed-off-by: rein yang <ruiruyang2@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-02 23:02:07 +08:00
Alex Brooks 12e75c6618 [Core] Integrate TrackingArgumentParser (#3369) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-06-02 21:56:24 +08:00
lsyyyyy bd37f3c27d Support VAE parallel for Bagel (#3982) 
Signed-off-by: siyuan.lei <siyuanlei37@gmail.com>
 2026-06-02 13:14:01 +00:00
Boopathy Kannappan 7c729e19d7 [Bugfix] Fix SD3 T5 truncation check device mismatch on long prompts (#3949) 
Signed-off-by: bkannappan <bkannappan@digitalocean.com>
 2026-06-02 11:04:08 +00:00
tzhouam cb08af976b fix: correct logging message format in hub_prefetch.py for better clarity during prefetch attempts 2026-06-02 09:01:47 +00:00
tzhouam 617c6ca55f feat: enhance model loading with prefetching and retry logic 
- Introduced `from_pretrained_with_prefetch` to handle racy cache scenarios by re-prefetching and retrying on failures.
- Updated various model pipelines to utilize the new prefetching mechanism, ensuring robust loading of model components.
- Prefetch logic added to multiple models, including HunyuanVideo, LongCatImage, and QwenImage, to mitigate issues with incomplete cache states.
 2026-06-02 09:01:47 +00:00
Zhang JianandCanlin Guo b550709bba [TTS][Model] support MOSS-TTS series (#3420) 
Signed-off-by: Zhang <jianmusings@gmail.com>
Signed-off-by: Zhang Jian <jianmusings@gmail.com>
Signed-off-by: jian <jianmusings@gmail.com>
Signed-off-by: Canlin Guo <961750412@qq.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
 2026-06-02 15:46:01 +08:00
tc-mb da9c45e1de add MiniCPM-o 4.5 recipe under recipes/OpenBMB (#4067) 
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
 2026-06-02 14:33:46 +08:00
NATURE 1fb423e5f6 [Perf][Bagel] Avoid per-step device syncs in Bagel img2img (#3987) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-06-02 10:58:32 +08:00
Chendi.Xue 2a2033df02 [XPU][Rebase v0.22] Fix for 0.22 rebase (#4059) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-06-02 10:16:44 +08:00
MaciejBalaNVSYLARlishunyang12
 		bc794e625f Add Cosmos3 model support (#3454) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Signed-off-by: MaciejBalaNV <mbala@nvidia.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
 2026-06-02 04:04:31 +08:00
tc-mbGKangarooGKangarooHongsheng Liu
 		2b7249fbb3 [Model]Support MiniCPM-o 4.5 (#3642) 
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: GKangaroo <1095103651@qq.com>
Co-authored-by: GKangaroo <gqx24@mails.tsinghua.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-01 22:12:24 +08:00
TJian 29343540f7 [ROCm] [CI] Bugfix Existing CI cases (#3946) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-06-01 22:02:15 +08:00
Yueqian LinandHongsheng Liu c067485030 [Perf][TTS] Bounded-K active-stream window for Stage 1 (RFC #3535) (#3592) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-01 21:59:10 +08:00
Mike QiuMike_QiuClaude Opus 4.7Hongsheng Liu
 		8d83fb59e9 [BugFix] Fix two stop reason for multimodal output (#3374) 
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
Signed-off-by: Mike Qiu <qdy220091330@gmail.com>
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-01 21:51:29 +08:00
LHXuuuandvraiti 41a795b5d6 [Metrics] Add audio SLOs + cross-stage transfer families + per-(stage, replica) wrap for upstream vllm:* (#3576) 
Signed-off-by: vraiti <vraiti@redhat.com>
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: vraiti <vraiti@redhat.com>
 2026-06-01 20:20:48 +08:00
TaffyOfficialandTaffyOfficial ee33601571 [Perf] Deduplicate AR prefix cache hidden-state CPU staging (#3734) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
 2026-06-01 09:35:07 +00:00
SYLARlishunyangHaco
 		b76291d6a1 [New Model] Add Lance (ByteDance) (#3710) 
Signed-off-by: lishunyang <lishunyang03@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com>
Co-authored-by: lishunyang <lishunyang03@gmail.com>
Co-authored-by: Haco <923390377@qq.com>
 2026-06-01 16:01:25 +08:00
boatmanandsphinxkkkbc 2dcfcc0519 [Feat]Support Nonverbal Tags in OmniVoice (#3968) 
Signed-off-by: sphinxkkkbc <binchengkang8@gmail.com>
Co-authored-by: sphinxkkkbc <binchengkang8@gmail.com>
 2026-06-01 15:51:05 +08:00
Yangshen DengMengamy-why-3459
 		634af60fba [Entrypoint] Add realtime OpenPI robot serving API (#3673) 
Signed-off-by: Yangshen Deng <yangshen.d@outlook.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Meng <meng_chen99@163.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-06-01 14:47:23 +08:00
王金旭andCursor ac4106900b [skip ci][Recipe] OpenBMB/VoxCPM2 (#3850) 
Signed-off-by: wjinxu <1299461899@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-01 14:39:54 +08:00
rongfu.lengtzhouamClaude Opus 4.8
 		9afdab4939 [Feat]upgrade vllm version [skip-ci] (#4022) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-01 12:08:10 +08:00
Yi LiuandCopilot ef71d74c81 [Bugfix] Fix FLUX W4A16/AutoRound quant_config propagation (#3587) 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-01 04:06:02 +00:00
+1
Zhou TaichangClaude Opus 4.7skfZheng WengangHongsheng LiuWeiQing ChenSYLARYueqian Linnainiu258JasonYueqian Lin
 		f92d84fcf9 [Rebase] Rebase to vllm releases/v0.22.0 (#3891) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: nainiu258 <cperfect02@163.com>
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com>
Co-authored-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
v0.22.0rc1 		2026-06-01 11:08:12 +08:00
Sy03Dan250124Yueqian LinYueqian Lin
 		d4c139508f [Feature] Add precomputed custom voices and Qwen3-TTS ref-context cache (#3492) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Dan250124 <416947747@qq.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-05-31 17:51:24 +00:00
Qing AnandSamit 57e5f0fe69 [model, omni] feat: Qwen3-Omni Thinker LoRA for RL training (#3915) 
Signed-off-by: Qing An <qa4@rice.edu>
Co-authored-by: Samit <285365963@qq.com>
 2026-06-01 00:10:14 +08:00
ChenWenjing 5dfdf584dd [Feat]audio streaming input for async chunk (#3614) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-05-31 17:39:25 +08:00
NumberWan 6f4bd3e45c [BugFix]Qwen-Image performance regression by using omni RMSNorm(RMSNorm backend) (#3933) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-05-31 08:06:18 +08:00
Canlin Guo 412bf42a75 [Refactor] Unify _talker_mtp_forward across GPU and NPU model runners (#3476) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-05-31 07:58:00 +08:00
zzh ffd9771c54 Fix hunyuan resolve stop token ids (#3896) 
Signed-off-by: zzh <943967662@qq.com>
 2026-05-31 07:47:36 +08:00
DMAN 11c4fcedd3 [Recipe] add mistralai voxtral tts recipe (#3498) 
Signed-off-by: Dmaner <2663515256@qq.com>
 2026-05-31 00:18:52 +08:00
wuhang b0f3672aec [Bugfix] Fix DiffusionWorker crash on SIGINT: ensure NCCL/ZMQ cleanup on shutdown (#3872) 
Signed-off-by: WU Hang <whlbx@hotmail.com>
 2026-05-31 00:13:23 +08:00
WeiQing Chen 2289478e3a [CI] Temporarily skip failing Bagel connector tests (#4005) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-05-30 17:46:29 +08:00
Alex BrooksandHongsheng Liu 8d13b54383 [CI/Bugfix] Async Request ID Aliasing (#3953) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-30 16:59:42 +08:00
wuhangandAlicia 802a6242f0 [Bugfix][Tests] Remove unnecessary device map in tests init (#3958) 
Signed-off-by: WU Hang <whlbx@hotmail.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-05-30 16:18:48 +08:00
Yuanheng ZhaoandZheng Wengang ea6d811b58 [Config Refactor] Migrate Ming-flash-omni-2.0 Image-Gen deploy configs (#3975) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Co-authored-by: Zheng Wengang <zwg0606@gmail.com>
 2026-05-30 08:13:21 +00:00
WeiQing Chen 5994a0bbf5 docs: update WeChat QR code (#4003) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-05-30 16:12:11 +08:00
Viacheslav KlimkovandYueqian Lin 914e92cf80 [Refactor][Qwen3-TTS] Extract reusable prompt-embeds builder and make tts_pad_embed a persistent buffer (#3992) 
Signed-off-by: Viacheslav Klimkov <vklimkov@nvidia.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-05-30 16:11:00 +08:00
dengyunyang 3b317285c4 [HunyuanImage][Perf] adapt to deploy config changes (#3996) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-05-30 14:48:50 +08:00
WeiQing Chen a3456752c5 [Diffusion] Support USP and VAE patch parallel for HunyuanVideo 1.5 (#3979) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-05-30 14:40:42 +08:00
Chendi.Xue f43bb124bc [CI][XPU]Fix sage_attn hard-code import for cuda (#3994) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-05-30 09:43:14 +08:00
fywc 272e626100 [Feat] support cache-dit for SenseNova-U1 (#3906) 
Signed-off-by: hanzheli <hanzheli@kuaishou.com>
 2026-05-29 17:40:54 +00:00
knlnguyen1802andSamit 5377108438 [minor, fix] Allow passing class interface as custom pipeline argument (#2973) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-05-29 23:39:38 +08:00
nperraud 9fdb3bcb89 [Bugfix][Model] Qwen3-TTS: don't collapse 2D ref_code list when estimating prompt length (#3940) 
Signed-off-by: nperraud <nathanael.perraudin@gmail.com>
 2026-05-29 09:18:25 -06:00
Chen ChengandClaude Opus 4.7 f1c2caa3c7 [Perf][Qwen3-TTS] Restore Code2Wav cross-request batching (RFC #3163 P0) (#3322) 
Signed-off-by: ischencheng <cheng21@seas.upenn.edu>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-29 08:48:58 -06:00
Lancer汪志鹏Gao Han
 		b690a10e3a [Feat] Support MagCache (#1287) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-05-29 22:26:23 +08:00
Aleksi VesantoandHongsheng Liu eba5617c95 [ROCm] Enable AITER backend with ring attention (#3511) 
Signed-off-by: Aleksi Vesanto <avesanto@amd.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-29 22:20:29 +08:00
amy-why-3459andHongsheng Liu 2696751003 [Perf] Qwen3-Omni performance optimization (#3878) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-29 22:15:23 +08:00