anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,829 Commits 235 Branches 31 Tags
69fd466fba56b9dcdcd1e76d9b7986ebf37fb3fa
Commit Graph
514 Commits
Author SHA1 Message Date
Gao Han 69fd466fba Merge branch 'main' into feat/diffusion-prefetch-protection 2026-06-02 23:22:08 +08:00
NumberWan 35ee3c7543 [BugFix]Qwen-Image performance regression by using torch RMSNorm(RMSNorm backend) (#4074) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-06-02 23:19:17 +08:00
lsyyyyy bd37f3c27d Support VAE parallel for Bagel (#3982) 
Signed-off-by: siyuan.lei <siyuanlei37@gmail.com>
 2026-06-02 13:14:01 +00:00
Boopathy Kannappan 7c729e19d7 [Bugfix] Fix SD3 T5 truncation check device mismatch on long prompts (#3949) 
Signed-off-by: bkannappan <bkannappan@digitalocean.com>
 2026-06-02 11:04:08 +00:00
tzhouam cb08af976b fix: correct logging message format in hub_prefetch.py for better clarity during prefetch attempts 2026-06-02 09:01:47 +00:00
tzhouam 617c6ca55f feat: enhance model loading with prefetching and retry logic 
- Introduced `from_pretrained_with_prefetch` to handle racy cache scenarios by re-prefetching and retrying on failures.
- Updated various model pipelines to utilize the new prefetching mechanism, ensuring robust loading of model components.
- Prefetch logic added to multiple models, including HunyuanVideo, LongCatImage, and QwenImage, to mitigate issues with incomplete cache states.
 2026-06-02 09:01:47 +00:00
NATURE 1fb423e5f6 [Perf][Bagel] Avoid per-step device syncs in Bagel img2img (#3987) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-06-02 10:58:32 +08:00
MaciejBalaNVSYLARlishunyang12
 		bc794e625f Add Cosmos3 model support (#3454) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Signed-off-by: MaciejBalaNV <mbala@nvidia.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
 2026-06-02 04:04:31 +08:00
TJian 29343540f7 [ROCm] [CI] Bugfix Existing CI cases (#3946) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-06-01 22:02:15 +08:00
SYLARlishunyangHaco
 		b76291d6a1 [New Model] Add Lance (ByteDance) (#3710) 
Signed-off-by: lishunyang <lishunyang03@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com>
Co-authored-by: lishunyang <lishunyang03@gmail.com>
Co-authored-by: Haco <923390377@qq.com>
 2026-06-01 16:01:25 +08:00
boatmanandsphinxkkkbc 2dcfcc0519 [Feat]Support Nonverbal Tags in OmniVoice (#3968) 
Signed-off-by: sphinxkkkbc <binchengkang8@gmail.com>
Co-authored-by: sphinxkkkbc <binchengkang8@gmail.com>
 2026-06-01 15:51:05 +08:00
Yi LiuandCopilot ef71d74c81 [Bugfix] Fix FLUX W4A16/AutoRound quant_config propagation (#3587) 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-01 04:06:02 +00:00
+1
Zhou TaichangClaude Opus 4.7skfZheng WengangHongsheng LiuWeiQing ChenSYLARYueqian Linnainiu258JasonYueqian Lin
 		f92d84fcf9 [Rebase] Rebase to vllm releases/v0.22.0 (#3891) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: nainiu258 <cperfect02@163.com>
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com>
Co-authored-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-06-01 11:08:12 +08:00
NumberWan 6f4bd3e45c [BugFix]Qwen-Image performance regression by using omni RMSNorm(RMSNorm backend) (#3933) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-05-31 08:06:18 +08:00
zzh ffd9771c54 Fix hunyuan resolve stop token ids (#3896) 
Signed-off-by: zzh <943967662@qq.com>
 2026-05-31 07:47:36 +08:00
wuhang b0f3672aec [Bugfix] Fix DiffusionWorker crash on SIGINT: ensure NCCL/ZMQ cleanup on shutdown (#3872) 
Signed-off-by: WU Hang <whlbx@hotmail.com>
 2026-05-31 00:13:23 +08:00
WeiQing Chen a3456752c5 [Diffusion] Support USP and VAE patch parallel for HunyuanVideo 1.5 (#3979) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-05-30 14:40:42 +08:00
fywc 272e626100 [Feat] support cache-dit for SenseNova-U1 (#3906) 
Signed-off-by: hanzheli <hanzheli@kuaishou.com>
 2026-05-29 17:40:54 +00:00
knlnguyen1802andSamit 5377108438 [minor, fix] Allow passing class interface as custom pipeline argument (#2973) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-05-29 23:39:38 +08:00
Lancer汪志鹏Gao Han
 		b690a10e3a [Feat] Support MagCache (#1287) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-05-29 22:26:23 +08:00
Aleksi VesantoandHongsheng Liu eba5617c95 [ROCm] Enable AITER backend with ring attention (#3511) 
Signed-off-by: Aleksi Vesanto <avesanto@amd.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-29 22:20:29 +08:00
Huanyu Yang 3457758114 [Bugfix] Fix IndexError in DistributedVaeExecutor when vae_patch_parallel_size < world_size (#3928) 
Signed-off-by: QingZhou-YangHY <3868850350@qq.com>
 2026-05-29 03:17:34 -07:00
WeiQing Chen 67bb9b4a12 [CI][Bugfix] Fix LTX audio-video warmup output typing (#3964) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-05-29 07:46:02 +00:00
boatmanandsphinxkkkbc 4d9262a6eb [Feat]Support voice clone for omnivoice in online serving & add seed parameter for reproducible (#3668) 
Signed-off-by: sphinxkkkbc <binchengkang8@gmail.com>
Co-authored-by: sphinxkkkbc <binchengkang8@gmail.com>
 2026-05-29 07:42:08 +00:00
knlnguyen1802SamitHongsheng Liu
 		c69ed4037d [feat] Support prompt embedding caching for diffusion model (#2962) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-29 15:36:03 +08:00
Alex Brooks 68a1d23e4d [Bugfix/Feature] Remove Hardcoded Flash Attention in Bagel & Support GQA in SDPA Backend (#3728) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-05-29 10:36:15 +08:00
knlnguyen1802andSamit 5eac50f22c [bugfix, rl] Fix sleep do not release full memory in custom pipeline (#3818) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-05-28 22:27:27 +08:00
WeiQing ChenlishunyangGao Han
 		126fb4b4a9 [Blackwell][1/N] Add SageAttention3 diffusion backend on blackwell(GB200/B200/RTX5090/PRO6000/DGX Spark available) (#3015) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-05-28 14:08:45 +00:00
Liang Lv b72ffb82bb [AutoRound] Support GLM-Image W4A16 quantization model (#3059) 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
 2026-05-28 07:19:06 +00:00
JiangJie Zhang 6fa9dbbe42 [Diffusion] Unify diffusion request identity on request_id (#3744) 
Signed-off-by: jader <yjader@foxmail.com>
 2026-05-28 14:56:56 +08:00
mglyn 2b999429c4 [Feature] Add cfg-parallel for LTX-2.3 (#3905) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-05-28 04:36:33 +00:00
WeiQing Chen 6031c7d393 [CI] Add HunyuanVideo 1.5 X2V accuracy tests (#3852) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-05-28 11:09:45 +08:00
SYLARfrauttauteffasuZhang Jian
 		373d3c7476 [Blackwell] Add CUDNN_ATTN and FLASHINFER_ATTN backends for diffusion (auto-route) (#3079) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Zhang <jianmusings@gmail.com>
Signed-off-by: Zhang Jian <jianmusings@gmail.com>
Co-authored-by: frauttauteffasu <247219357+frauttauteffasu@users.noreply.github.com>
Co-authored-by: Zhang Jian <jianmusings@gmail.com>
 2026-05-28 11:09:25 +08:00
Chendi.Xue 1903cee56a [MXFP8][XPU] enable mxfp8 using vLLM main repo method (#3782) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-05-28 02:47:22 +00:00
Yuanheng Zhao 39705f9ee8 [Feat] Support online quantization (fp8/int8) for DreamID-Omni (#3902) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-05-28 02:31:58 +00:00
LancerandAlicia 5c3c81920b [Bugfix] Ensure stage and diffusion subprocesses exit when parent dies unexpectedly (#3751) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-05-28 08:22:04 +08:00
akshatvishu 1f3a3e3848 Fix Ovis image text encoder dtype (#3876) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-05-28 08:21:37 +08:00
Zheng WengangClaude Opus 4.6Yuanheng Zhao
 		a698269e47 [Model] Add Ming-flash-omni-2.0 Image Generation (Diffusion) Stage (#2875) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com>
 2026-05-27 23:21:20 +08:00
JohnJan 36dc32ff81 [Feature] support SP for FLUX.2-dev (#3244) 2026-05-27 20:43:02 +08:00
bjf-frzandHongsheng Liu c33ef34654 [Bugfix]Fix Hunyuan Image3 denoise flow alignment (#3857) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-27 19:18:01 +08:00
rongfu.lengandPeter Pan c349f62839 [Feat] opt qwen image model load use ColumnParallelLinear replace ReplicatedLinear (#3875) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: Peter Pan <Peter.Pan@daocloud.io>
 2026-05-27 09:02:17 +00:00
WeiQing ChenandSYLAR 41cb6378a2 [Perf] Trim HunyuanVideo encoder padding tokens (#3844) 
Signed-off-by: david6666666 <530634352@qq.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-05-27 14:17:10 +08:00
JasonandZeyu Huang | 黃澤宇 753508fe02 [Bugfix] Fix Qwen-Image-Edit-2511 TeaCache zero_cond_t handling (#3219) 
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn>
Signed-off-by: Jason <72191212+JasonJ2021@users.noreply.github.com>
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
 2026-05-27 11:06:18 +08:00
mglyn 385df0359e [BugFix] Fix LTX-2.3 audio latent padding for sequence parallelism (#3854) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-05-26 08:45:38 +00:00
NumberWan 11a951b962 [BugFix]: CUDA device-side assert failures on single-stage BAGEL i2i requests (#3680) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-05-26 03:01:10 +00:00
Alex Brooks cc0e15ba57 [Bugfix] Fix LTX2 CacheDiT Integration (#3621) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-05-24 21:55:34 +08:00
fywc d01eaca39a [Feat] support cache-dit for DreamID-Omni (#3265) 
Signed-off-by: hanzheli <hanzheli@kuaishou.com>
Signed-off-by: fywc <hanzheli@kuaishou.com>
 2026-05-23 08:09:33 +08:00
Alex Brooks 23e243378f [Bugfix] Set separate CFG flag in Helios for CacheDiT (#3756) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-05-22 19:56:41 +08:00
ANHDYandwangmengdi 28467dc19a [feature]: support Hidream-I1-Full model (#2572) 
Signed-off-by: wangmengdi <160612767@qq.com>
Co-authored-by: wangmengdi <wangmengdi@cmss.chinamobile.com>
 2026-05-22 15:23:03 +08:00
Samit 8e937400dc Fix OmniDiffusionConfig master_port selection for parallel launches (#3803) 2026-05-22 11:12:29 +08:00