anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,582 Commits 235 Branches 31 Tags
f1abb13958b35bd5615c7cfef19f91e9a90926eb
Commit Graph
105 Commits
Author SHA1 Message Date
Hongsheng Liu f1abb13958 [Diffusion] Register runtime cache for direct DLO H2D 
Avoid recurrent mmap-to-pinned packing in no-AllGather DLO by registering transform-complete runtime-cache mappings under an explicit per-worker budget. Copy mapped tensors directly into the rotating device buffers while preserving bounded-staging fallback and ordered shutdown cleanup.

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-17 11:50:32 +08:00
Hongsheng Liu b732ad2cea [Diffusion] Document runtime cache hardware tradeoffs 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-17 00:43:06 +08:00
Hongsheng Liu 99347e1ba7 [Diffusion] Tighten runtime cache compatibility guards 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-16 23:26:25 +08:00
Hongsheng Liu db86a6a078 [Diffusion] Add node-local runtime weight cache for DLO 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-16 23:15:48 +08:00
Hongsheng Liu 9d2bb23ff6 [Diffusion] Add loader-owned host-weight plans for DLO (TP=1) (#6213) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-16 11:02:38 +08:00
Hongsheng Liu 1260ceaae6 [Docs] Split diffusion attention and CPU offload guides (#6075) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-15 14:41:09 +08:00
baonudesifeizhai b300f66656 [Bugfix] Fix HSDP compatibility with the new online FP8 linear method (#5677) 
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
 2026-08-14 15:45:55 +08:00
Hongsheng Liu 1ff2753469 docs: align User Guide feature taxonomy (#6045) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-12 00:20:56 +08:00
Jerry Song 89015d5cad [Feature]: Adds distilled LoRA support for diffusion models (#2783) 
Signed-off-by: Songrui625 <songrui625@gmail.com>
 2026-08-11 14:23:43 +00:00
KristofferDidan DengHongsheng Liu
 		4ab1b4ff9c [Perf][Flux2][HunyuanVideo1.5] Skip attention-mask to avoid varlen path (#4645) 
Signed-off-by: kTorp <Kristoffer.Torp@amd.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-11 15:31:02 +08:00
fan2956andFan c27623c29c [Bugfix] rainfusion support all shape video (#6000) 
Signed-off-by: Fan <fan@FandeMacBook-Air.local>
Co-authored-by: Fan <fan@FandeMacBook-Air.local>
 2026-08-11 05:10:50 +08:00
Mu GuanLin d8c35663b4 [Model][Feat] Support Minimax-H3 quality grading requests through dynamic loading/unloading with Cache-DiT (#5853) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-08-07 09:07:23 +00:00
RuQing Xu ab459b36c2 [Kernel] Refresh FlashInfer attention; Add quantized attention support (Blackwell QK16/V8) (#5344) 
Signed-off-by: Ruqing Xu <7891482+xrq-phys@users.noreply.github.com>
 2026-08-07 16:14:26 +08:00
Hongsheng Liu 62bf8d630d docs: document distributed layerwise offload compatibility (#5839) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-06 23:21:43 +08:00
Bo Lilishunyang12lishunyang12
 		d219d93bbb [Attention] Refine TRTLLM attention support for MiniMax H3 (#5779) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
 2026-08-06 09:38:17 +00:00
Zijun HuangandWeiQing Chen 93457f29ff [NPU][Quantization] Add RainFusion attention and INT8 online quantization for MiniMax H3 (#5706) 
Signed-off-by: Zijun Huang <zijunhua@usc.edu>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-08-06 14:15:21 +08:00
bjf-frz 9f978923f2 [docs] consolidate diffusion execution modes (#5599) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
 2026-08-05 22:41:18 +08:00
Isotr0py 24741961eb [Model] Add MiniMax H3 diffusion support (#5691) 2026-08-03 14:10:40 +08:00
SYLAR 896f05e7e6 [Kernel] Add typed SAGE for TRTLLM_ATTN (#5509) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-07-31 23:11:39 +00:00
evanchuenganddavid6666666 744c65b730 [Feature] Distributed layerwise offload with DP multi-concurrency + mmap weight… (#5397) 
Signed-off-by: evanchueng <evanchueng@outlook.com>
Signed-off-by: david6666666 <530634352@qq.com>
Co-authored-by: david6666666 <530634352@qq.com>
 2026-07-30 13:35:38 +00:00
SYLARandlishunyang12 7762d664d8 [Attention] Add trtllm diffusion attention backend with Skip-Softmax (#5283) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-07-27 22:48:11 +08:00
TaffyOfficialandTaffyOfficial 93786c9cd3 [perf] Add configurable diffusion compile granularity (#4603) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
 2026-07-24 15:22:06 +00:00
Diya Peng b471a5cf23 [Diffusion] Add FLUX.2-dev CPU Offload (Layerwise) (#5256) 
Signed-off-by: dpeng123 <diyapeng123@gmail.com>
 2026-07-23 14:53:04 +08:00
bjf-frzandHongsheng Liu be667a8067 [Refact] Refactor diffusion output streaming (#5166) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-23 10:06:08 +08:00
bjf-frz 1a4a042a83 [REFACT]Refactor diffusion outputs to payload metadata (#4922) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
 2026-07-14 17:24:03 +08:00
Hongsheng Liuandhsliu_ustc 625892032b [Doc] Standardize serving names and identifiers (#5081) 
Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
 2026-07-13 23:59:42 +08:00
Rahul Steiger 66b9ad287a Extend Cosmos3 CPU offloading to component and layerwise paths (#4695) 2026-07-13 09:48:53 +00:00
Samit b96bc5ac87 Support HuggingFace kernels package for native diffusion attention backends (#4926) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-07-08 14:34:36 +08:00
JiangJie ZhangandSamit c45ac74d43 [Core][Frontend] Support request-level batching for diffusion pipelines (#4079) 
Signed-off-by: jader <yjader@foxmail.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-06-29 18:09:19 +02:00
Rahul SteigerandHongsheng Liu 327e9dcaf2 [Feature] Spatially-sharded (SP) decode for the Wan VAE (#4620) 
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-27 09:01:25 +08:00
MrDongslsMr.DongYueqian LinYueqian Lin
 		1951dca786 [Model] add soulx-singer support (#3862) 
Signed-off-by: Mr.Dong <1042542469@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Mr.Dong <1042542469@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-06-24 05:36:45 +00:00
Chendi.XueandHongsheng Liu e7f0db1067 [SDXL] SDXL model enabling (#4331) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-16 22:51:11 +08:00
汪志鹏Yufeng ZhuZhengyuan SuClaude Opus 4.6Cursor
 		46e0b71423 [Enhancement] Introduce High-Performance MoT (Mixture-of-Tokens) Kernels: Triton Implementation (#3960) 
Signed-off-by: Yufeng Zhu <yzhu802@gatech.edu>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Zhengyuan Su <su.zhengyuan@u.nus.edu>
Co-authored-by: Yufeng Zhu <yzhu802@gatech.edu>
Co-authored-by: Zhengyuan Su <su.zhengyuan@u.nus.edu>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-16 03:08:02 +00:00
WeiQing ChenlishunyangGao Han
 		126fb4b4a9 [Blackwell][1/N] Add SageAttention3 diffusion backend on blackwell(GB200/B200/RTX5090/PRO6000/DGX Spark available) (#3015) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-05-28 14:08:45 +00:00
SYLARfrauttauteffasuZhang Jian
 		373d3c7476 [Blackwell] Add CUDNN_ATTN and FLASHINFER_ATTN backends for diffusion (auto-route) (#3079) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Zhang <jianmusings@gmail.com>
Signed-off-by: Zhang Jian <jianmusings@gmail.com>
Co-authored-by: frauttauteffasu <247219357+frauttauteffasu@users.noreply.github.com>
Co-authored-by: Zhang Jian <jianmusings@gmail.com>
 2026-05-28 11:09:25 +08:00
Samit 083b5e363f [Diffusion] Support LoRA in step-wise execution (#3639) 
Signed-off-by: samithuang <285365963@qq.com>
 2026-05-20 22:26:18 +08:00
Rustam Khadipash 89f8819525 [Feature] Add support for Pipeline Parallel and integrate it into Wan 2.2 (#2322) 
Signed-off-by: Rustam Khadipash <16683750+hadipash@users.noreply.github.com>
 2026-05-19 09:42:17 +08:00
Canlin GuoandHongsheng Liu ef7f2f1cd2 [Docs] Refactor the attention backend docs/skill (#3475) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-11 09:13:07 +08:00
Nick CaoandClaude 65bc968465 [Refactor] Rename SupportsModuleOffload to SupportsComponentDiscovery (#3354) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-08 14:16:49 +00:00
Chenguang Zheng eeb7e698c9 [Clean] Remove multi-replica Bagel CI and related docs/configs (#3407) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-05-08 09:28:52 +08:00
Zheng WengangPeiqi YinyinpeyinpeiqiHongsheng LiuGao HanChenguang Zheng
 		1e5f288a91 [FEAT] support multi-stage deployment (#2396) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: Zheng Wengang <zwg0606@gmail.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Chenguang Zheng <645327136@qq.com>
 2026-05-06 19:33:18 +08:00
Chendi.XueandCanlin Guo 353ac8a402 [Enhancement] Offload transformer after switch to transformer-2 (#3224) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-05-05 23:48:46 +00:00
Ting FUasukaqaq-sjaderHongsheng LiuGao Han
 		e49fbd8a2d [Feat] DiffusionEngine Support async batch infer (#2729) 
Signed-off-by: Semmer <semmer@live.cn>
Signed-off-by: jader <yjader@foxmail.com>
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: jader <yjader@foxmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-05-06 09:47:13 +12:00
WeiQing ChenandSYLAR 954d0642a4 [Docs] Update diffusion quantization guides (#3200) 
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-28 04:33:24 -07:00
lsyyyyyNick CaoClaudeSYLAR汪志鹏
 		755a2d9d96 [Feat] support layerwise offload for Bagel (#2734) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Signed-off-by: siyuan.lei <siyuanlei37@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-04-25 15:44:56 +08:00
jiangmengyu18andbetta18 c174c95ab0 [NPU] [Quant] Support HunyuanImage3 offline quantization with vLLM-Ascend on diffusion path (#2979) 
Signed-off-by: betta18 <jiangmengyu1@huawei.com>
Signed-off-by: jiangmengyu18 <56633611+jiangmengyu18@users.noreply.github.com>
Co-authored-by: betta18 <jiangmengyu1@huawei.com>
 2026-04-25 06:14:01 +00:00
LancerandHongsheng Liu 345504b68f [Feat] support quantization for GLM-IMAGE (#2292) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-24 13:49:49 +08:00
WeiQing Chen d4cbdff9cd [Doc] Add diffusion attention backend docs (#3011) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-04-23 11:30:47 +08:00
Nick CaoandClaude e3b0afbb97 [Refactor] Let diffusion pipelines declare offloadable modules via SupportsModuleOffload (#2427) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-23 04:23:00 +08:00
Liang LvandSYLAR ee15f39ea7 [AutoRound] Support Qwen Omni W4A16 quantization model (#2670) 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-22 19:54:45 +08:00