anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,228 Commits 235 Branches 31 Tags
44ea27c8094095bbffd88fa3befdfaa55ba4bc50
Commit Graph
150 Commits
Author SHA1 Message Date
Isotr0py be8a72f0a6 Stack torch.compile with CUDA graph decode instead of choosing one 
Regionally compile the DiT blocks even when enable_cuda_graph_decode is set
and let graph capture record the compiled (fused) kernels; inductor's own
cudagraphs stay off so the two graph layers never nest. Measured on GB200
(1024x1024, 50 steps, seed 42, BF16): combo 3.04s vs compile-only 3.2s,
graph-only 4.0s, eager 4.3s per image; 44.5-45.6 dB PSNR vs eager.
 2026-09-22 08:58:32 +00:00
Isotr0py d3cf26c1b1 Wire enable_cuda_graph_decode through engine args, stage config and CLI 
Follows the vae_use_tiling path: OmniDiffusionConfig field, OmniEngineArgs,
StageDeployConfig, _DiffusionConfigProjection and --[no-]enable-cuda-graph-decode.
Documents interplay with enforce_eager and cudagraph_mode.
 2026-09-21 08:36:34 +00:00
Isotr0py 9174554e40 Merge remote-tracking branch 'origin/main' into qwen-image-2.1 
# Conflicts:
#	vllm_omni/diffusion/executor/multiproc_executor.py
#	vllm_omni/diffusion/offloader/sequential_backend.py
 2026-09-21 07:32:04 +00:00
Wu JIAZHENAcerak01-fyamy-why-3459
 		102ba716bf [Feat] Add native Mooncake KV transfer from AR to DiT (#7166) 
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Co-authored-by: Acerak01-fy <wfy2003324@163.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-20 14:44:55 +00:00
Tianyu Guo 3ea9e60501 Fix diffusion API initialization (#7868) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-20 12:08:50 +08:00
akshatvishu 415005be22 [Model] Add circlestone-labs/Anima (#4083) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-09-19 16:16:38 +00:00
TaffyOfficialTaffyOfficialGao Han
 		698f716012 [Refactor] Reject unknown diffusion config fields (#5172) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Signed-off-by: zuiho <2324465096@qq.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-19 15:37:34 +00:00
Tianyu Guo ab4cf24b3e Optimize Qwen-Image 2.1 attention and decode graph reuse (#7790) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 19:37:19 +08:00
Isotr0py 4fc3cf8c60 Enable CUDA Graph for DiT decode layers (#4) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* feat(qwen_image_21): opt-in CUDA graph capture for KV-cache decode steps

Decode steps (1..N) recompute only the target image's tokens at a fixed
shape, but the per-request dict KV cache made addresses unstable, so the
step could not be captured. Prefill now copies prefix K/V into persistent
fixed-address buffers (keyed by branch/batch/prefix_len/target_tokens/
dtype/device/backend) and re-points the caller's cache dicts at them;
decode replays a graph captured per key over the full 32-block forward.
Eager fallback (logged) for non-CUDA, SP/ring/TP>1, HSDP, compiled or
offload-hooked blocks, KV-cache quantization, dynamic LoRA wrappers,
padded text masks, unknown keys, and capture failure.

Off by default; opt in via OmniDiffusionConfig.enable_cuda_graph_decode
or the QwenImage21Transformer2DModel constructor kwarg.

E2E (GB200, T2I 1024x1024, 50 steps, CFG 4.0, CUDNN_ATTN):
171.2 -> 142.1 ms/step, 8.70s -> 7.24s total (1.20x), 44.9 dB PSNR
vs eager output.

* clean

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 10:02:39 +08:00
Zhenqiang Xuandzqxu 242c3cddeb [Hardware][Ascend] Integrate MindIE dense and sparse quantized attention for Wan2.2 T2V (#7561) 
Signed-off-by: zqxu <xuzhenqiang3@huawei.com>
Co-authored-by: zqxu <xuzhenqiang3@huawei.com>
 2026-09-18 01:28:27 +00:00
Chenchao Xu d0755b5716 [Model] Add π0.5 (Pi0.5) VLA model support (#6950) 
Signed-off-by: chenchaox <chenchaox@nvidia.com>
 2026-09-17 05:18:51 -07:00
HAAZZZEEEE becfefd440 [Hardware][Ascend] Add Wan2.2 MXFP4 UOS, Smooth and W4A8 fallback (#7210) 
Signed-off-by: HAAZZZEEEE <wang-taicheng@qq.com>
 2026-09-17 03:11:40 +00:00
Wu JIAZHENJaderHongsheng Liu
 		d554e6b67e [feat] Extend disaggregated MiniMax-H3 stage with VAE encoders (#6939) 
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Signed-off-by: Jader <yjader@foxmail.com>
Co-authored-by: Jader <yjader@foxmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-16 16:52:35 +08:00
Zheng Wengangandwuli666 275720b378 [Bugfix] Keep async video jobs queued until inference and abort on delete (#6759) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: wuli666 <421774554@qq.com>
Co-authored-by: wuli666 <421774554@qq.com>
 2026-09-15 20:43:30 -07:00
yzhautouskay 6040bb97e2 [Perf][Cosmos3] Add SeaCache support for Cosmos3 (#6922) 
Signed-off-by: Yuliya Zhautouskaya <yzhautouskay@nvidia.com>
 2026-09-15 14:57:15 -07:00
Samit 96360ccb00 [Perf][Diffusion] Optimize HSDP startup via Rank-0 shared weight loading and accelerated LoRA delta computation (#7005) 
Signed-off-by: samithuang <285365963@qq.com>
 2026-09-11 08:28:42 +00:00
NancyFyongNancyFyongSamit
 		a2486dd00b [Core][Diffusion] Add a typed pre-D2H video media contract (#6615) 
Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: NancyFyong <NancyFyong@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-09-10 09:50:54 +00:00
Chenguang Zheng 087ac46318 [Refactor 1/N] Centralize Omni config resolution (#5140) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-09-08 10:30:26 +08:00
summerandAlicia 4b81f692d1 [Bugfix][Diffusion] Canonicalize cache_backend=None to "none" (#7041) 
Signed-off-by: summer <128961079+zhang-keliang@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-09-07 15:56:51 +08:00
Zeyu Huang | 黃澤宇asukaqaq-sHongsheng Liu
 		f26f9c3a7f [Diffusion] Implement paged AR↔DiT KV connector ("v1", reusing vllm's native mooncake) (#6310) 
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-06 05:41:54 +08:00
SYLAR ea3cf99894 [diffusion][feature] Add component-selective offload policies (#5929) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-09-06 05:40:09 +08:00
Hongsheng Liu d4096e01fe [Model] Add native MAGI-2 Preview diffusion support (#5918) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-05 15:32:30 +08:00
wangyulishunyang12CursorBo Li
 		95a9311696 [CI] Select mirror_hardwares presets via MIRROR_HW (H100/B200) (#5543) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-09-02 15:39:49 +00:00
Wang fuyinzwhzzz0821zwhzzz0821
 		fe4a2a7477 [Diffusion] Add paged KV cache support for HunyuanImage3 DiT on NPU and GPU (#6563) 
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Signed-off-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Co-authored-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
Co-authored-by: zwhzzz0821 <2831474076@qq.com>
 2026-08-31 10:15:46 +00:00
baonudesifeizhai 8be601ed7f [Diffusion] Add native SymmMem Fast Ulysses transport (#6340) 
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
 2026-08-30 04:14:21 +00:00
WeiQing ChenandGao Han 9a88d2d21d [diffusion][bugfix] Separate Diffusers hooks from model metadata (#6749) 
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-29 17:37:14 +00:00
HAAZZZEEEE c50b7de167 [NPU][Diffusion] MiniMax-H3 RainFusion end_step tail fallback (#6037) 
Signed-off-by: HAAZZZEEEE <wang-taicheng@qq.com>
 2026-08-29 18:36:05 +08:00
Canlin Guolishunyang12yuanwu
 		b6a4baa659 [Feature] Support text encoder Disaggregation for H3 (#5885) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: yuanwu <yuan.wu@intel.com>
 2026-08-28 14:27:05 +08:00
Honghan ZhuClaude Opus 4.7Gao Han
 		7ae15c74c8 text and image metric (#6150) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-26 02:54:45 +00:00
Hongsheng Liuandhsliuustc0106 c608910d20 [Diffusion] Register HWR mmap for direct DLO H2D (#6591) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-25 23:30:11 +08:00
Hongsheng Liuandhsliuustc0106 8360346a66 [Diffusion] Integrate no-AllGather DLO with Host Weight Runtime (#6486) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-25 08:03:53 +08:00
bjf-frzandHongsheng Liu 072bfc02dd [FEAT]Add FastVideo VSA backend for Wan2.2 (#4820) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-24 14:48:31 +08:00
Rahul SteigerHongsheng LiuRahul Steiger
 		b39f599b09 [Core] Add UniProc diffusion executor for single-GPU (#6308) 
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Signed-off-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
 2026-08-22 21:58:21 +08:00
Wang fuyinandHongsheng Liu 72ee535fcd [Diffusion] Add native paged KV backend for diffusion workers (#6102) 
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 21:45:45 +08:00
Shin 70bdd49402 [Model] Add HiDream-O1-Image support (#5194) 
Signed-off-by: yixiaoer <shin@yixiaoer.sg>
 2026-08-21 14:15:37 +08:00
bjf-frzandHongsheng Liu 14bd8f8772 [REFACT]Refactor diffusion parallel state (#5531) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:23:10 +08:00
Yukim1andzwhzzz0821 0a1846aa45 [Diffusion] Add native KV cache initialization and Scheduler-managed block allocation (#6094) 
Signed-off-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
Co-authored-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
 2026-08-18 07:00:24 +00:00
Yichen WangandDidan Deng 2a9f6aaa8e Add π0 VLA model support (#4222) 
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-08-17 14:29:17 +08:00
SYLAR ce6ab0ca46 [Model] Add LTX-2.5 support (#6070) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-08-15 18:27:23 +08:00
Jerry Song 89015d5cad [Feature]: Adds distilled LoRA support for diffusion models (#2783) 
Signed-off-by: Songrui625 <songrui625@gmail.com>
 2026-08-11 14:23:43 +00:00
Zeyu Huang | 黃澤宇andHongsheng Liu 33dd9a5a3e [Core][Refactor][Diffusion] refactor request scheduler's admission wait policy (#5843) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-10 15:06:27 +08:00
Yukim1zwhzzz0821Hongsheng Liu
 		6ae7ff7846 [Diffusion] Prepare HunyuanImage3 for Scheduler-managed paged KV cache (#5541) 
Signed-off-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
Co-authored-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-10 08:48:56 +08:00
Diya Pengandlishunyang12 45629a8e26 [MiniMax-H3] TeaCache support and Cache-DiT validation (#5840) 
Signed-off-by: dpeng123 <diyapeng123@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
 2026-08-09 15:07:21 +00:00
NumberWan cfa6d66ed6 [BugFix][Nightly CI] Opt in FA deterministic for Qwen-Image accuracy (#5887) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-08-07 17:15:22 +08:00
Isotr0pylishunyang12SYLARDavid Chen
 		48298030d4 [Diffusion] Add Minimax-H3 modular pipeline support (#5720) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: David Chen <530634352@qq.com>
 2026-08-06 11:43:23 +00:00
Bo Lilishunyang12lishunyang12
 		d219d93bbb [Attention] Refine TRTLLM attention support for MiniMax H3 (#5779) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
 2026-08-06 09:38:17 +00:00
Zijun HuangandWeiQing Chen 93457f29ff [NPU][Quantization] Add RainFusion attention and INT8 online quantization for MiniMax H3 (#5706) 
Signed-off-by: Zijun Huang <zijunhua@usc.edu>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-08-06 14:15:21 +08:00
SYLARandlishunyang12 1c2a81f6d8 feat(minimax-h3): enable 2x RTX 4090/5090 with DLO (#5764) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-08-06 09:02:29 +08:00
WeiQing Chen ae7bee0c43 [Feature] Align MiniMax H3 official input matrix (#5752) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-08-05 07:12:01 +00:00
Isotr0py 24741961eb [Model] Add MiniMax H3 diffusion support (#5691) 2026-08-03 14:10:40 +08:00