Commit Graph
984 Commits
Author SHA1 Message Date
Isotr0py 3affc7766d Enable VAE tiling for Qwen-Image-2.1 (#5)
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix(diffusion): Qwen-Image-2.1 VAE spatial ratio, tiling e2e verification

- AutoencoderKLQwenImage21: derive spatial_compression_ratio from the
  architecture (2 ** len(temperal_downsample) = 16x) instead of trusting
  the checkpoint's scale_factor_spatial=8, which corrupted tiled
  decode/encode geometry (output cropped to half size) and tiling gating.
- pipeline: _read_vae_scale_factor prefers the architecture-derived value
  for the same reason.
- models/qwen_image_21/__init__: lazy exports to break the circular import
  distributed VAE -> package __init__ -> pipeline -> distributed VAE.
- Add VAE unit tests: ratio derivation, tiled vs untiled decode/encode
  consistency, distributed tile_split/exec/merge == tiled_decode.
- e2e matrix (GB200, seed 42, 20 steps) all green; see e2e_out/qwen21_vae/.
- examples: qwen_image_21_edit.py gains --vae-use-tiling.
- docs: mark VAE patch-parallel verified for Qwen-Image-2.1; recipe notes
  the default-tile banding limitation at 2048px+.

* clean tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix cudnn determination

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
2026-09-18 10:02:50 +08:00
Isotr0py 4fc3cf8c60 Enable CUDA Graph for DiT decode layers (#4)
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* feat(qwen_image_21): opt-in CUDA graph capture for KV-cache decode steps

Decode steps (1..N) recompute only the target image's tokens at a fixed
shape, but the per-request dict KV cache made addresses unstable, so the
step could not be captured. Prefill now copies prefix K/V into persistent
fixed-address buffers (keyed by branch/batch/prefix_len/target_tokens/
dtype/device/backend) and re-points the caller's cache dicts at them;
decode replays a graph captured per key over the full 32-block forward.
Eager fallback (logged) for non-CUDA, SP/ring/TP>1, HSDP, compiled or
offload-hooked blocks, KV-cache quantization, dynamic LoRA wrappers,
padded text masks, unknown keys, and capture failure.

Off by default; opt in via OmniDiffusionConfig.enable_cuda_graph_decode
or the QwenImage21Transformer2DModel constructor kwarg.

E2E (GB200, T2I 1024x1024, 50 steps, CFG 4.0, CUDNN_ATTN):
171.2 -> 142.1 ms/step, 8.70s -> 7.24s total (1.20x), 44.9 dB PSNR
vs eager output.

* clean

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
2026-09-18 10:02:39 +08:00
Zhenqiang Xuandzqxu 242c3cddeb [Hardware][Ascend] Integrate MindIE dense and sparse quantized attention for Wan2.2 T2V (#7561)
Signed-off-by: zqxu <xuzhenqiang3@huawei.com>
Co-authored-by: zqxu <xuzhenqiang3@huawei.com>
2026-09-18 01:28:27 +00:00
Anjie Hou f8197e120f [5/N] Schedule chunk transfers as their own stage (#7406)
Signed-off-by: specture724 <specture724@gmail.com>
2026-09-18 09:27:56 +08:00
Alex Brooks 6bda549ac2 [Bugfix] Fix Parallel State Initialization For MoE + Diffusion (#7676)
Signed-off-by: Alex Brooks <albrooks@redhat.com>
2026-09-17 17:15:37 -06:00
Joshna-Medisetty 77d8de70da [XPU][HunyuanImage3] Fix HunyuanImage-3.0 text-to-image on XPU (#7674)
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
2026-09-17 19:23:27 +00:00
dongbo910220 a6570b6814 [Diffusion] Fuse ERNIE-Image Q/K RoPE for eager execution (#7502)
Signed-off-by: dongbo910220 <1275604947@qq.com>
2026-09-17 19:06:45 +00:00
BeatSeat 9cdc57d2e7 [Bugfix][MAGI-2] Fix layerwise offload OOM by pointing offload block attrs to 'block' (#7523) (#7540)
Signed-off-by: BeatSeat <wendavid552@gmail.com>
2026-09-17 19:02:08 +00:00
dongbo910220 97912de8d8 [Kernel] Fuse LongCat paired Q/K RoPE (#7500)
Signed-off-by: dongbo910220 <1275604947@qq.com>
2026-09-18 02:27:57 +08:00
2ab5d1701a [4/N] Unify worker-side MP4 encoding and wire Wan into it (#7048)
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-09-17 23:33:56 +08:00
Chenchao Xu d0755b5716 [Model] Add π0.5 (Pi0.5) VLA model support (#6950)
Signed-off-by: chenchaox <chenchaox@nvidia.com>
2026-09-17 05:18:51 -07:00
WeiQing Chen cb439f3e84 [Diffusion][Model] Add direct FastH3 8-Step V2 checkpoint loading (#7610)
Signed-off-by: david6666666 <530634352@qq.com>
2026-09-17 08:41:52 +00:00
HAAZZZEEEE becfefd440 [Hardware][Ascend] Add Wan2.2 MXFP4 UOS, Smooth and W4A8 fallback (#7210)
Signed-off-by: HAAZZZEEEE <wang-taicheng@qq.com>
2026-09-17 03:11:40 +00:00
NumberWanandwangyu 81b5be274d [Bugfix][Qwen-Image] Restore RotaryEmbedding CUDA RoPE for Diffusers e2e (#7513)
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
2026-09-16 08:54:28 -07:00
dongbo910220 217a6af52c [Diffusion] Fuse Qwen-Image-Edit select01 modulation for CUDA (#5921)
Signed-off-by: dongbo910220 <1275604947@qq.com>
2026-09-16 19:06:28 +08:00
dengyunyang 249e84d21b [Boogu-Image] Fuse QKV/FFN projections + switch to diffusion RMSNorm (#6649)
Signed-off-by: dengyunyang <584797741@qq.com>
2026-09-16 03:13:54 -07:00
d554e6b67e [feat] Extend disaggregated MiniMax-H3 stage with VAE encoders (#6939)
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Signed-off-by: Jader <yjader@foxmail.com>
Co-authored-by: Jader <yjader@foxmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-09-16 16:52:35 +08:00
SYLARandlishunyang12 507cb1d834 [Attention] [P1] Keep H3 VSA policy in the model and share sparse primitives (#7535)
Signed-off-by: lishunyang12 <lishunyang12@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang12@users.noreply.github.com>
2026-09-16 15:47:31 +08:00
Isotr0py 2912672b2e Initialize QMI pipeline implementation (#2)
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
2026-09-16 15:14:23 +08:00
linzhenpl07andClaude Opus 5 9005d78903 [LingBot World] Reuse the text encode for a prompt already encoded (#6845)
Signed-off-by: linzhenpl07 <linzhenpl07@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 15:01:11 +08:00
Zhou Taichang 5e9024171d [Bugfix] Skip generic text-only warmup for DreamZero (#7548)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-09-16 14:43:18 +08:00
vOv 21d86ec923 [Diffusion] MAGI-2: regional compile coverage (#7174)
Signed-off-by: cr-gao <gaochenrui@sjtu.edu.cn>
2026-09-16 13:01:26 +08:00
Zheng Wengangandwuli666 275720b378 [Bugfix] Keep async video jobs queued until inference and abort on delete (#6759)
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: wuli666 <421774554@qq.com>
Co-authored-by: wuli666 <421774554@qq.com>
2026-09-15 20:43:30 -07:00
HAAZZZEEEE 3eb5c3b4e6 [Bugfix][Wan2.2] Apply UniPC flow shift once (#7576)
Signed-off-by: HAAZZZEEEE <wang-taicheng@qq.com>
2026-09-16 10:06:41 +08:00
yzhautouskay 6040bb97e2 [Perf][Cosmos3] Add SeaCache support for Cosmos3 (#6922)
Signed-off-by: Yuliya Zhautouskaya <yzhautouskay@nvidia.com>
2026-09-15 14:57:15 -07:00
dongbo910220 952022c8ec [Diffusion][Perf] Add Qwen-Image QK RoPE Triton path (#5931)
Signed-off-by: dongbo910220 <1275604947@qq.com>
2026-09-15 14:28:48 +00:00
wtz2333 a3dee6fdf5 [Lingbot World] Bound image condition storage and extend temporal RoPE (#6838)
Signed-off-by: wtz2333 <2955110911@qq.com>
2026-09-15 18:49:55 +08:00
Anjie HouandClaude Opus 5 85da943391 [Refactor][Diffusion] Complete generic backend plan consumption (#7313)
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 08:40:04 +08:00
kunkun c68eadbbe0 [Performance] Optimize MammothModa2 AR → DiT hidden-state transfer (#7102)
Signed-off-by: kunkunblueberry <1833921874@qq.com>
Signed-off-by: kunkun <72174834+kunkunblueberry@users.noreply.github.com>
2026-09-14 15:25:08 -07:00
heyuanliu-intel 40f8e94400 [Model][MiniMax-H3] Multi-GPU serving fixes and layer-wise offload component selection (#7047)
Signed-off-by: heyuanliu-intel <heyuan.liu@intel.com>
2026-09-14 10:02:30 -07:00
Tai An 1b6cd282f0 fix(qwen-image): drop the shadowed dead _get_qwen_prompt_embeds duplicate (#7461)
Signed-off-by: Anai-Guo <antai12232931@outlook.com>
2026-09-14 13:48:51 +00:00
Oliver7th 767cc7977e [Bugfix][NPU] Materialize causal mask in FlashAttention dense path (#7324)
Signed-off-by: Oliver7th <fangzhengyi@huawei.com>
2026-09-14 01:46:29 +00:00
linzhenpl07andClaude Sonnet 5 311e714d34 [AR-Diffusion] Add session-owned streaming VAE decode (#6533)
Signed-off-by: linzhenpl07 <linzhenpl07@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-13 18:17:11 +08:00
KANG Qihan f7d9deb45a [Kernel][Boogu-Image] Fuse Q/K RMSNorm + interleaved RoPE via fused_qk_norm_rope (#6982)
Signed-off-by: Qihan Kang <rollykanggg@gmail.com>
2026-09-13 09:39:27 +08:00
7e520f9cc4 [3/N] Encode streamed video on the worker with bounded batching (#7018)
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-09-12 20:58:11 +08:00
gerayandHongsheng Liu 9edc1b4667 Optimize CosyVoice3 Stage1 flow batching (#4876)
Signed-off-by: gerayking <399geray@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-09-12 20:31:49 +08:00
AliciaandHongsheng Liu 28291df179 [Refactor][Diffusion] Remove model-specific names from LoRA and ModelOpt loader defaults (#5907)
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-09-12 20:30:03 +08:00
junpengw67-maxand吴俊鹏 bc0c9f4b45 [Bugfix][Model][Lance] Support decoded video frames in video editing (#5128)
Signed-off-by: 吴俊鹏 <248679769+junpengw67-max@users.noreply.github.com>
Co-authored-by: 吴俊鹏 <248679769+junpengw67-max@users.noreply.github.com>
2026-09-12 04:23:24 +00:00
DanaerLeeandHongsheng Liu 220366db00 [Perf][Diffusion] Run MammothModa2 DiT attention through the shared attention layer (#7094)
Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-09-12 11:31:05 +08:00
MaciejBalaNV 13b85c5622 [Bugfix] Add explicit error when using CFGP with distilled Cosmos3 models (#7427)
Signed-off-by: Maciej Bala <mbala@nvidia.com>
2026-09-11 22:59:49 +00:00
Yueqian LinandSy03 c111f3a2d6 [Model] Add Tencent AuK speech generation and editing (encoder + diffusion pipeline) (#7385)
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Sy03 <1370724210@qq.com>
2026-09-11 20:09:58 +00:00
78e1a0ae5c [Bugfix][Model] Fix FLUX.2 Klein multi-image edit metadata (#7430)
Signed-off-by: QI JIA <qi.jia@shengshu.ai>
Co-authored-by: QI JIA <qi.jia@shengshu.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-11 14:18:07 +00:00
wtz2333andZhou Taichang 02aaa34059 [Model] Add LingBot World Ulysses sequence parallelism (#6841)
Signed-off-by: wtz2333 <2955110911@qq.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
2026-09-11 18:37:12 +08:00
hyw 43b8de9b0e [Kernel][MiniMax-H3] Run Q/K RMSNorm-RoPE in one launch (#7167)
Signed-off-by: hyw <yuweih205@gmail.com>
2026-09-11 09:18:12 +00:00
hyw f62d4ad4a7 [Model][ERNIE-Image] Delay AdaLN modulation broadcast (#7171)
Signed-off-by: hyw <yuweih205@gmail.com>
2026-09-11 09:15:32 +00:00
hyw 3f7c217457 [Model] Avoid scalar synchronizations in GLM-Image preparation (#7172)
Signed-off-by: hyw <yuweih205@gmail.com>
2026-09-11 09:09:26 +00:00
a7ab91fb85 [Example] Migrate HunyuanImage-3.0 to model_extras + shared task examples (#5559)
Signed-off-by: suyanli220 <suyanli220@gmail.com>
Signed-off-by: suyan.li <suyan.li@bytedance.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: suyan.li <suyan.li@bytedance.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-09-11 16:47:25 +08:00
Samit 96360ccb00 [Perf][Diffusion] Optimize HSDP startup via Rank-0 shared weight loading and accelerated LoRA delta computation (#7005)
Signed-off-by: samithuang <285365963@qq.com>
2026-09-11 08:28:42 +00:00
Yan Caoandyancaocn bad50980c9 [Bugfix] Fix Helios USP: per-component split for correct sequence parallelism (#6930)
Signed-off-by: yancaocn <yancaochn@163.com>
Co-authored-by: yancaocn <yancaochn@163.com>
2026-09-11 11:43:34 +08:00
WeiQing Chen 3d952d1334 [Bugfix] Keep MiniMax-H3 reference audio budgets separate (#7281)
Signed-off-by: david6666666 <530634352@qq.com>
2026-09-11 09:26:35 +08:00