Commit Graph
15 Commits
Author SHA1 Message Date
Isotr0py a04b95bf6b Address review: tuple image input, NPU index_select rope, per-key decode graph ownership
- Normalize tuple image inputs in pre_process_func (empty tuple -> T2I path)
- Use torch.index_select in QwenImage21Rope for NPU compatibility (#6320 style)
- Refuse concurrent aliasing of shared decode-graph prefix KV entries: track a
  single live owner per key, non-owners fall back to eager decode
2026-09-21 08:36:28 +00:00
Tianyu Guo 9dabaf8b80 Default Qwen-Image 2.1 CFG scale to 1.0 (#7859)
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
2026-09-20 11:51:35 +08:00
Isotr0py 5b540f55d3 Fix decode cuda graph for Qwen-Image-2.1 model-level cpu offload (#7801) 2026-09-19 00:10:53 +08:00
Isotr0py 47e6acc1e3 Add FP8 KV cache support for Qwen-Image-2.1 DiT (#7797) 2026-09-19 00:06:58 +08:00
Isotr0py 2a92b71cb9 Add FP8 online qunatization support for Qwen-Image-2.1 (#7792)
Signed-off-by: Isotr0py <mozf@inferact.ai>
2026-09-18 22:33:16 +08:00
Tianyu Guo ab4cf24b3e Optimize Qwen-Image 2.1 attention and decode graph reuse (#7790)
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
2026-09-18 19:37:19 +08:00
Tianyu Guo f930348e79 Preserve Qwen-Image 2.1 condition latent order (#7766)
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
2026-09-18 18:07:45 +08:00
Tianyu Guo b0a309e5e7 Isolate Qwen-Image 2.1 prefill and decode scheduling (#7770)
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
2026-09-18 14:11:22 +08:00
Tianyu GuoandIsotr0py a26013145c Align Qwen-Image 2.1 preprocessing, CFG defaults and prefix cache (#7769)
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-09-18 13:53:15 +08:00
Tianyu Guo 936c0cc7fe Align Qwen-Image-2.1 VAE normalization with Diffusers (#7768)
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
2026-09-18 13:51:06 +08:00
Tianyu Guo 93d7a06e7f Use pre-norm text features for Qwen-Image 2.1 (#7765)
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
2026-09-18 13:50:36 +08:00
Tianyu Guo eb54a3186b Separate Qwen-Image 2.1 batches by condition image layout (#7764)
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
2026-09-18 13:47:35 +08:00
Isotr0py 3affc7766d Enable VAE tiling for Qwen-Image-2.1 (#5)
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix(diffusion): Qwen-Image-2.1 VAE spatial ratio, tiling e2e verification

- AutoencoderKLQwenImage21: derive spatial_compression_ratio from the
  architecture (2 ** len(temperal_downsample) = 16x) instead of trusting
  the checkpoint's scale_factor_spatial=8, which corrupted tiled
  decode/encode geometry (output cropped to half size) and tiling gating.
- pipeline: _read_vae_scale_factor prefers the architecture-derived value
  for the same reason.
- models/qwen_image_21/__init__: lazy exports to break the circular import
  distributed VAE -> package __init__ -> pipeline -> distributed VAE.
- Add VAE unit tests: ratio derivation, tiled vs untiled decode/encode
  consistency, distributed tile_split/exec/merge == tiled_decode.
- e2e matrix (GB200, seed 42, 20 steps) all green; see e2e_out/qwen21_vae/.
- examples: qwen_image_21_edit.py gains --vae-use-tiling.
- docs: mark VAE patch-parallel verified for Qwen-Image-2.1; recipe notes
  the default-tile banding limitation at 2048px+.

* clean tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix cudnn determination

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
2026-09-18 10:02:50 +08:00
Isotr0py 4fc3cf8c60 Enable CUDA Graph for DiT decode layers (#4)
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* feat(qwen_image_21): opt-in CUDA graph capture for KV-cache decode steps

Decode steps (1..N) recompute only the target image's tokens at a fixed
shape, but the per-request dict KV cache made addresses unstable, so the
step could not be captured. Prefill now copies prefix K/V into persistent
fixed-address buffers (keyed by branch/batch/prefix_len/target_tokens/
dtype/device/backend) and re-points the caller's cache dicts at them;
decode replays a graph captured per key over the full 32-block forward.
Eager fallback (logged) for non-CUDA, SP/ring/TP>1, HSDP, compiled or
offload-hooked blocks, KV-cache quantization, dynamic LoRA wrappers,
padded text masks, unknown keys, and capture failure.

Off by default; opt in via OmniDiffusionConfig.enable_cuda_graph_decode
or the QwenImage21Transformer2DModel constructor kwarg.

E2E (GB200, T2I 1024x1024, 50 steps, CFG 4.0, CUDNN_ATTN):
171.2 -> 142.1 ms/step, 8.70s -> 7.24s total (1.20x), 44.9 dB PSNR
vs eager output.

* clean

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
2026-09-18 10:02:39 +08:00
Isotr0py 2912672b2e Initialize QMI pipeline implementation (#2)
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
2026-09-16 15:14:23 +08:00