anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,228 Commits 235 Branches 31 Tags
44ea27c8094095bbffd88fa3befdfaa55ba4bc50
Commit Graph
680 Commits
Author SHA1 Message Date
Isotr0py 44ea27c809 fix pre-commit 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-23 06:22:17 +00:00
Isotr0py be8a72f0a6 Stack torch.compile with CUDA graph decode instead of choosing one 
Regionally compile the DiT blocks even when enable_cuda_graph_decode is set
and let graph capture record the compiled (fused) kernels; inductor's own
cudagraphs stay off so the two graph layers never nest. Measured on GB200
(1024x1024, 50 steps, seed 42, BF16): combo 3.04s vs compile-only 3.2s,
graph-only 4.0s, eager 4.3s per image; 44.5-45.6 dB PSNR vs eager.
 2026-09-22 08:58:32 +00:00
Isotr0py 45aa5b16d9 Rename internal temperal_upsample to temporal_upsample in Qwen-Image-2.1 VAE 
Match the Qwen-Image 2.0 split: decoder kwargs/attrs use the correct
temporal_upsample spelling while the checkpoint-facing temperal_downsample
config key is kept as-is. Fixes pre-commit typos rewriting the name.
 2026-09-22 08:58:31 +00:00
Isotr0py 8e0695f362 Merge branch 'main' into qwen-image-2.1 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-22 10:11:12 +08:00
shihongzhi c11596c637 [Model] MammothModa2: enable Cache-DiT acceleration for the DiT stage (#7291) 
Signed-off-by: shihongzhi <shi65881583@gmail.com>
 2026-09-21 18:34:12 -07:00
Isotr0py 3e274a0024 Merge remote-tracking branch 'origin/main' into qwen-image-2.1 
# Conflicts:
#	tests/entrypoints/openai_api/test_image_server.py
#	vllm_omni/entrypoints/openai/api_server.py
 2026-09-22 01:23:45 +00:00
wangyu 279b1b9687 [Bugfix] Isolate weekly CPU tests from inherited attention env and hf_api mocks (#7565) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: [Your Name] [Your Email]
 2026-09-21 15:55:00 +00:00
0z5a 2c2cffd2aa [Bugfix] Preserve NextStep rectangular latent dimensions (#7853) 
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
 2026-09-21 08:26:14 -07:00
tsinghua-code e5c0afc994 [Model] Enable online FP8 linears for LingBot World (#7549) 
Signed-off-by: tsinghua-code <zr-liu25@mails.tsinghua.edu.cn>
 2026-09-21 12:21:46 +00:00
Zhou TaichangandClaude Fable 5.1 5356357cf0 [LingBot World] Shard the streaming VAE decode across the Ulysses ranks (#7651) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-21 19:00:46 +08:00
wtz2333 8efd3c2f6d [Feature] Add experimental LingBot World last-step KV reuse (#7816) 
Signed-off-by: wtz2333 <2955110911@qq.com>
 2026-09-21 17:02:15 +08:00
Isotr0py 3d6a1c4d9a Address review: examples policy, SPDX headers, recipe docs, test coverage 
- Register QwenImage21Pipeline in model_extras _EXTRA_SPECS with an
  image-to-image prompt builder that forwards height/width
- Rename run_qwen_image_21_edit.sh to run_qwen_image_21.sh
- Fix SPDX-FileCopyrightText headers
- Recipe: document mixed-KV-phase admission, add head-vs-diffusers A/B
  comparison section (measurements pending)
- Add test_registry.py and test_pipeline.py (preprocess, prepare_latents
  ordering, mixed-phase _assemble_kv_cache, post-process); extend
  test_batch_compatibility.py
 2026-09-21 08:36:42 +00:00
Isotr0py a04b95bf6b Address review: tuple image input, NPU index_select rope, per-key decode graph ownership 
- Normalize tuple image inputs in pre_process_func (empty tuple -> T2I path)
- Use torch.index_select in QwenImage21Rope for NPU compatibility (#6320 style)
- Refuse concurrent aliasing of shared decode-graph prefix KV entries: track a
  single live owner per key, non-owners fall back to eager decode
 2026-09-21 08:36:28 +00:00
WeiQing Chen 84977d9543 [Bugfix][MiniMax-H3] Fix Hopper keyframe and modulation precision (#7913) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-09-21 16:26:09 +08:00
Isotr0py 9174554e40 Merge remote-tracking branch 'origin/main' into qwen-image-2.1 
# Conflicts:
#	vllm_omni/diffusion/executor/multiproc_executor.py
#	vllm_omni/diffusion/offloader/sequential_backend.py
 2026-09-21 07:32:04 +00:00
Ding ZuhaoandCursor 154a1f25f5 [Bugfix] Honor the requested output canvas in BAGEL img2img (#7287) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-21 07:29:24 +00:00
Zhou TaichangandClaude Fable 5.1 86fcfe95f1 [LingBot World] Eleven bit-identical removals of repeated work in the realtime path (#7648) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-21 06:12:45 +00:00
DMAN d139c600c7 [Model] Add online FP8 support for Boogu-Image MLLM (#7342) 
Signed-off-by: Dmaner <2663515256@qq.com>
 2026-09-21 05:13:37 +00:00
Zeyu Huang | 黃澤宇 e36babd48f [Core][Diffusion] Camera interaction for diffusion streaming generation (LingBot World 2 as example) (#7198) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-09-20 16:19:41 +00:00
汪志鹏 139a47a578 [Model] Add MiniMax-H3 long-video latent continuation with driving audio (#7838) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-09-20 23:44:33 +08:00
Wu JIAZHENAcerak01-fyamy-why-3459
 		102ba716bf [Feat] Add native Mooncake KV transfer from AR to DiT (#7166) 
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Co-authored-by: Acerak01-fy <wfy2003324@163.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-20 14:44:55 +00:00
AliciaandHongsheng Liu d02681bf01 [Refactor][Diffusion] Move single-model layers into their model directories (#5908) 
Signed-off-by: congw729 <115451386+congw729@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-20 18:19:24 +08:00
xiaoyu-xyz 5a93ec1b4e [Model][Frontend] MiniMax-H3: Add latent-mask editing (#7465) 
Signed-off-by: xiaoyu <1259084489@qq.com>
 2026-09-20 03:54:44 +00:00
Tianyu Guo 9dabaf8b80 Default Qwen-Image 2.1 CFG scale to 1.0 (#7859) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-20 11:51:35 +08:00
ooooooyeandWeiQing Chen b2b6c6c7a9 [Perf][MiniMax-H3] Cut video VAE encode/decode memory peaks for low-memory serving (#7241) 
Signed-off-by: brandneway <gyuan4892@gmail.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-09-20 10:56:54 +08:00
NumberWan 23d8c83dab [Bugfix][Qwen-Image] Keep AutoRound W4A16 on CPU when offloading; Fixes #7555 (#7579) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-09-19 16:34:11 +00:00
akshatvishu 415005be22 [Model] Add circlestone-labs/Anima (#4083) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-09-19 16:16:38 +00:00
Sunamy-why-3459Yueqian Lin
 		7b4fc7ce2e [Model] Migrate MammothModa2 DiT to the shared diffusion runtime (#7134) 
Signed-off-by: levius <2114377220@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-09-18 13:29:18 -07:00
Joshna-MedisettyandCursor ca3ccfc9a3 [XPU][Bugfix] Make AR-Diffusion KV preallocation device-agnostic (#7777) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-18 14:41:41 -05:00
Isotr0py 5b540f55d3 Fix decode cuda graph for Qwen-Image-2.1 model-level cpu offload (#7801) 2026-09-19 00:10:53 +08:00
Isotr0py 47e6acc1e3 Add FP8 KV cache support for Qwen-Image-2.1 DiT (#7797) 2026-09-19 00:06:58 +08:00
Isotr0py 2a92b71cb9 Add FP8 online qunatization support for Qwen-Image-2.1 (#7792) 
Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 22:33:16 +08:00
Tianyu Guo ab4cf24b3e Optimize Qwen-Image 2.1 attention and decode graph reuse (#7790) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 19:37:19 +08:00
Tianyu Guo f930348e79 Preserve Qwen-Image 2.1 condition latent order (#7766) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 18:07:45 +08:00
Tianyu Guo b0a309e5e7 Isolate Qwen-Image 2.1 prefill and decode scheduling (#7770) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 14:11:22 +08:00
Tianyu GuoandIsotr0py a26013145c Align Qwen-Image 2.1 preprocessing, CFG defaults and prefix cache (#7769) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-09-18 13:53:15 +08:00
Tianyu Guo 936c0cc7fe Align Qwen-Image-2.1 VAE normalization with Diffusers (#7768) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 13:51:06 +08:00
Tianyu Guo 93d7a06e7f Use pre-norm text features for Qwen-Image 2.1 (#7765) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 13:50:36 +08:00
Tianyu Guo eb54a3186b Separate Qwen-Image 2.1 batches by condition image layout (#7764) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 13:47:35 +08:00
Isotr0py 3affc7766d Enable VAE tiling for Qwen-Image-2.1 (#5) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix(diffusion): Qwen-Image-2.1 VAE spatial ratio, tiling e2e verification

- AutoencoderKLQwenImage21: derive spatial_compression_ratio from the
  architecture (2 ** len(temperal_downsample) = 16x) instead of trusting
  the checkpoint's scale_factor_spatial=8, which corrupted tiled
  decode/encode geometry (output cropped to half size) and tiling gating.
- pipeline: _read_vae_scale_factor prefers the architecture-derived value
  for the same reason.
- models/qwen_image_21/__init__: lazy exports to break the circular import
  distributed VAE -> package __init__ -> pipeline -> distributed VAE.
- Add VAE unit tests: ratio derivation, tiled vs untiled decode/encode
  consistency, distributed tile_split/exec/merge == tiled_decode.
- e2e matrix (GB200, seed 42, 20 steps) all green; see e2e_out/qwen21_vae/.
- examples: qwen_image_21_edit.py gains --vae-use-tiling.
- docs: mark VAE patch-parallel verified for Qwen-Image-2.1; recipe notes
  the default-tile banding limitation at 2048px+.

* clean tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix cudnn determination

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 10:02:50 +08:00
Isotr0py 4fc3cf8c60 Enable CUDA Graph for DiT decode layers (#4) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* feat(qwen_image_21): opt-in CUDA graph capture for KV-cache decode steps

Decode steps (1..N) recompute only the target image's tokens at a fixed
shape, but the per-request dict KV cache made addresses unstable, so the
step could not be captured. Prefill now copies prefix K/V into persistent
fixed-address buffers (keyed by branch/batch/prefix_len/target_tokens/
dtype/device/backend) and re-points the caller's cache dicts at them;
decode replays a graph captured per key over the full 32-block forward.
Eager fallback (logged) for non-CUDA, SP/ring/TP>1, HSDP, compiled or
offload-hooked blocks, KV-cache quantization, dynamic LoRA wrappers,
padded text masks, unknown keys, and capture failure.

Off by default; opt in via OmniDiffusionConfig.enable_cuda_graph_decode
or the QwenImage21Transformer2DModel constructor kwarg.

E2E (GB200, T2I 1024x1024, 50 steps, CFG 4.0, CUDNN_ATTN):
171.2 -> 142.1 ms/step, 8.70s -> 7.24s total (1.20x), 44.9 dB PSNR
vs eager output.

* clean

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 10:02:39 +08:00
Zhenqiang Xuandzqxu 242c3cddeb [Hardware][Ascend] Integrate MindIE dense and sparse quantized attention for Wan2.2 T2V (#7561) 
Signed-off-by: zqxu <xuzhenqiang3@huawei.com>
Co-authored-by: zqxu <xuzhenqiang3@huawei.com>
 2026-09-18 01:28:27 +00:00
Joshna-Medisetty 77d8de70da [XPU][HunyuanImage3] Fix HunyuanImage-3.0 text-to-image on XPU (#7674) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
 2026-09-17 19:23:27 +00:00
dongbo910220 a6570b6814 [Diffusion] Fuse ERNIE-Image Q/K RoPE for eager execution (#7502) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2026-09-17 19:06:45 +00:00
BeatSeat 9cdc57d2e7 [Bugfix][MAGI-2] Fix layerwise offload OOM by pointing offload block attrs to 'block' (#7523) (#7540) 
Signed-off-by: BeatSeat <wendavid552@gmail.com>
 2026-09-17 19:02:08 +00:00
dongbo910220 97912de8d8 [Kernel] Fuse LongCat paired Q/K RoPE (#7500) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2026-09-18 02:27:57 +08:00
Anjie HouClaude Opus 5Hongsheng Liu
 		2ab5d1701a [4/N] Unify worker-side MP4 encoding and wire Wan into it (#7048) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-17 23:33:56 +08:00
Chenchao Xu d0755b5716 [Model] Add π0.5 (Pi0.5) VLA model support (#6950) 
Signed-off-by: chenchaox <chenchaox@nvidia.com>
 2026-09-17 05:18:51 -07:00
WeiQing Chen cb439f3e84 [Diffusion][Model] Add direct FastH3 8-Step V2 checkpoint loading (#7610) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-09-17 08:41:52 +00:00
HAAZZZEEEE becfefd440 [Hardware][Ascend] Add Wan2.2 MXFP4 UOS, Smooth and W4A8 fallback (#7210) 
Signed-off-by: HAAZZZEEEE <wang-taicheng@qq.com>
 2026-09-17 03:11:40 +00:00