anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,976 Commits 235 Branches 31 Tags
2a92b71cb96e0585138b5adf28524b217fcd4db7
Commit Graph
606 Commits
Author SHA1 Message Date
Isotr0py 2a92b71cb9 Add FP8 online qunatization support for Qwen-Image-2.1 (#7792) 
Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 22:33:16 +08:00
Tianyu Guo ab4cf24b3e Optimize Qwen-Image 2.1 attention and decode graph reuse (#7790) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 19:37:19 +08:00
Tianyu Guo f930348e79 Preserve Qwen-Image 2.1 condition latent order (#7766) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 18:07:45 +08:00
Tianyu Guo b0a309e5e7 Isolate Qwen-Image 2.1 prefill and decode scheduling (#7770) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 14:11:22 +08:00
Tianyu GuoandIsotr0py a26013145c Align Qwen-Image 2.1 preprocessing, CFG defaults and prefix cache (#7769) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-09-18 13:53:15 +08:00
Tianyu Guo 936c0cc7fe Align Qwen-Image-2.1 VAE normalization with Diffusers (#7768) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 13:51:06 +08:00
Tianyu Guo 93d7a06e7f Use pre-norm text features for Qwen-Image 2.1 (#7765) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 13:50:36 +08:00
Tianyu Guo eb54a3186b Separate Qwen-Image 2.1 batches by condition image layout (#7764) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 13:47:35 +08:00
Isotr0py 3affc7766d Enable VAE tiling for Qwen-Image-2.1 (#5) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix(diffusion): Qwen-Image-2.1 VAE spatial ratio, tiling e2e verification

- AutoencoderKLQwenImage21: derive spatial_compression_ratio from the
  architecture (2 ** len(temperal_downsample) = 16x) instead of trusting
  the checkpoint's scale_factor_spatial=8, which corrupted tiled
  decode/encode geometry (output cropped to half size) and tiling gating.
- pipeline: _read_vae_scale_factor prefers the architecture-derived value
  for the same reason.
- models/qwen_image_21/__init__: lazy exports to break the circular import
  distributed VAE -> package __init__ -> pipeline -> distributed VAE.
- Add VAE unit tests: ratio derivation, tiled vs untiled decode/encode
  consistency, distributed tile_split/exec/merge == tiled_decode.
- e2e matrix (GB200, seed 42, 20 steps) all green; see e2e_out/qwen21_vae/.
- examples: qwen_image_21_edit.py gains --vae-use-tiling.
- docs: mark VAE patch-parallel verified for Qwen-Image-2.1; recipe notes
  the default-tile banding limitation at 2048px+.

* clean tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix cudnn determination

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 10:02:50 +08:00
Isotr0py 4fc3cf8c60 Enable CUDA Graph for DiT decode layers (#4) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* feat(qwen_image_21): opt-in CUDA graph capture for KV-cache decode steps

Decode steps (1..N) recompute only the target image's tokens at a fixed
shape, but the per-request dict KV cache made addresses unstable, so the
step could not be captured. Prefill now copies prefix K/V into persistent
fixed-address buffers (keyed by branch/batch/prefix_len/target_tokens/
dtype/device/backend) and re-points the caller's cache dicts at them;
decode replays a graph captured per key over the full 32-block forward.
Eager fallback (logged) for non-CUDA, SP/ring/TP>1, HSDP, compiled or
offload-hooked blocks, KV-cache quantization, dynamic LoRA wrappers,
padded text masks, unknown keys, and capture failure.

Off by default; opt in via OmniDiffusionConfig.enable_cuda_graph_decode
or the QwenImage21Transformer2DModel constructor kwarg.

E2E (GB200, T2I 1024x1024, 50 steps, CFG 4.0, CUDNN_ATTN):
171.2 -> 142.1 ms/step, 8.70s -> 7.24s total (1.20x), 44.9 dB PSNR
vs eager output.

* clean

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 10:02:39 +08:00
Isotr0py 2912672b2e Initialize QMI pipeline implementation (#2) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-16 15:14:23 +08:00
Zhou TaichangandClaude Opus 5 aff7d64948 [Rebase] Rebase to vLLM 0.29.0 (#7230) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-10 12:08:28 +08:00
Mu GuanLin 4b8dcac681 [Kernel] Enable LTX DiffVAE fusions on SM100 and SM103 (#7350) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-10 03:07:35 +00:00
hyw 6fb7b0a05e [Model] Skip unused frame packing in Wan2.2 S2V (#7155) 
Signed-off-by: hyw <yuweih205@gmail.com>
 2026-09-09 14:34:46 +00:00
NumberWan 038c9b948f [Bugfix] Resolve #6931 hub FA3 on torch 2.13 via kernels 0.16.1 (#7185) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-09-09 10:20:34 +00:00
Bo Li cdd4638b17 [Feature][Diffusion] Expose detailed pipeline timings (#6822) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-09-09 09:58:42 +00:00
Anjie HouandClaude Sonnet 5 ecdd762ff5 [2/N] Add a minimal temporal chunk callback for MiniMax-H3 (#7017) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-09 16:40:32 +08:00
Mu GuanLin ffa991b24f [Perf][Model] Add optimized LTX-2.5 DiffVAE operators (#7308) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-09 08:31:16 +00:00
Mu GuanLin ab56148505 [Bugfix] Make LTX vocoder decoding deterministic (#7231) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-09 06:37:01 +00:00
andyluo7 e0d0879794 [Bugfix][Magi2] Allow import without an active Triton driver (#7239) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-09-08 22:48:08 -07:00
boatman cdf027874c [Feat][OmniVoice]Support Varlen Attn, Request-Batch and Step-Execution (#6408) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-08 18:58:38 -07:00
Harry MellorHongsheng LiuClaude Opus 5
 		fdaf4e31bd Add vLLM-Omni library info to Hugging Face Hub requests (#5381) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-08 10:08:38 -06:00
liangmenghuangandClaude Fable 5.1 7a80b79698 [Bugfix][Model] GR00T-N1.7: honor the per-request seed for flow-matching noise (#7253) 
Signed-off-by: liangmengh <liangmengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-08 15:53:20 +00:00
liangmenghuangandClaude Fable 5.1 d7740336ff [Bugfix][Model] GR00T-N1.7: honor processor_config image pipeline (use_albumentations / crop_fraction / letter_box_transform) to match Isaac-GR00T (#7083) 
Signed-off-by: liangmengh <liangmengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-08 18:59:01 +08:00
Honghan Zhu c086554d1f [Perf]Boogu-Image Skip dense attention masks (#6871) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
 2026-09-08 00:53:20 -07:00
ZacheryAU b1ec136bd7 [Frontend] Add omni benchmark support for image and video endpoints (#4728) 2026-09-08 09:28:21 +08:00
Anjie HouClaude Sonnet 5Hongsheng Liu
 		6942a5b600 [1/N] Stream Wan VAE chunks to the media consumer (#7016) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-08 07:41:13 +08:00
Rahul SteigerandNils Hoffmann 3b335814cc [Bugfix][Cosmos3] Fix distributed Transfer output envelope (#7205) 
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Co-authored-by: Nils Hoffmann <nhoffmann@nvidia.com>
 2026-09-08 07:34:08 +08:00
WeiQing Chen 30d6a0b4e6 [Bugfix][MiniMax-H3] Stabilize keyframe VAE encoding (#7191) 
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
 2026-09-07 21:52:54 +08:00
Mu GuanLin 3204a0b2ad [Model][Perf] Optimize LTX Ulysses sequence parallelism (#7079) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-07 10:48:26 +00:00
Rahul SteigerandRahul Steiger f080f41e03 [Bugfix][Cosmos3] Auto-pad non-divisible Ulysses GEN sequences (#6918) 
Signed-off-by: Rahul Steiger <rsteiger@nvl72d188-T17.cm.cluster>
Co-authored-by: Rahul Steiger <rsteiger@nvl72d188-T17.cm.cluster>
 2026-09-07 08:58:28 +00:00
Rahul SteigerRahul SteigerRahul Steiger
 		677265b64b [Bugfix][Diffusion] Make Cosmos3 transfer image arrays writable (#6915) 
Signed-off-by: Rahul Steiger <rsteiger@nvl72d179-T17.cm.cluster>
Co-authored-by: Rahul Steiger <rsteiger@nvl72d179-T17.cm.cluster>
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
 2026-09-07 01:06:16 -07:00
summerandAlicia 4b81f692d1 [Bugfix][Diffusion] Canonicalize cache_backend=None to "none" (#7041) 
Signed-off-by: summer <128961079+zhang-keliang@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-09-07 15:56:51 +08:00
BruceLoveDecimal 5cd925631c [Feature][LingBot World]stepwise execution (#6844) 
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
 2026-09-07 13:38:23 +08:00
NumberWanandAlicia 7f32c326c6 [BugFix][Qwen-Image-Edit] restore txt_seq_lens RoPE width after #3588 regression (#5586) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-09-07 12:10:47 +08:00
Zeyu Huang | 黃澤宇 2df95de421 [Core][Diffusion] Refactor prompt updates into a modality-ready interaction framework (#6294) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-09-07 11:40:04 +08:00
SYLARandhsliuustc0106 14d2d00536 [Bugfix] Fix Hunyuan Image-3 expert mapping contract (#7021) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-06 21:43:37 +08:00
汪志鹏andClaude Opus 5 af74a5a155 [Feature][MiniMax-H3] Support the full LightX2V Turbo LoRA matrix (#7062) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-06 20:23:36 +08:00
DMAN 06541d1f52 [Model] FP8 Support For Boogu-Image (#6925) 
Signed-off-by: Dmaner <2663515256@qq.com>
 2026-09-06 04:48:38 +00:00
prettygirlisnotme 32370968d0 [Model] Add Boogu-Image Turbo native DMD path (#6699) 
Signed-off-by: Dragonborn <1078757180@qq.com>
 2026-09-06 08:44:15 +08:00
SYLAR ea3cf99894 [diffusion][feature] Add component-selective offload policies (#5929) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-09-06 05:40:09 +08:00
Hongsheng Liu d4096e01fe [Model] Add native MAGI-2 Preview diffusion support (#5918) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-05 15:32:30 +08:00
vOv 13c617a140 [Model] Add sequence parallelism to SANA-Video 2B (#5940) 
Signed-off-by: cr-gao <gaochenrui@sjtu.edu.cn>
 2026-09-05 15:22:53 +08:00
TriggerandGao Han ea14b04e9f [Bugfix][BAGEL] Restore original denoise step count (#7049) 
Signed-off-by: Trigger <Meng.Bo.Wang@outlook.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-04 17:05:05 +00:00
SYLARandhsliuustc0106 22cefe04b4 [Bugfix] Update Hub kernel dependency floor (#7020) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-04 22:15:55 +08:00
MikeyDong1 41f4f9d8c9 [Perf][DreamZero] Remove redundant CUDA compile guard (#6983) 
Signed-off-by: MikeyDong1 <xdong-1219@outlook.com>
 2026-09-04 09:01:05 -04:00
Shenglei FuandClaude Opus 4.8 0e903e9695 [Diffusion] Enable request-level batching for Boogu-Image (T2I) (#6968) 
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-09-04 16:27:51 +08:00
Mu GuanLin 175dec11be [Perf][Model] Optimize LTX video output transport (#7000) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-04 06:47:28 +00:00
Trigger e43c787544 [Model] Add BAGEL step execution and continuous batching (#6359) 2026-09-04 01:36:11 +00:00
Zeng Chuang b5e898f295 [Feat] Add support and testcase for HunyuanImage-3.0-Instruct-Distil (#4048) 
Signed-off-by: zengchuang <zengchuang3@huawei.com>
Signed-off-by: zengchuang-hw <zengchuang1005@gmail.com>
 2026-09-03 02:43:13 +00:00