anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,228 Commits 235 Branches 31 Tags
44ea27c8094095bbffd88fa3befdfaa55ba4bc50
Commit Graph
27 Commits
Author SHA1 Message Date
Isotr0py be8a72f0a6 Stack torch.compile with CUDA graph decode instead of choosing one 
Regionally compile the DiT blocks even when enable_cuda_graph_decode is set
and let graph capture record the compiled (fused) kernels; inductor's own
cudagraphs stay off so the two graph layers never nest. Measured on GB200
(1024x1024, 50 steps, seed 42, BF16): combo 3.04s vs compile-only 3.2s,
graph-only 4.0s, eager 4.3s per image; 44.5-45.6 dB PSNR vs eager.
 2026-09-22 08:58:32 +00:00
Isotr0py 3e274a0024 Merge remote-tracking branch 'origin/main' into qwen-image-2.1 
# Conflicts:
#	tests/entrypoints/openai_api/test_image_server.py
#	vllm_omni/entrypoints/openai/api_server.py
 2026-09-22 01:23:45 +00:00
Sy03 60a933b9ab [Model] Default Qwen3-TTS to experimental Model Runner V2 (#7930) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-21 18:56:06 +00:00
Isotr0py 3d6a1c4d9a Address review: examples policy, SPDX headers, recipe docs, test coverage 
- Register QwenImage21Pipeline in model_extras _EXTRA_SPECS with an
  image-to-image prompt builder that forwards height/width
- Rename run_qwen_image_21_edit.sh to run_qwen_image_21.sh
- Fix SPDX-FileCopyrightText headers
- Recipe: document mixed-KV-phase admission, add head-vs-diffusers A/B
  comparison section (measurements pending)
- Add test_registry.py and test_pipeline.py (preprocess, prepare_latents
  ordering, mixed-phase _assemble_kv_cache, post-process); extend
  test_batch_compatibility.py
 2026-09-21 08:36:42 +00:00
Tianyu Guo 9dabaf8b80 Default Qwen-Image 2.1 CFG scale to 1.0 (#7859) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-20 11:51:35 +08:00
Isotr0py 47e6acc1e3 Add FP8 KV cache support for Qwen-Image-2.1 DiT (#7797) 2026-09-19 00:06:58 +08:00
Isotr0py 2a92b71cb9 Add FP8 online qunatization support for Qwen-Image-2.1 (#7792) 
Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 22:33:16 +08:00
Tianyu Guo ab4cf24b3e Optimize Qwen-Image 2.1 attention and decode graph reuse (#7790) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 19:37:19 +08:00
Tianyu Guo b0a309e5e7 Isolate Qwen-Image 2.1 prefill and decode scheduling (#7770) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 14:11:22 +08:00
Isotr0py 3affc7766d Enable VAE tiling for Qwen-Image-2.1 (#5) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix(diffusion): Qwen-Image-2.1 VAE spatial ratio, tiling e2e verification

- AutoencoderKLQwenImage21: derive spatial_compression_ratio from the
  architecture (2 ** len(temperal_downsample) = 16x) instead of trusting
  the checkpoint's scale_factor_spatial=8, which corrupted tiled
  decode/encode geometry (output cropped to half size) and tiling gating.
- pipeline: _read_vae_scale_factor prefers the architecture-derived value
  for the same reason.
- models/qwen_image_21/__init__: lazy exports to break the circular import
  distributed VAE -> package __init__ -> pipeline -> distributed VAE.
- Add VAE unit tests: ratio derivation, tiled vs untiled decode/encode
  consistency, distributed tile_split/exec/merge == tiled_decode.
- e2e matrix (GB200, seed 42, 20 steps) all green; see e2e_out/qwen21_vae/.
- examples: qwen_image_21_edit.py gains --vae-use-tiling.
- docs: mark VAE patch-parallel verified for Qwen-Image-2.1; recipe notes
  the default-tile banding limitation at 2048px+.

* clean tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix cudnn determination

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 10:02:50 +08:00
Isotr0py 2912672b2e Initialize QMI pipeline implementation (#2) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-16 15:14:23 +08:00
chi030303 c66df06c0f [Doc] Qwen3-TTS: add 0.6B on 1x A100 40GB (#7289) 
Signed-off-by: chi030303 <106855944+chi030303@users.noreply.github.com>
 2026-09-09 07:56:14 +00:00
GXINandHongsheng Liu 3ab582c993 [Bugfix][Qwen3-TTS] Reject task mismatches before EngineCore dispatch (#6113) 
Signed-off-by: XIN GAO <1037396230@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-02 11:16:55 -07:00
Sy03 3adee400a0 [Bugfix][Qwen3-TTS] Handle codec generations that exhaust their token budget (#6728) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-08-29 08:59:05 -07:00
akshatvishuandHongsheng Liu e96d3fe2a6 [ROCm][Recipe] Add MI300X coverage for verified recipes (#6207) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-17 13:58:51 +08:00
faceless void 11f6a0b72e Make speech streaming default to SSE (#4679) 
Signed-off-by: syd520zy <529477025@qq.com>
 2026-06-30 02:17:24 +00:00
Zening Chen 0383fde38a [Doc] Qwen3-TTS: add 0.6B on 1x RTX 4090 and fix broken links (#4026) 
Signed-off-by: Zening Chen <59945561+zeningc@users.noreply.github.com>
Signed-off-by: zeningc <zening.chen@yahoo.com>
 2026-06-18 17:31:36 +08:00
Honghan ZhuClaude Opus 4.8zhuhonghan
 		679cbe226a [Doc] Qwen image 2512 recipe (#4236) 
Signed-off-by: dph97 <nenbaying@163.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: zhuhonghan <z00806815@china.huawei.com>
 2026-06-17 17:04:58 +08:00
baonudesifeizhai 076c34abfb （Phase 2）Add ModelOpt mixed FP8/NVFP4 support for image generation (#3570) 
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
 2026-05-22 10:39:10 +08:00
Shin da53618793 [Recipe] Qwen/Qwen-Image-Edit (#3684) 
Signed-off-by: yixiaoer <shin@yixiaoer.sg>
 2026-05-19 19:56:38 +08:00
bjf-frzandbjfwhite 1fa734419e [Refactor]Rename diffusion benchmark backend to endpoint (#3137) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Signed-off-by: bjfwhite <baijingfan1@huawei.com>
Co-authored-by: bjfwhite <baijingfan1@huawei.com>
 2026-05-19 11:30:33 +08:00
Hongsheng Liuanddeepseek-v4-pro c0e132d973 [Doc] Reorganize available recipes into a table (#3671) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Co-authored-by: deepseek-v4-pro <noreply@anthropic.com>
 2026-05-18 00:02:34 +08:00
chzhang2021Hongsheng LiuChonghao Zhang
 		aca4b7d65c Add Qwen3 TTS Model recipe (#3130) 
Signed-off-by: Chonghao Zhang <chzhang2021@gmail.com>
Signed-off-by: chzhang2021 <chzhang2021@gmail.com>
Signed-off-by: Chonghao Zhang <chonghaoz@meta.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Chonghao Zhang <chonghaoz@meta.com>
 2026-05-08 13:08:39 -07:00
Ting FUasukaqaq-sjaderHongsheng LiuGao Han
 		e49fbd8a2d [Feat] DiffusionEngine Support async batch infer (#2729) 
Signed-off-by: Semmer <semmer@live.cn>
Signed-off-by: jader <yjader@foxmail.com>
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: jader <yjader@foxmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-05-06 09:47:13 +12:00
amy-why-3459 0470e96771 [Docs] Modify Qwen3-Omni's recipe (#3109) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-27 21:41:13 +08:00
wuhang 2e067bbc24 [Docs] CLI Docs updates (#2978) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-04-24 11:25:22 +08:00
Hongsheng LiuandCopilot 5a68c217e2 [skip ci][recipe] draft vllm-omni recipes (#2646) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-04-18 10:29:17 +08:00