Tianyu Guo
3ea9e60501
Fix diffusion API initialization ( #7868 )
...
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
2026-09-20 12:08:50 +08:00
Tianyu Guo
9dabaf8b80
Default Qwen-Image 2.1 CFG scale to 1.0 ( #7859 )
...
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
2026-09-20 11:51:35 +08:00
Isotr0py
5b540f55d3
Fix decode cuda graph for Qwen-Image-2.1 model-level cpu offload ( #7801 )
2026-09-19 00:10:53 +08:00
Isotr0py
47e6acc1e3
Add FP8 KV cache support for Qwen-Image-2.1 DiT ( #7797 )
2026-09-19 00:06:58 +08:00
Isotr0py
2a92b71cb9
Add FP8 online qunatization support for Qwen-Image-2.1 ( #7792 )
...
Signed-off-by: Isotr0py <mozf@inferact.ai >
2026-09-18 22:33:16 +08:00
Tianyu Guo
ab4cf24b3e
Optimize Qwen-Image 2.1 attention and decode graph reuse ( #7790 )
...
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
2026-09-18 19:37:19 +08:00
Tianyu Guo
f930348e79
Preserve Qwen-Image 2.1 condition latent order ( #7766 )
...
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
2026-09-18 18:07:45 +08:00
Tianyu Guo
b0a309e5e7
Isolate Qwen-Image 2.1 prefill and decode scheduling ( #7770 )
...
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
2026-09-18 14:11:22 +08:00
Tianyu Guo and Isotr0py
a26013145c
Align Qwen-Image 2.1 preprocessing, CFG defaults and prefix cache ( #7769 )
...
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-09-18 13:53:15 +08:00
Tianyu Guo
936c0cc7fe
Align Qwen-Image-2.1 VAE normalization with Diffusers ( #7768 )
...
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
2026-09-18 13:51:06 +08:00
Tianyu Guo
93d7a06e7f
Use pre-norm text features for Qwen-Image 2.1 ( #7765 )
...
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
2026-09-18 13:50:36 +08:00
Tianyu Guo
eb54a3186b
Separate Qwen-Image 2.1 batches by condition image layout ( #7764 )
...
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
2026-09-18 13:47:35 +08:00
Isotr0py
3affc7766d
Enable VAE tiling for Qwen-Image-2.1 ( #5 )
...
* init
Signed-off-by: Isotr0py <mozf@inferact.ai >
* fix
Signed-off-by: Isotr0py <mozf@inferact.ai >
* test(diffusion): add Qwen-Image-2.1 unit tests
Signed-off-by: Isotr0py <mozf@inferact.ai >
* fix(diffusion): Qwen-Image-2.1 VAE spatial ratio, tiling e2e verification
- AutoencoderKLQwenImage21: derive spatial_compression_ratio from the
architecture (2 ** len(temperal_downsample) = 16x) instead of trusting
the checkpoint's scale_factor_spatial=8, which corrupted tiled
decode/encode geometry (output cropped to half size) and tiling gating.
- pipeline: _read_vae_scale_factor prefers the architecture-derived value
for the same reason.
- models/qwen_image_21/__init__: lazy exports to break the circular import
distributed VAE -> package __init__ -> pipeline -> distributed VAE.
- Add VAE unit tests: ratio derivation, tiled vs untiled decode/encode
consistency, distributed tile_split/exec/merge == tiled_decode.
- e2e matrix (GB200, seed 42, 20 steps) all green; see e2e_out/qwen21_vae/.
- examples: qwen_image_21_edit.py gains --vae-use-tiling.
- docs: mark VAE patch-parallel verified for Qwen-Image-2.1; recipe notes
the default-tile banding limitation at 2048px+.
* clean tests
Signed-off-by: Isotr0py <mozf@inferact.ai >
* fix cudnn determination
Signed-off-by: Isotr0py <mozf@inferact.ai >
---------
Signed-off-by: Isotr0py <mozf@inferact.ai >
2026-09-18 10:02:50 +08:00
Isotr0py
4fc3cf8c60
Enable CUDA Graph for DiT decode layers ( #4 )
...
* init
Signed-off-by: Isotr0py <mozf@inferact.ai >
* fix
Signed-off-by: Isotr0py <mozf@inferact.ai >
* test(diffusion): add Qwen-Image-2.1 unit tests
Signed-off-by: Isotr0py <mozf@inferact.ai >
* feat(qwen_image_21): opt-in CUDA graph capture for KV-cache decode steps
Decode steps (1..N) recompute only the target image's tokens at a fixed
shape, but the per-request dict KV cache made addresses unstable, so the
step could not be captured. Prefill now copies prefix K/V into persistent
fixed-address buffers (keyed by branch/batch/prefix_len/target_tokens/
dtype/device/backend) and re-points the caller's cache dicts at them;
decode replays a graph captured per key over the full 32-block forward.
Eager fallback (logged) for non-CUDA, SP/ring/TP>1, HSDP, compiled or
offload-hooked blocks, KV-cache quantization, dynamic LoRA wrappers,
padded text masks, unknown keys, and capture failure.
Off by default; opt in via OmniDiffusionConfig.enable_cuda_graph_decode
or the QwenImage21Transformer2DModel constructor kwarg.
E2E (GB200, T2I 1024x1024, 50 steps, CFG 4.0, CUDNN_ATTN):
171.2 -> 142.1 ms/step, 8.70s -> 7.24s total (1.20x), 44.9 dB PSNR
vs eager output.
* clean
Signed-off-by: Isotr0py <mozf@inferact.ai >
---------
Signed-off-by: Isotr0py <mozf@inferact.ai >
2026-09-18 10:02:39 +08:00
Isotr0py
2912672b2e
Initialize QMI pipeline implementation ( #2 )
...
* init
Signed-off-by: Isotr0py <mozf@inferact.ai >
* fix
Signed-off-by: Isotr0py <mozf@inferact.ai >
---------
Signed-off-by: Isotr0py <mozf@inferact.ai >
2026-09-16 15:14:23 +08:00
herotai214
bbac4df7d3
[Refactor] P0.2: Migrate API server helpers out of api_server ( #5453 )
...
Signed-off-by: herotai214 <herotai214@gmail.com >
2026-09-10 15:03:21 +08:00
Zhou Taichang and Claude Opus 5
aff7d64948
[Rebase] Rebase to vLLM 0.29.0 ( #7230 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-10 12:08:28 +08:00
Sy03
333f14e97e
[Bugfix][MiniCPM-o] Align structured chat content with native omni rendering ( #7344 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-09-10 11:38:11 +08:00
Mu GuanLin
4b8dcac681
[Kernel] Enable LTX DiffVAE fusions on SM100 and SM103 ( #7350 )
...
Signed-off-by: mglyn <1203789601@qq.com >
2026-09-10 03:07:35 +00:00
Gao Han and amy-why-3459
6e63ea875e
[Bugfix] Allow LLM replicas on different GPUs to initialize concurrently ( #7292 )
...
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-09-10 09:54:20 +08:00
Joshna-Medisetty
51cf142bc1
[Bugfix][XPU] Restore N-D output shape for W8A16 FP8 linear ( #7301 )
...
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com >
Signed-off-by: Joshna-Medisetty <joshna.medisetty@intel.com >
2026-09-09 17:11:31 -05:00
Sy03
b02862beee
[Model] Optimize MOSS-TTS Local batched execution and streaming codec ( #7202 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-09-09 13:02:12 -07:00
hyw
6fb7b0a05e
[Model] Skip unused frame packing in Wan2.2 S2V ( #7155 )
...
Signed-off-by: hyw <yuweih205@gmail.com >
2026-09-09 14:34:46 +00:00
0e602509aa
[BugFix] Add field validators for /v1/audio/generate request ( #4741 )
...
Signed-off-by: Shaun Walsh <shaunwalsh24@gmail.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: Nick Cao <ncao@redhat.com >
2026-09-09 13:38:42 +00:00
kunkun
2f1845fa4e
fix: reserve Qwen3-Omni NVFP4 backend fix ( #7200 )
...
Signed-off-by: kunkunblueberry <1833921874@qq.com >
2026-09-09 20:35:16 +08:00
Zheng Wengang and amy-why-3459
7c1b1e4189
[Bugfix][Engine] Group overlapping device stages into one sequential init component ( #7328 )
...
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-09-09 20:25:41 +08:00
zyz111222
6e79d582e3
[Bugfix][Ascend] fix npu 310/a5 bugs ( #6685 )
...
Signed-off-by: zouyizhou <zouyizhou@huawei.com >
2026-09-09 19:03:09 +08:00
NumberWan
038c9b948f
[Bugfix] Resolve #6931 hub FA3 on torch 2.13 via kernels 0.16.1 ( #7185 )
...
Signed-off-by: NumberWan <wantszkin2003@gmail.com >
2026-09-09 10:20:34 +00:00
Bo Li
cdd4638b17
[Feature][Diffusion] Expose detailed pipeline timings ( #6822 )
...
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com >
2026-09-09 09:58:42 +00:00
Anjie Hou and Claude Sonnet 5
ecdd762ff5
[2/N] Add a minimal temporal chunk callback for MiniMax-H3 ( #7017 )
...
Signed-off-by: specture724 <specture724@gmail.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-09-09 16:40:32 +08:00
Mu GuanLin
ffa991b24f
[Perf][Model] Add optimized LTX-2.5 DiffVAE operators ( #7308 )
...
Signed-off-by: mglyn <1203789601@qq.com >
2026-09-09 08:31:16 +00:00
Mu GuanLin
ab56148505
[Bugfix] Make LTX vocoder decoding deterministic ( #7231 )
...
Signed-off-by: mglyn <1203789601@qq.com >
2026-09-09 06:37:01 +00:00
NATURE
5b002727b0
[Core] Split Omni connector model runner mixin ( #6903 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-09-09 13:49:43 +08:00
andyluo7
e0d0879794
[Bugfix][Magi2] Allow import without an active Triton driver ( #7239 )
...
Signed-off-by: andyluo7 <andy.luo@amd.com >
2026-09-08 22:48:08 -07:00
xutianle
968f3e379c
[Bugfix][CosyVoice3] Resolve hash snapshot pipeline ( #6896 )
...
Signed-off-by: xutianle <xutianle@fudan.edu.cn >
2026-09-08 20:33:38 -07:00
Allen Wu and TRAE CLI
f2e30f46b6
[Bugfix][TTS] Isolate shared Higgs v3 reference encode from request cancellation ( #7076 )
...
Signed-off-by: Allen Wu <allenwu2795@gmail.com >
Co-authored-by: TRAE CLI <traecli@bytedance.com >
2026-09-08 20:32:05 -07:00
Jim Ban
358b7d79c1
[Bugfix][Frontend] Enforce image pixel limits for video input references ( #6963 )
...
Signed-off-by: BANANASJIM <bananasjim1@gmail.com >
2026-09-08 20:31:54 -07:00
Ziming Wang
43b697541a
[Bugfix][Frontend] Clear the P0 multimodal cache through the renderer ( #7003 )
...
Signed-off-by: ZenAlexa <zimingwang945@gmail.com >
2026-09-08 20:31:42 -07:00
2bae550c04
[Frontend] Opt-in WebSocket TTS split_granularity and session seed ( #7046 )
...
Signed-off-by: rk9595 <rakesh.kariya@somaiya.edu >
Signed-off-by: Rakesh Kariya <rakesh.kariya@somaiya.edu >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-08 19:15:59 -07:00
zhengjia and Cursor
81c1c460b3
[Bugfix][Frontend] Accept the msgpack-numpy package's numpy markers on the OpenPI endpoint ( #6051 )
...
Signed-off-by: zjli2013 <leezhengjiang@126.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-08 19:15:48 -07:00
NancyFyong and NancyFyong
e74fe3c6e8
[Model] Add Audio8 TTS Preview 0.6B (DualAR, 44.1 kHz codec) ( #6157 )
...
Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com >
Co-authored-by: NancyFyong <NancyFyong@users.noreply.github.com >
2026-09-08 19:00:25 -07:00
boatman
cdf027874c
[Feat][OmniVoice]Support Varlen Attn, Request-Batch and Step-Execution ( #6408 )
...
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com >
2026-09-08 18:58:38 -07:00
eval
d7678e2bb3
[Bugfix][MiniCPM-o] Fix the audio_embeds input path ( #5730 )
...
Signed-off-by: eval-dev <0xe5bca0@gmail.com >
Signed-off-by: eval <74645252+eval-dev@users.noreply.github.com >
2026-09-08 18:41:42 -07:00
Zhichao Zhang
5078c7ad66
[Bugfix][MiMo-Audio] Align code2wav decode with tokenizer device ( #6539 )
...
Signed-off-by: chaosansui <zzc15560846421@163.com >
Signed-off-by: Zhichao Zhang <60429419+smartDream-chao@users.noreply.github.com >
2026-09-08 18:38:11 -07:00
Anjie Hou and Claude Opus 5
b6f4df320d
[Refactor][Diffusion] Resolve offload topology through one plan resolver ( #7209 )
...
Signed-off-by: specture724 <specture724@gmail.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-09 08:17:38 +08:00
Tianyao Wu
b3dd45874a
[Bugfix] Build the forced-aligner prompt without a chat template (word timestamps one bin late) ( #7240 )
...
Signed-off-by: Tianyao Wu <rayroy31@gmail.com >
2026-09-08 19:27:38 +00:00
Lei Ke and KrystalRay
5e4baea64d
[Bugfix][NPU] Limit MiniMax H3 modulation grid size ( #6794 )
...
Signed-off-by: KrystalRay <keeleiray@gmail.com >
Co-authored-by: KrystalRay <keeleiray@gmail.com >
2026-09-08 09:08:46 -07:00
fdaf4e31bd
Add vLLM-Omni library info to Hugging Face Hub requests ( #5381 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-08 10:08:38 -06:00
liangmenghuang and Claude Fable 5.1
7a80b79698
[Bugfix][Model] GR00T-N1.7: honor the per-request seed for flow-matching noise ( #7253 )
...
Signed-off-by: liangmengh <liangmengh@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-08 15:53:20 +00:00
Hongsheng Liu
7422a85cc7
[Bugfix] Skip HWR store-size scans when no limit is configured ( #7131 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-09-08 22:54:49 +08:00