tzhouam and Claude
25057ad8df
fix(diffusion): gate meta-safe weight move to online quant only (fix HSDP regression)
...
The previous commit (93730cfd) replaced the blanket module.to() in
_process_weights_after_loading with an unconditional per-parameter move.
That broke the FSDP/HSDP path: after apply_hsdp_to_model the params are
FSDP-sharded DTensors whose storage cannot be cross-device reassigned via
`param.data = param.data.to(cpu)`, raising at load time:
RuntimeError: Attempted to set the storage of a tensor on device
"cuda:0" to a storage on different device "cpu". This is no longer
allowed; the devices must match.
Regressed nightly test_zimage[layerwise_hsdp] (Diffusion X2I Function
Test with L4), which passed pre-fix. The FP8 online-quant variants it was
meant to fix (qwen/bagel fp8, parallel_*_fp8_*) all passed.
Meta-device params only ever arise from online quant (uses_meta_device),
so gate the meta-safe per-parameter move to that case and keep the
original FSDP/HSDP-aware whole-module module.to() for everything else.
Online quant initializes on the accelerator, so its params are normally
already on the target device (the per-param loop is a no-op move) — the
only behavioral change vs. the original is skipping meta stragglers.
Co-Authored-By: Claude <noreply@anthropic.com >
2026-07-11 14:21:16 +08:00
tzhouam and Claude
f74e98119e
fix(diffusion): make FP8 online-quant weight loading meta-device safe
...
Newer upstream vLLM (65a7b4628409, 585 commits past the v0.24.0 tag that
vllm-omni main tracks) reworked online quantization: online-quant linear
methods now declare `uses_meta_device=True` and create weights on the
`meta` device, materializing them just-in-time via the layerwise
online-process loader as each layer's weights finish loading.
The diffusion loader's `_process_weights_after_loading` did a blanket
`module.to(target_device)`, which raises
`NotImplementedError: Cannot copy out of meta tensor; no data!` on any
layer whose weights were not fully materialized during load (padded /
partially-loaded "straggler" layers), crashing diffusion worker startup.
This regressed the nightly diffusion FP8 tests on dev/vllm-align (all
green on main nightly): Quantization Test (H100 + L4,
test_single_stage_qwen_image_fp8 / test_bagel_fp8_generates_image) and
the Qwen-Image Multi-GPU vae_patch_parallel_2 case (fp8).
Mirror upstream's loader contract:
- Materialize online-quant stragglers via finalize_layerwise_processing()
before processing (imported lazily, only when online quant is in use,
so older vLLM is unaffected). model_config=None is safe: it is only
used for vLLM Attention/MLAAttention layers, which diffusion DiT models
do not have.
- Replace the blanket module.to() with a per-parameter move that mirrors
upstream's device_loading_context: move only real (non-meta) params to
the target device for processing and restore them afterward, leaving
meta params for the quant method / finalize step.
Co-Authored-By: Claude <noreply@anthropic.com >
2026-07-11 14:21:16 +08:00
Zhou Taichang
a560ed184d
[Rebase] Rebase to vllm v0.24.0 ( #4709 )
2026-06-30 19:41:06 +08:00
Nick Cao and Claude
6646a66d41
[BugFix] Use ModuleDiscovery for HSDP DiT sharding to support nested pipelines ( #3456 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-06-23 02:54:14 +00:00
Kwokho Tsui and Kwokho Tsui
7e152ca64b
[Bugfix] Fix HSDP + FP8 online quantization compatibility ( #4494 )
...
Signed-off-by: Kwokho Tsui <kh_tsui@163.com >
Co-authored-by: Kwokho Tsui <kh_tsui@163.com >
2026-06-19 15:06:27 +08:00
Kwokho Tsui and Kwokho
3bce5e2e3a
[Bugfix] Cosmos3: fix HSDP dtype mismatch on time_embedder via _hsdp_ignored_modules ( #4452 )
...
Signed-off-by: Kwokho <kh_tsui@163.com >
Co-authored-by: Kwokho <kh_tsui@163.com >
2026-06-16 04:06:14 +00:00
Alex Brooks
7dcbeaea03
[Core] Cleanup DiffusersPipelineLoader ( #1932 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-06-09 11:55:32 +08:00
SYLAR
29b18ea298
[Quant] ModelOpt FP8 for Wan2.2 & HunyuanVideo-1.5 video-gen ( #3305 )
...
Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com >
2026-06-08 02:53:06 +08:00
Zhou Taichang
af02713952
[Bugfix] harden diffusion model prefetch against transformers v5 shard-resolution race ( #4076 )
2026-06-04 12:14:25 +08:00
+1
f92d84fcf9
[Rebase] Rebase to vllm releases/v0.22.0 ( #3891 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: david6666666 <530634352@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: nainiu258 <cperfect02@163.com >
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com >
Co-authored-by: Zheng Wengang <zwg0606@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-06-01 11:08:12 +08:00
knlnguyen1802 and Samit
5377108438
[minor, fix] Allow passing class interface as custom pipeline argument ( #2973 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-05-29 23:39:38 +08:00
knlnguyen1802 and Samit
5eac50f22c
[bugfix, rl] Fix sleep do not release full memory in custom pipeline ( #3818 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Co-authored-by: Samit <285365963@qq.com >
2026-05-28 22:27:27 +08:00
Chendi.Xue
1903cee56a
[MXFP8][XPU] enable mxfp8 using vLLM main repo method ( #3782 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-05-28 02:47:22 +00:00
baonudesifeizhai
076c34abfb
（Phase 2）Add ModelOpt mixed FP8/NVFP4 support for image generation ( #3570 )
...
Signed-off-by: roG0d <baonudesifeizhai@gmail.com >
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com >
2026-05-22 10:39:10 +08:00
Canlin Guo and Hongsheng Liu
11c4c7f0ff
[Diffusion][Attention] Support per-role attention backend via CLI ( #2681 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-10 11:59:05 +08:00
baonudesifeizhai and roG0d
c4a0990044
（Phase 1）Add ModelOpt FP8 auto-detect support for diffusion checkpoints #2709 ( #2913 )
...
Signed-off-by: roG0d <rodgarcas98@gmail.com >
Signed-off-by: roG0d <baonudesifeizhai@gmail.com >
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com >
Co-authored-by: roG0d <rodgarcas98@gmail.com >
2026-05-09 16:55:05 +08:00
lsyyyyy and 汪志鹏
32663f21d5
[Feat] support hsdp for Bagel ( #3150 )
...
Signed-off-by: siyuan.lei <siyuanlei37@gmail.com >
Signed-off-by: lsyyyyy <siyuanlei37@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-05-08 15:20:42 +08:00
0a8204cb81
[Quantization] Redo Z-Image text encoder FP8 online quantization ( #3279 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Isotr0py <2037008807@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-05-06 16:37:36 +00:00
Hongsheng Liu
7fc528f7a9
Revert "[Quantization] Enable FP8 online quantization for Z-image text encoder" ( #3272 )
2026-04-30 15:08:17 +08:00
Isotr0py
ac66282746
[Quantization] Enable FP8 online quantization for Z-image text encoder ( #1338 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-04-30 14:12:01 +08:00
Zhou Taichang and Yuanheng Zhao
7f2b1e9386
[Rebase] Rebase to vllm 0.20.0 ( #3232 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com >
2026-04-30 09:56:59 +12:00
Zeyu Huang | 黃澤宇
d8cc7a0409
[feat]: General diffusers adapter backend to run diffusion models ( #2724 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
2026-04-23 00:07:18 +08:00
Yi Liu and Hongsheng Liu
c1d2dcc3ac
[AutoRound] Add offline quantized W4A16 model support ( #1777 )
...
Signed-off-by: yiliu30 <yi4.liu@intel.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-02 13:57:53 +08:00
SYLAR
6b93459826
[Core] Unified quantization framework ( #1764 )
2026-03-24 18:56:55 +08:00
WeiQing Chen
d6b55b9196
[Quantization] feat: add qwen-image gguf adapter ( #1755 )
...
Signed-off-by: David Chen <530634352@qq.com >
2026-03-23 13:24:54 +00:00
Chendi.Xue
28aee51a59
[FP8] enable hunyuan-image-3 diffusion model with fp8 online quant ( #1935 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-03-22 13:02:16 +08:00
Lancer
af012babf8
[Bugfix] Fix cpu offload and quantization compatibility ( #1473 )
...
Signed-off-by: Lancer <maruixiang6688@gmail.com >
2026-03-14 18:55:30 +00:00
Samit
22a51a7cdb
[Feature][Wan2.2] Speed up diffusion model startup by multi-thread weight loading ( #1504 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
2026-03-02 14:53:25 +00:00
Isotr0py
1f862ce04f
[Chore] Cleanup dead code in GGUF DiT code path ( #1533 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-02-27 07:17:27 +00:00
0b884ebc6c
[Feature]: Native GGUF Quantization Support for DiT ( #1285 )
...
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Isotr0py <2037008807@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-27 11:39:14 +08:00
Canlin Guo and Hongsheng Liu
f53152d340
[Feature] Support HSDP for diffusion models ( #1339 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-26 21:31:10 +08:00
knlnguyen1802
9994594782
[Misc] Support WorkerWrapperBase and CustomPipeline for Diffusion Worker ( #764 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
2026-02-12 19:23:51 +08:00
SYLAR
830d261f9d
[Feature]: FP8 Quantization Support for DiT ( #1034 )
...
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-02-11 21:01:35 +08:00
5037af1033
[Feature] Diffusion LoRA Adapter Support (PEFT compatible) for vLLM alignment ( #758 )
...
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk >
Signed-off-by: Andy Zhou <jzhoubc@connect.ust.hk >
Signed-off-by: dongbo910220 <1275604947@qq.com >
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: dongbo910220 <1275604947@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-01-27 10:22:40 +08:00
8939eb835d
[Rebase] Rebase to vllm 0.12.0 ( #335 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-19 16:42:02 +08:00
Isotr0py
059f45e48d
[Diffusion] Refactor diffusion models weights loading ( #157 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-06 11:12:13 +00:00
