tzhouam and Claude
f74e98119e
fix(diffusion): make FP8 online-quant weight loading meta-device safe
...
Newer upstream vLLM (65a7b4628409, 585 commits past the v0.24.0 tag that
vllm-omni main tracks) reworked online quantization: online-quant linear
methods now declare `uses_meta_device=True` and create weights on the
`meta` device, materializing them just-in-time via the layerwise
online-process loader as each layer's weights finish loading.
The diffusion loader's `_process_weights_after_loading` did a blanket
`module.to(target_device)`, which raises
`NotImplementedError: Cannot copy out of meta tensor; no data!` on any
layer whose weights were not fully materialized during load (padded /
partially-loaded "straggler" layers), crashing diffusion worker startup.
This regressed the nightly diffusion FP8 tests on dev/vllm-align (all
green on main nightly): Quantization Test (H100 + L4,
test_single_stage_qwen_image_fp8 / test_bagel_fp8_generates_image) and
the Qwen-Image Multi-GPU vae_patch_parallel_2 case (fp8).
Mirror upstream's loader contract:
- Materialize online-quant stragglers via finalize_layerwise_processing()
before processing (imported lazily, only when online quant is in use,
so older vLLM is unaffected). model_config=None is safe: it is only
used for vLLM Attention/MLAAttention layers, which diffusion DiT models
do not have.
- Replace the blanket module.to() with a per-parameter move that mirrors
upstream's device_loading_context: move only real (non-meta) params to
the target device for processing and restore them afterward, leaving
meta params for the quant method / finalize step.
Co-Authored-By: Claude <noreply@anthropic.com >
2026-07-11 14:21:16 +08:00
Zhou Taichang
a560ed184d
[Rebase] Rebase to vllm v0.24.0 ( #4709 )
2026-06-30 19:41:06 +08:00
Nick Cao and Claude
6646a66d41
[BugFix] Use ModuleDiscovery for HSDP DiT sharding to support nested pipelines ( #3456 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-06-23 02:54:14 +00:00
Kwokho Tsui and Kwokho Tsui
7e152ca64b
[Bugfix] Fix HSDP + FP8 online quantization compatibility ( #4494 )
...
Signed-off-by: Kwokho Tsui <kh_tsui@163.com >
Co-authored-by: Kwokho Tsui <kh_tsui@163.com >
2026-06-19 15:06:27 +08:00
Kwokho Tsui and Kwokho
3bce5e2e3a
[Bugfix] Cosmos3: fix HSDP dtype mismatch on time_embedder via _hsdp_ignored_modules ( #4452 )
...
Signed-off-by: Kwokho <kh_tsui@163.com >
Co-authored-by: Kwokho <kh_tsui@163.com >
2026-06-16 04:06:14 +00:00
Alex Brooks
7dcbeaea03
[Core] Cleanup DiffusersPipelineLoader ( #1932 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-06-09 11:55:32 +08:00
SYLAR
29b18ea298
[Quant] ModelOpt FP8 for Wan2.2 & HunyuanVideo-1.5 video-gen ( #3305 )
...
Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com >
2026-06-08 02:53:06 +08:00
knlnguyen1802 and Samit
5377108438
[minor, fix] Allow passing class interface as custom pipeline argument ( #2973 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-05-29 23:39:38 +08:00
knlnguyen1802 and Samit
5eac50f22c
[bugfix, rl] Fix sleep do not release full memory in custom pipeline ( #3818 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Co-authored-by: Samit <285365963@qq.com >
2026-05-28 22:27:27 +08:00
Chendi.Xue
1903cee56a
[MXFP8][XPU] enable mxfp8 using vLLM main repo method ( #3782 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-05-28 02:47:22 +00:00
Canlin Guo and Hongsheng Liu
11c4c7f0ff
[Diffusion][Attention] Support per-role attention backend via CLI ( #2681 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-10 11:59:05 +08:00
baonudesifeizhai and roG0d
c4a0990044
（Phase 1）Add ModelOpt FP8 auto-detect support for diffusion checkpoints #2709 ( #2913 )
...
Signed-off-by: roG0d <rodgarcas98@gmail.com >
Signed-off-by: roG0d <baonudesifeizhai@gmail.com >
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com >
Co-authored-by: roG0d <rodgarcas98@gmail.com >
2026-05-09 16:55:05 +08:00
lsyyyyy and 汪志鹏
32663f21d5
[Feat] support hsdp for Bagel ( #3150 )
...
Signed-off-by: siyuan.lei <siyuanlei37@gmail.com >
Signed-off-by: lsyyyyy <siyuanlei37@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-05-08 15:20:42 +08:00
0a8204cb81
[Quantization] Redo Z-Image text encoder FP8 online quantization ( #3279 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Isotr0py <2037008807@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-05-06 16:37:36 +00:00
Hongsheng Liu
7fc528f7a9
Revert "[Quantization] Enable FP8 online quantization for Z-image text encoder" ( #3272 )
2026-04-30 15:08:17 +08:00
Isotr0py
ac66282746
[Quantization] Enable FP8 online quantization for Z-image text encoder ( #1338 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-04-30 14:12:01 +08:00
Zeyu Huang | 黃澤宇
d8cc7a0409
[feat]: General diffusers adapter backend to run diffusion models ( #2724 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
2026-04-23 00:07:18 +08:00
Yi Liu and Hongsheng Liu
c1d2dcc3ac
[AutoRound] Add offline quantized W4A16 model support ( #1777 )
...
Signed-off-by: yiliu30 <yi4.liu@intel.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-02 13:57:53 +08:00
SYLAR
6b93459826
[Core] Unified quantization framework ( #1764 )
2026-03-24 18:56:55 +08:00
Chendi.Xue
28aee51a59
[FP8] enable hunyuan-image-3 diffusion model with fp8 online quant ( #1935 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-03-22 13:02:16 +08:00
Lancer
af012babf8
[Bugfix] Fix cpu offload and quantization compatibility ( #1473 )
...
Signed-off-by: Lancer <maruixiang6688@gmail.com >
2026-03-14 18:55:30 +00:00
Samit
22a51a7cdb
[Feature][Wan2.2] Speed up diffusion model startup by multi-thread weight loading ( #1504 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
2026-03-02 14:53:25 +00:00
0b884ebc6c
[Feature]: Native GGUF Quantization Support for DiT ( #1285 )
...
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Isotr0py <2037008807@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-27 11:39:14 +08:00
Canlin Guo and Hongsheng Liu
f53152d340
[Feature] Support HSDP for diffusion models ( #1339 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-26 21:31:10 +08:00
knlnguyen1802
9994594782
[Misc] Support WorkerWrapperBase and CustomPipeline for Diffusion Worker ( #764 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
2026-02-12 19:23:51 +08:00
SYLAR
830d261f9d
[Feature]: FP8 Quantization Support for DiT ( #1034 )
...
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-02-11 21:01:35 +08:00
5037af1033
[Feature] Diffusion LoRA Adapter Support (PEFT compatible) for vLLM alignment ( #758 )
...
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk >
Signed-off-by: Andy Zhou <jzhoubc@connect.ust.hk >
Signed-off-by: dongbo910220 <1275604947@qq.com >
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: dongbo910220 <1275604947@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-01-27 10:22:40 +08:00
8939eb835d
[Rebase] Rebase to vllm 0.12.0 ( #335 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-19 16:42:02 +08:00
Isotr0py
059f45e48d
[Diffusion] Refactor diffusion models weights loading ( #157 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-06 11:12:13 +00:00
