admin
321e8bb47f
fix(quant): load offline-quantized compressed-tensors checkpoints directly on CPU under offload
...
The diffusion loader treated any quant config without the MXFP8/
is_checkpoint_quantized markers as online quantization: it loaded every
component on the accelerator before offloading, so serving the int8
weight-only compressed-tensors artifact OOM'd at model construction
(encoder 17 GiB + VAE + packed DiT skeleton > 24 GiB). compressed-tensors
is always a serialized format with no on-load quantization pass — detect
it (through component wrappers too) and load on the host, round-tripping
each quantized module through the accelerator only for the post-load
Marlin repack, which has no CPU path.
2026-10-08 07:36:01 -04:00
SYLAR and Hongsheng Liu
0e44fc7406
[diffusion][feature] Add offline SVDQuant W4A4 support ( #6162 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-27 08:45:53 +08:00
SYLAR
1c917b2751
[Bugfix] Fix Qwen3-Omni AWQ quantization name mapping ( #5687 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
2026-08-23 15:13:36 +00:00
SYLAR and Hongsheng Liu
d1e230c95b
[Diffusion][Quantization] Enable MiniMax-H3 global FP8 with DLO ( #5910 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-15 23:01:26 +08:00
wuli666 and wuli666
6603450d71
[Bugfix][Quantization] Initialize component quantization base state ( #5103 )
...
Signed-off-by: wuli666 <djjpro975@gmail.com >
Co-authored-by: wuli666 <djjpro975@gmail.com >
2026-07-14 12:06:55 +00:00
Yi Liu and Copilot
99234881f5
[Quant][AutoRound] Add W4A16 support for Qwen-Image ( #3588 )
...
Signed-off-by: yiliu30 <yi4.liu@intel.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-06-30 16:27:28 +08:00
baonudesifeizhai
076c34abfb
（Phase 2）Add ModelOpt mixed FP8/NVFP4 support for image generation ( #3570 )
...
Signed-off-by: roG0d <baonudesifeizhai@gmail.com >
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com >
2026-05-22 10:39:10 +08:00
Yiyang "Ian" Liu and Copilot
53a9cf49a6
fix: do not apply FP8 quant config to vision/audio encoders for pre-quantized checkpoints ( #2702 )
...
Signed-off-by: Yiyang Liu <37043548+ianliuy@users.noreply.github.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-04-14 13:52:32 +00:00
SYLAR
6b93459826
[Core] Unified quantization framework ( #1764 )
2026-03-24 18:56:55 +08:00
admin
