admin
321e8bb47f
fix(quant): load offline-quantized compressed-tensors checkpoints directly on CPU under offload
...
The diffusion loader treated any quant config without the MXFP8/
is_checkpoint_quantized markers as online quantization: it loaded every
component on the accelerator before offloading, so serving the int8
weight-only compressed-tensors artifact OOM'd at model construction
(encoder 17 GiB + VAE + packed DiT skeleton > 24 GiB). compressed-tensors
is always a serialized format with no on-load quantization pass — detect
it (through component wrappers too) and load on the host, round-tripping
each quantized module through the accelerator only for the post-load
Marlin repack, which has no CPU path.
2026-10-08 07:36:01 -04:00
admin
2f8363c5fc
fix(quant): resolve hyphenated registry methods and disk-dict from_config in quant factory
...
_normalize_method_name canonicalizes dashes to underscores for _OVERRIDES
lookup, but vLLM's QUANTIZATION_METHODS keeps historical spellings
('compressed-tensors'), so both --quantization compressed-tensors and
checkpoint auto-detection raised 'Unknown quantization method'. Resolve
the normalized name back to the registry spelling, and fall back to
from_config for registry configs whose __init__ does not take the
serialized schema (compressed-tensors config_groups/format/ignore).
2026-10-08 07:00:05 -04:00
Zhou Taichang
0b0d2d7009
[Rebase] Align vLLM-Omni with vLLM v0.31.0 ( #8459 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-10-05 14:11:01 +08:00
MaciejBalaNV
7d6e2ade6c
[bugfix] Fix FP8 Quack kernels when using Context Parallelism and reduce number of recompilations ( #7458 )
...
Signed-off-by: Maciej Bala <mbala@nvidia.com >
2026-09-22 11:38:17 -07:00
ooooooye
823af92347
[Bugfix][Ascend NPU][Diffusion] Load over-wide unquantized fallback weights straight into host memory under DLOFix/mxfp online offload after quant ( #7009 )
...
Signed-off-by: brandneway <gyuan4892@gmail.com >
2026-09-18 07:36:48 +00:00
HAAZZZEEEE
becfefd440
[Hardware][Ascend] Add Wan2.2 MXFP4 UOS, Smooth and W4A8 fallback ( #7210 )
...
Signed-off-by: HAAZZZEEEE <wang-taicheng@qq.com >
2026-09-17 03:11:40 +00:00
DMAN
06541d1f52
[Model] FP8 Support For Boogu-Image ( #6925 )
...
Signed-off-by: Dmaner <2663515256@qq.com >
2026-09-06 04:48:38 +00:00
Zhou Taichang and Claude Opus 5
651b03225b
[Rebase] Rebase to vllm 0.28.0 ( #6606 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-27 06:12:55 +00:00
SYLAR and Hongsheng Liu
0e44fc7406
[diffusion][feature] Add offline SVDQuant W4A4 support ( #6162 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-27 08:45:53 +08:00
jl9876 and Chendi.Xue
0d5d1115d6
[AutoRound] Add offline quantized MXFP4 model support ( #5544 )
...
Signed-off-by: jl9876 <jia.li@intel.com >
Co-authored-by: Chendi.Xue <chendi.xue@intel.com >
2026-08-25 14:44:38 +00:00
SYLAR
1c917b2751
[Bugfix] Fix Qwen3-Omni AWQ quantization name mapping ( #5687 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
2026-08-23 15:13:36 +00:00
SYLAR and Hongsheng Liu
d1e230c95b
[Diffusion][Quantization] Enable MiniMax-H3 global FP8 with DLO ( #5910 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-15 23:01:26 +08:00
fcf1085395
[Refactor][OutputProcessor 2/8]: OmniRequestOutput should inherit RequestOutput and no nested wrap-up ( #5146 )
...
Signed-off-by: Boao Shi <aoibosh@connect.hku.hk >
Signed-off-by: bowieshi <aoibosh@connect.hku.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: R2-Y <ruiruyang2@gmail.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: Shi Boao Bowie <aoibosh@connect.hku.hk >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: ruirui(rein) yang <73573651+R2-Y@users.noreply.github.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-12 22:58:23 +08:00
Zhou Taichang and Claude Opus 5
2ce15ab7a0
[Rebase] Rebase to vllm 0.27.0 ( #5976 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-11 15:23:46 +00:00
Zijun Huang and WeiQing Chen
93457f29ff
[NPU][Quantization] Add RainFusion attention and INT8 online quantization for MiniMax H3 ( #5706 )
...
Signed-off-by: Zijun Huang <zijunhua@usc.edu >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
2026-08-06 14:15:21 +08:00
SYLAR and lishunyang12
779466becb
[Bugfix][Quant] Skip quack FP8 GEMM when scales are unpopulated ( #5262 )
...
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
2026-07-21 10:11:16 +00:00
bastefaniak and Roger Wang
07957e12f4
Fix FP8 Quack GEMM call when inference_mode() and torch.compile() is used. ( #5153 )
...
Signed-off-by: Bartosz Stefaniak <bstefaniak@nvidia.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-07-16 15:01:21 -07:00
wuli666 and wuli666
6603450d71
[Bugfix][Quantization] Initialize component quantization base state ( #5103 )
...
Signed-off-by: wuli666 <djjpro975@gmail.com >
Co-authored-by: wuli666 <djjpro975@gmail.com >
2026-07-14 12:06:55 +00:00
Diya Peng and diyapeng
e0570a0859
[Quantization] Add BitsAndBytes W4 online quantization for diffusion transformer ( #5037 )
...
Signed-off-by: diyapeng <50388704+diyapeng@users.noreply.github.com >
Co-authored-by: diyapeng <50388704+diyapeng@users.noreply.github.com >
2026-07-14 15:48:48 +08:00
Isotr0py
b95905deb1
[Quantization] Migrate GGUF diffusion model support to OOT plugin ( #4769 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-07-11 18:48:43 +00:00
Aleksi Vesanto and WeiQing Chen
0455bffc37
[ROCm][Perf] Add ROCm support for MXFP4 (w4a4) online quantization ( #4104 )
...
Signed-off-by: Aleksi Vesanto <avesanto@amd.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
2026-07-03 13:17:32 +08:00
Renzheng Wang
6b11f6b41d
[Bugfix][Quant] Gate quack FP8 to datacenter Blackwell (sm_100.x) only ( #4817 )
...
Signed-off-by: wangrzneu <wangrzneu@gmail.com >
2026-07-01 16:51:17 +00:00
Zhou Taichang
a560ed184d
[Rebase] Rebase to vllm v0.24.0 ( #4709 )
2026-06-30 19:41:06 +08:00
Yi Liu and Copilot
99234881f5
[Quant][AutoRound] Add W4A16 support for Qwen-Image ( #3588 )
...
Signed-off-by: yiliu30 <yi4.liu@intel.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-06-30 16:27:28 +08:00
SYLAR
b5352a5869
[Quant][Perf] Default Blackwell FP8 GEMM to quack CuteDSL fused-bias kernel ( #4241 )
...
Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com >
2026-06-08 19:44:20 +00:00
+1
f92d84fcf9
[Rebase] Rebase to vllm releases/v0.22.0 ( #3891 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: david6666666 <530634352@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: nainiu258 <cperfect02@163.com >
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com >
Co-authored-by: Zheng Wengang <zwg0606@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-06-01 11:08:12 +08:00
Chendi.Xue
1903cee56a
[MXFP8][XPU] enable mxfp8 using vLLM main repo method ( #3782 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-05-28 02:47:22 +00:00
baonudesifeizhai
076c34abfb
（Phase 2）Add ModelOpt mixed FP8/NVFP4 support for image generation ( #3570 )
...
Signed-off-by: roG0d <baonudesifeizhai@gmail.com >
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com >
2026-05-22 10:39:10 +08:00
8297570c2b
[AutoRound] Support WAN2.2 W4A16 quantization model ( #3353 )
...
Signed-off-by: lvliang-intel <liang1.lv@intel.com >
Signed-off-by: hyh_hh <huyinghong1@huawei.com >
Co-authored-by: hxhhhlalala <hyh_hh@163.com >
Co-authored-by: hyh_hh <huyinghong1@huawei.com >
2026-05-21 17:54:41 +08:00
hxhhhlalala and hyh_hh
9dd36e3afd
[NPU][Quant] Add W4A4 MXFP4 online & MXFP4 dual-scale online/offline quantization support for Wan2.2 T2V / I2V inference on Ascend NPU ( #3578 )
...
Signed-off-by: hyh_hh <huyinghong1@huawei.com >
Co-authored-by: hyh_hh <huyinghong1@huawei.com >
2026-05-20 06:21:29 +00:00
WeiQing Chen
a3d4ed809d
[Quantization][tools] Add diffusion quantization output comparison tool ( #3175 )
...
Signed-off-by: david6666666 <530634352@qq.com >
Signed-off-by: David Chen <530634352@qq.com >
2026-05-20 00:14:08 +08:00
hxhhhlalala and hyh_hh
a715abd447
[NPU][Quant] Add W8A8 MXFP8 online/offline quantization support for Wan2.2 T2V / I2V / TI2V inference on Ascend NPU ( #3140 )
...
Signed-off-by: hyh_hh <huyinghong1@huawei.com >
Co-authored-by: hyh_hh <huyinghong1@huawei.com >
2026-05-13 17:14:42 +08:00
baonudesifeizhai and roG0d
c4a0990044
（Phase 1）Add ModelOpt FP8 auto-detect support for diffusion checkpoints #2709 ( #2913 )
...
Signed-off-by: roG0d <rodgarcas98@gmail.com >
Signed-off-by: roG0d <baonudesifeizhai@gmail.com >
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com >
Co-authored-by: roG0d <rodgarcas98@gmail.com >
2026-05-09 16:55:05 +08:00
Liang Lv and SYLAR
ee15f39ea7
[AutoRound] Support Qwen Omni W4A16 quantization model ( #2670 )
...
Signed-off-by: lvliang-intel <liang1.lv@intel.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-04-22 19:54:45 +08:00
Yiyang "Ian" Liu and Copilot
53a9cf49a6
fix: do not apply FP8 quant config to vision/audio encoders for pre-quantized checkpoints ( #2702 )
...
Signed-off-by: Yiyang Liu <37043548+ianliuy@users.noreply.github.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-04-14 13:52:32 +00:00
Yi Liu and Hongsheng Liu
c1d2dcc3ac
[AutoRound] Add offline quantized W4A16 model support ( #1777 )
...
Signed-off-by: yiliu30 <yi4.liu@intel.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-02 13:57:53 +08:00
SYLAR
6b93459826
[Core] Unified quantization framework ( #1764 )
2026-03-24 18:56:55 +08:00
admin
Zhou Taichang