(Phase 2)Add ModelOpt mixed FP8/NVFP4 support for image generation (#3570)
Signed-off-by: roG0d <baonudesifeizhai@gmail.com> Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
This commit is contained in:
1 parent
7d8816b46b
commit
076c34abfb
16 files changed
+768
-68
No files matched your search
@@ -23,10 +23,10 @@ if TYPE_CHECKING:
|
||||
)
|
||||
|
||||
|
||||
# Pre-quantized checkpoints (modelopt FP8/FP4/MXFP8) only quantize the
|
||||
# Pre-quantized checkpoints (modelopt FP8/FP4/MXFP8/mixed) only quantize the
|
||||
# Thinker LM. Vision and audio encoder weights remain in BF16 with no
|
||||
# corresponding scale tensors in the checkpoint.
|
||||
PRE_QUANTIZED_METHODS: frozenset[str] = frozenset({"modelopt", "modelopt_fp4", "modelopt_mxfp8"})
|
||||
PRE_QUANTIZED_METHODS: frozenset[str] = frozenset({"modelopt", "modelopt_fp4", "modelopt_mxfp8", "modelopt_mixed"})
|
||||
|
||||
|
||||
def resolve_encoder_quant_config(
|
||||
|
||||
Reference in new issue
Block a user