(Phase 1)Add ModelOpt FP8 auto-detect support for diffusion checkpoints #2709 (#2913)

Signed-off-by: roG0d <rodgarcas98@gmail.com>
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
Co-authored-by: roG0d <rodgarcas98@gmail.com>
This commit is contained in:
baonudesifeizhaiandroG0d authored and GitHub committed 2026-05-09 16:55:05 +08:00
1 parent 40a07e0d80
commit c4a0990044
26 files changed
+1214 -176

No files matched your search

+10
View File
@@ -338,6 +338,16 @@ class OmniServeCommand(CLISubcommand):
'Example: \'{"method":"gguf","gguf_model":"/path/to/model.gguf"}\'.'
),
)
omni_config_group.add_argument(
"--force-cutlass-fp8",
action="store_true",
default=None,
help=(
"Diffusion-only runtime override for ModelOpt FP8 checkpoints: "
"force CUTLASS FP8 linear kernels on CUDA SM89+ devices. "
"Ignored for BF16, non-ModelOpt FP8, ROCm, and older CUDA GPUs."
),
)
# HSDP (Hybrid Sharded Data Parallel) parameters
omni_config_group.add_argument(