Signed-off-by: roG0d <rodgarcas98@gmail.com> Signed-off-by: roG0d <baonudesifeizhai@gmail.com> Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com> Co-authored-by: roG0d <rodgarcas98@gmail.com>
This commit is contained in:
1 parent
40a07e0d80
commit
c4a0990044
26 files changed
+1214
-176
No files matched your search
@@ -338,6 +338,16 @@ class OmniServeCommand(CLISubcommand):
|
||||
'Example: \'{"method":"gguf","gguf_model":"/path/to/model.gguf"}\'.'
|
||||
),
|
||||
)
|
||||
omni_config_group.add_argument(
|
||||
"--force-cutlass-fp8",
|
||||
action="store_true",
|
||||
default=None,
|
||||
help=(
|
||||
"Diffusion-only runtime override for ModelOpt FP8 checkpoints: "
|
||||
"force CUTLASS FP8 linear kernels on CUDA SM89+ devices. "
|
||||
"Ignored for BF16, non-ModelOpt FP8, ROCm, and older CUDA GPUs."
|
||||
),
|
||||
)
|
||||
|
||||
# HSDP (Hybrid Sharded Data Parallel) parameters
|
||||
omni_config_group.add_argument(
|
||||
|
||||
Reference in new issue
Block a user