Wire enable_cuda_graph_decode through engine args, stage config and CLI
Follows the vae_use_tiling path: OmniDiffusionConfig field, OmniEngineArgs, StageDeployConfig, _DiffusionConfigProjection and --[no-]enable-cuda-graph-decode. Documents interplay with enforce_eager and cudagraph_mode.
This commit is contained in:
1 parent
a04b95bf6b
commit
d3cf26c1b1
5 files changed
+34
No files matched your search
@@ -604,6 +604,16 @@ class OmniServeCommand(CLISubcommand):
|
||||
"Disable for fixed-shape workloads with --no-diffusion-compile-dynamic."
|
||||
),
|
||||
)
|
||||
omni_config_group.add_argument(
|
||||
"--enable-cuda-graph-decode",
|
||||
action=argparse.BooleanOptionalAction,
|
||||
default=None,
|
||||
help=(
|
||||
"Capture fixed-shape KV-cache decode steps into CUDA graphs on models that implement them "
|
||||
"(currently Qwen-Image-2.1). Independent of compilation_config.cudagraph_mode; "
|
||||
"disable with --no-enable-cuda-graph-decode. --enforce-eager also disables it."
|
||||
),
|
||||
)
|
||||
omni_config_group.add_argument(
|
||||
"--fa-deterministic",
|
||||
dest="fa_deterministic",
|
||||
|
||||
Reference in new issue
Block a user