Wire enable_cuda_graph_decode through engine args, stage config and CLI
Follows the vae_use_tiling path: OmniDiffusionConfig field, OmniEngineArgs, StageDeployConfig, _DiffusionConfigProjection and --[no-]enable-cuda-graph-decode. Documents interplay with enforce_eager and cudagraph_mode.
This commit is contained in:
1 parent
a04b95bf6b
commit
d3cf26c1b1
5 files changed
+34
No files matched your search
@@ -955,6 +955,18 @@ class OmniDiffusionConfig:
|
||||
|
||||
# Compilation
|
||||
enforce_eager: bool = False
|
||||
# Capture fixed-shape KV-cache decode (denoising) steps into CUDA graphs.
|
||||
# Currently only Qwen-Image-2.1's transformer implements this; other models
|
||||
# ignore the flag. This is a per-model diffusion path, independent of the
|
||||
# AR engine's ``compilation_config.cudagraph_mode`` (which does not apply
|
||||
# to diffusion stages) and of ``diffusion_compile_granularity`` — models
|
||||
# with decode graphs skip automatic torch.compile of their DiT blocks, so
|
||||
# the two never stack. ``enforce_eager=True`` forces eager decode and
|
||||
# disables capture regardless of this flag; unsupported configurations
|
||||
# (SP/TP, ring, HSDP, dynamic LoRA, padded masks, quantized prefix KV, or
|
||||
# a second in-flight request aliasing the same graph key) log and fall
|
||||
# back to eager decode.
|
||||
enable_cuda_graph_decode: bool = True
|
||||
# Controls the generic compilation path used when a pipeline does not
|
||||
# provide its own setup_compile() implementation.
|
||||
diffusion_compile_granularity: str = "regional"
|
||||
|
||||
Reference in new issue
Block a user