Gate persistent DiT staging on decode-graph models

ModelLevelOffloadBackend passed persistent_dit_staging=(device is CUDA) for
every model, so Qwen-Image 2.0/FLUX/WAN-style sequential offload kept DiT
device storage alive across swaps instead of freeing VRAM. Only retain
staging when a DiT actually captures weight pointers (enable_cuda_graph_decode);
other models keep plain move semantics. Add tests pinning both behaviors.
This commit is contained in:
Isotr0py committed 2026-09-22 08:58:32 +00:00
1 parent be8a72f0a6
commit 0c82bb1315
2 files changed
+56 -2

No files matched your search

@@ -415,8 +415,14 @@ class ModelLevelOffloadBackend(OffloadBackend):
offload_dit_modules=selected_dits,
offload_encoder_modules=selected_encoders,
# Fixed DiT staging keeps decode-graph weight pointers valid;
# other platforms keep plain move semantics.
persistent_dit_staging=self.device.type == "cuda",
# only models that capture weight pointers (Qwen-Image-2.1's
# CUDA-graph decode) pay for it — every other model keeps
# plain move semantics so offloading actually frees DiT VRAM
# while the encoders run.
persistent_dit_staging=(
self.device.type == "cuda"
and any(getattr(dit, "enable_cuda_graph_decode", False) for dit in dits)
),
)
except BaseException:
try: