[Diffusion] Register runtime cache for direct DLO H2D

Avoid recurrent mmap-to-pinned packing in no-AllGather DLO by registering transform-complete runtime-cache mappings under an explicit per-worker budget. Copy mapped tensors directly into the rotating device buffers while preserving bounded-staging fallback and ordered shutdown cleanup.

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
Hongsheng Liu committed 2026-08-17 11:50:32 +08:00
1 parent b732ad2cea
commit f1abb13958
18 files changed
+976 -74

No files matched your search

+12 -5
View File
@@ -805,11 +805,18 @@ class DiffusionWorker:
def shutdown(self) -> None:
"""Shutdown the worker and cleanup distributed environment."""
if self.model_runner is not None:
mgr = getattr(self.model_runner, "kv_transfer_manager", None)
if mgr is not None:
mgr.shutdown_prefetch()
destroy_distributed_env()
try:
if self.model_runner is not None:
mgr = getattr(self.model_runner, "kv_transfer_manager", None)
try:
offload_backend = getattr(self.model_runner, "offload_backend", None)
if offload_backend is not None:
offload_backend.disable()
finally:
if mgr is not None:
mgr.shutdown_prefetch()
finally:
destroy_distributed_env()
class CustomPipelineWorkerExtension: