[Diffusion][Perf] Remove Redundant Communication Cost by Refining SP Hook Design (#1275)
Signed-off-by: mxuax <mxuax@connect.ust.hk> Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
This commit is contained in:
1 parent
d3ea94309d
commit
ac397ae2a0
3 files changed
+52
-5
No files matched your search
@@ -31,6 +31,16 @@ class ForwardContext:
|
||||
# Original sequence length before padding (for removing padding in gather)
|
||||
sp_original_seq_len: int | None = None
|
||||
|
||||
# SP active scope tracking
|
||||
# Tracks the depth of SP sharding - incremented on shard, decremented on gather
|
||||
# Used by attention layers to determine if SP communication should be enabled
|
||||
_sp_shard_depth: int = 0
|
||||
|
||||
@property
|
||||
def sp_active(self) -> bool:
|
||||
"""Returns True when inside an SP sharded region (between shard and gather)."""
|
||||
return self._sp_shard_depth > 0
|
||||
|
||||
def __post_init__(self):
|
||||
pass
|
||||
|
||||
|
||||
Reference in new issue
Block a user