[Perf][Wan2.2] Skip attention mask for zero-padded SP sequences to avoid varlen path (#3763)

Signed-off-by: jjuvonen-amd <joni.juvonen@amd.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
Joni JuvonenandHongsheng Liu authored and GitHub committed 2026-06-13 07:50:39 +08:00
1 parent 9577ccba50
commit 20b7a67b0d
4 files changed
+79 -24

No files matched your search

+8
View File
@@ -130,6 +130,14 @@ class DiffusionParallelConfig:
use_hsdp: bool = False
"""Enable Hybrid Sharded Data Parallel (HSDP) for model weight sharding."""
mask_sp_padding: bool = False
"""If True, generate a boolean attention mask for zero-padded SP tokens
when sequence length is not divisible by the SP world size. The mask
routes attention through the varlen path (unpad→kernel→repad), which is
correct but carries additional overhead. When False (default), padding
tokens are left unmasked; since _shard_with_auto_pad always pads with
zeros, their contribution to attention output is negligible."""
hsdp_shard_size: int = -1
"""Number of GPUs to shard weights across within each replica group. -1 means auto-calculate."""