[Feat] support for multi-block layerwise offloading, fix top-level parameters/buffers staying on CPU (#1486)
Signed-off-by: Lancer <maruixiang6688@gmail.com> Signed-off-by: Lancer <402430575@qq.com> Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
This commit is contained in:
1 parent
e23b2634d1
commit
328de586aa
13 files changed
+288
-38
No files matched your search
@@ -802,7 +802,7 @@ omni = Omni(model="your-model", enable_layerwise_offload=True)
|
||||
|
||||
```python
|
||||
class WanTransformer3DModel(nn.Module):
|
||||
_layerwise_offload_blocks_attr = "blocks" # Attribute name containing transformer blocks
|
||||
_layerwise_offload_blocks_attrs = ["blocks"] # Attribute name containing transformer blocks
|
||||
|
||||
def __init__(self):
|
||||
self.blocks = nn.ModuleList([...]) # Transformer blocks
|
||||
|
||||
Reference in new issue
Block a user