[Refactor] Rename SupportsModuleOffload to SupportsComponentDiscovery (#3354)
Signed-off-by: Nick Cao <ncao@redhat.com> Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
1 parent
b968373c88
commit
65bc968465
9 files changed
+41
-36
No files matched your search
@@ -38,14 +38,15 @@ vllm-omni serve diffusion Wan-AI/Wan2.2-T2V-A14B-Diffusers --enable-cpu-offload
|
||||
|
||||
### To Support a Model
|
||||
|
||||
Implement the `SupportsModuleOffload` protocol to declare which
|
||||
submodules participate in offloading:
|
||||
Implement the `SupportsComponentDiscovery` protocol to declare which
|
||||
submodules serve as pipeline components (used by offloading, HSDP
|
||||
sharding, and other framework features):
|
||||
|
||||
```python
|
||||
from typing import ClassVar
|
||||
from vllm_omni.diffusion.models.interface import SupportsModuleOffload
|
||||
from vllm_omni.diffusion.models.interface import SupportsComponentDiscovery
|
||||
|
||||
class MyPipeline(nn.Module, SupportsModuleOffload):
|
||||
class MyPipeline(nn.Module, SupportsComponentDiscovery):
|
||||
_dit_modules: ClassVar[list[str]] = ["transformer"]
|
||||
_encoder_modules: ClassVar[list[str]] = ["text_encoder", "vision_model"]
|
||||
_vae_modules: ClassVar[list[str]] = ["vae"]
|
||||
@@ -158,7 +159,7 @@ class Flux2Transformer2DModel(nn.Module):
|
||||
The offloader discovers pipeline components in two ways:
|
||||
|
||||
1. **Protocol-based** (preferred): If the pipeline implements
|
||||
`SupportsModuleOffload`, its `_dit_modules`, `_encoder_modules`,
|
||||
`SupportsComponentDiscovery`, its `_dit_modules`, `_encoder_modules`,
|
||||
`_vae_modules`, and `_resident_modules` class variables are used
|
||||
directly. All attribute names support dotted paths (e.g.
|
||||
`"pipe.transformer"`, `"bagel.time_embedder"`) for nested submodules.
|
||||
|
||||
Reference in new issue
Block a user