[Refactor] Migrate existing pipelines to use SupportsComponentDiscovery for offload discovery (#3076)
Signed-off-by: Nick Cao <ncao@redhat.com> Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
1 parent
8fa0394d29
commit
e957d16f8d
43 files changed
+392
-66
No files matched your search
@@ -145,7 +145,7 @@ python end2end.py \
|
||||
## Reducing GPU Memory Usage
|
||||
|
||||
For hardware with limited VRAM, enable **module-wise CPU offload** with
|
||||
`--enable-cpu-offload`. The pipeline implements `SupportsModuleOffload`, so the
|
||||
`--enable-cpu-offload`. The pipeline implements `SupportsComponentDiscovery`, so the
|
||||
vision encoder (`vision_model`) and the Qwen3 LLM (`language_model`) are
|
||||
swapped between CPU and GPU on demand:
|
||||
|
||||
|
||||
Reference in new issue
Block a user