[Bagel]: Support SP (#1903)
Signed-off-by: princepride <wangzhipeng628@gmail.com> Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com> Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com> Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
This commit is contained in:
1 parent
d6b55b9196
commit
77d773ad47
11 files changed
+615
-13
No files matched your search
@@ -215,7 +215,7 @@ class DiffusionModelRunner:
|
||||
grad_context = torch.no_grad() if use_hsdp else torch.inference_mode()
|
||||
with grad_context:
|
||||
# The manager handles the check for need_recv_cache internally
|
||||
self.kv_transfer_manager.receive_multi_kv_cache(
|
||||
self.kv_transfer_manager.receive_multi_kv_cache_distributed(
|
||||
req,
|
||||
cfg_kv_collect_func=getattr(self.od_config, "cfg_kv_collect_func", None),
|
||||
target_device=getattr(self.pipeline, "device", None),
|
||||
|
||||
Reference in new issue
Block a user