[Feature]Support async computation and communication across stages by chunks (#727)

Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
authored and GitHub committed 2026-01-26 11:21:24 +08:00
1 parent 578ed1967e
commit 52bf1e2f40
29 files changed
+1258 -396

No files matched your search

@@ -207,10 +207,11 @@ class GPUDiffusionModelRunner:
start_time = time.time()
while True:
get_key = f"omni_{from_stage}_to_{to_stage}_kv_cache_{req.request_id}"
result = self.connector.get(
from_stage=from_stage,
to_stage=to_stage,
request_id=f"kv_cache_{req.request_id}",
get_key=get_key,
)
if result:
break