[Feat] Add native Mooncake KV transfer from AR to DiT (#7166)
Signed-off-by: asukaqaq-s <1311722138@qq.com> Signed-off-by: Acerak01-fy <wfy2003324@163.com> Co-authored-by: Acerak01-fy <wfy2003324@163.com> Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
This commit is contained in:
53 files changed
+2564
-204
No files matched your search
@@ -58,6 +58,7 @@ def _od_config(**overrides):
|
||||
gpu_memory_utilization=0.9,
|
||||
kv_cache_memory_bytes=None,
|
||||
max_num_seqs=1,
|
||||
diffusion_kv_max_rows_per_request=1,
|
||||
max_num_batched_tokens=64,
|
||||
num_gpus=1,
|
||||
)
|
||||
|
||||
Reference in new issue
Block a user