[Perf][PrefixCache] Avoid per-step blocking write in OmniTensorPrefix… (#4106)

Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
LHXuuuandHongsheng Liu authored and GitHub committed 2026-06-17 16:53:47 +08:00
1 parent 890803ca06
commit dff757da9c
4 files changed
+257 -18

No files matched your search

@@ -20,7 +20,7 @@ stage_args:
trust_remote_code: true
engine_output_type: text
distributed_executor_backend: "mp"
enable_prefix_caching: false
enable_prefix_caching: true
hf_config_name: thinker_config
tensor_parallel_size: 2
final_output: true