anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Perf] Deduplicate AR prefix cache hidden-state CPU staging (#3734)

Browse Source 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
This commit is contained in:
TaffyOfficialandTaffyOfficial authored and GitHub committed 2026-06-01 09:35:07 +00:00
1 parent b76291d6a1
commit ee33601571
7 files changed
+386 -14
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

vllm_omni/worker/gpu_generation_model_runner.py
+2
View File
@@ -355,6 +355,7 @@ class GPUGenerationModelRunner(OmniGPUModelRunner, OmniConnectorModelRunnerMixin
None,
None,
None,
None,
ec_connector_output,
cudagraph_stats,
multimodal_outputs,
@@ -398,6 +399,7 @@ class GPUGenerationModelRunner(OmniGPUModelRunner, OmniConnectorModelRunnerMixin
spec_decode_metadata,
spec_decode_common_attn_metadata,
hidden_states,
_hidden_states_cpu,
sample_hidden_states,
aux_hidden_states,
ec_connector_output,