[feature] cpu worker support

Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
This commit is contained in:
齐保元 committed 2025-11-14 18:44:28 +08:00
1 parent 8b30818f2e
commit 5c038ae553
2 files changed
+3 -3

No files matched your search

@@ -37,7 +37,7 @@ logger = logging.getLogger(__name__)
class CPUDiffusionModelRunner(OmniCPUModelRunner):
"""Diffusion model runner for vLLM-omni (non-autoregressive).
- Reuses GPUModelRunner preparation, multimodal handling, and TP/PP/DP glue.
- Reuses CPUModelRunner preparation, multimodal handling, and TP/PP/DP glue.
- Does not compute logits or perform token sampling.
- Executes diffusion process and returns tensors via `pooler_output`.
"""
+2 -2
View File
@@ -40,9 +40,9 @@ class OmniCPUModelRunner(CPUModelRunner):
output.
The updated states are used by the `_prepare_inputs` function to create
the input GPU tensors for the model.
the input CPU tensors for the model.
The SamplingMetadata is updated and copied to the GPU if there is a
The SamplingMetadata is updated and copied to the CPU if there is a
new/resumed/paused/finished request in the batch.
"""
# Remove finished requests from the cached states.