[Core][Frontend] Support request-level batching for diffusion pipelines (#4079)

Signed-off-by: jader <yjader@foxmail.com>
Co-authored-by: Samit <285365963@qq.com>
This commit is contained in:
JiangJie ZhangandSamit authored and GitHub committed 2026-06-29 18:09:19 +02:00
1 parent 504698dec8
commit c45ac74d43
123 files changed
+4160 -2233

No files matched your search

@@ -160,9 +160,11 @@ python examples/offline_inference/text_to_image/text_to_image.py \
--output flux2-dev.png
```
### Batch Requests (Multiple Prompts)
### Multiple Prompts
You can pass multiple prompts in a single `generate` call.
You can pass multiple prompts in a single `generate` call. For diffusion
pipelines, each prompt is submitted as a separate logical request; compatible
requests may be automatically batched by the scheduler and runner.
```python
from vllm_omni.entrypoints.omni import Omni
@@ -181,17 +183,8 @@ if __name__ == "__main__":
!!! info
Not all models support batch inference, and batch requesting mostly does not provide significant
performance improvement. This feature is primarily for interface compatibility with vLLM and to
allow for future improvements.
!!! info
For diffusion pipelines, the stage config field `stage_args.[].runtime.max_batch_size` is 1 by
default, and the input list is sliced into single-item requests before feeding into the diffusion
pipeline. For models that do internally support batched inputs, you can
[modify this configuration](https://github.com/vllm-project/vllm-omni/tree/main/configuration/stage_configs.md) to let the model accept a
longer batch of prompts.
For diffusion request-level batching controls such as `max_num_seqs`, see
[Request-Level Batching](../../diffusion/request_batching.md).
### Negative Prompts