[Feat] DiffusionEngine Support async batch infer (#2729)

Signed-off-by: Semmer <semmer@live.cn>
Signed-off-by: jader <yjader@foxmail.com>
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: jader <yjader@foxmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
This commit is contained in:
authored and GitHub committed 2026-05-06 09:47:13 +12:00
1 parent 2b85474c23
commit e49fbd8a2d
32 files changed
+1951 -190

No files matched your search

@@ -33,6 +33,12 @@ vllm serve Qwen/Qwen-Image --omni \
--port 8091
```
To replay step-wise continuous batching, compare
`vllm serve Qwen/Qwen-Image --omni --port 8089 --step-execution --max-num-seqs 1`
against the same command with `--max-num-seqs 8`. `--step-execution` is the
feature gate, and increasing `--max-num-seqs` above `1` lets the scheduler
keep more compatible requests active at once.
## 3.2 Key Parameters
| Parameter | Description |
@@ -70,6 +76,7 @@ python benchmarks/diffusion/diffusion_benchmark_serving.py \
| `--task` | Task type (e.g., `t2i`) |
| `--num-prompts` | Total number of requests |
| `--max-concurrency` | Client-side concurrency |
| `--warmup-concurrency` | Warmup concurrency for batch-shape prewarming |
| `--random-request-config`| JSON string defining random request |
---
@@ -133,6 +140,12 @@ python benchmarks/diffusion/diffusion_benchmark_serving.py \
]'
```
For a continuous-batching A/B replay, run the same benchmark command twice
against the two serving configs above and keep traffic plus warmup settings
identical across both runs. If the measured run uses `--max-concurrency 8`,
warm with `--warmup-requests 8 --warmup-concurrency 8` so the first measured
batch does not include compile or CUDA-graph capture overhead.
---
# 6. Performance Metrics
@@ -163,6 +176,7 @@ To ensure consistent and comparable benchmark results:
* Record parallel configuration
* Record benchmark parameters (resolution, concurrency, number of prompts)
* Ensure no background workload on GPUs during testing
* When comparing continuous batching, keep warmup settings the same across runs
---