[Feat] DiffusionEngine Support async batch infer (#2729)
Signed-off-by: Semmer <semmer@live.cn> Signed-off-by: jader <yjader@foxmail.com> Signed-off-by: asukaqaq-s <1311722138@qq.com> Co-authored-by: asukaqaq-s <1311722138@qq.com> Co-authored-by: jader <yjader@foxmail.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com> Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
This commit is contained in:
32 files changed
+1951
-190
No files matched your search
@@ -33,6 +33,12 @@ vllm serve Qwen/Qwen-Image --omni \
|
||||
--port 8091
|
||||
```
|
||||
|
||||
To replay step-wise continuous batching, compare
|
||||
`vllm serve Qwen/Qwen-Image --omni --port 8089 --step-execution --max-num-seqs 1`
|
||||
against the same command with `--max-num-seqs 8`. `--step-execution` is the
|
||||
feature gate, and increasing `--max-num-seqs` above `1` lets the scheduler
|
||||
keep more compatible requests active at once.
|
||||
|
||||
## 3.2 Key Parameters
|
||||
|
||||
| Parameter | Description |
|
||||
@@ -70,6 +76,7 @@ python benchmarks/diffusion/diffusion_benchmark_serving.py \
|
||||
| `--task` | Task type (e.g., `t2i`) |
|
||||
| `--num-prompts` | Total number of requests |
|
||||
| `--max-concurrency` | Client-side concurrency |
|
||||
| `--warmup-concurrency` | Warmup concurrency for batch-shape prewarming |
|
||||
| `--random-request-config`| JSON string defining random request |
|
||||
|
||||
---
|
||||
@@ -133,6 +140,12 @@ python benchmarks/diffusion/diffusion_benchmark_serving.py \
|
||||
]'
|
||||
```
|
||||
|
||||
For a continuous-batching A/B replay, run the same benchmark command twice
|
||||
against the two serving configs above and keep traffic plus warmup settings
|
||||
identical across both runs. If the measured run uses `--max-concurrency 8`,
|
||||
warm with `--warmup-requests 8 --warmup-concurrency 8` so the first measured
|
||||
batch does not include compile or CUDA-graph capture overhead.
|
||||
|
||||
---
|
||||
|
||||
# 6. Performance Metrics
|
||||
@@ -163,6 +176,7 @@ To ensure consistent and comparable benchmark results:
|
||||
* Record parallel configuration
|
||||
* Record benchmark parameters (resolution, concurrency, number of prompts)
|
||||
* Ensure no background workload on GPUs during testing
|
||||
* When comparing continuous batching, keep warmup settings the same across runs
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in new issue
Block a user