[Frontend] Add omni benchmark support for image and video endpoints (#4728)

This commit is contained in:
ZacheryAU authored and GitHub committed 2026-09-08 09:28:21 +08:00
1 parent e1c59edec6
commit b1ec136bd7
21 files changed
+1626 -72

No files matched your search

+70
View File
@@ -436,6 +436,76 @@ This should be seen as an edge case, and if this behavior can be avoided by sett
</details>
### Image / video generation
For `/v1/images/generations`, `/v1/images/edits`, and `/v1/videos`, set `--endpoint` to that path and omit `--backend`. The endpoint path is registered as the bench request adapter, so a separate named backend is not required.
Example (`/v1/images/generations`):
```bash
vllm bench serve --omni \
--endpoint /v1/images/generations \
--dataset-name random \
--model ~/models/Qwen/Qwen-Image \
--tokenizer ~/models/Qwen/Qwen-Image/tokenizer \
--max-concurrency 1 \
--num-warmups 2 \
--num-prompts 10 \
--random-input-len 64 \
--random-output-len 1 \
--ignore-eos \
--percentile-metrics e2el \
--extra-body '{
"num_inference_steps": 20,
"seed": 42,
"true_cfg_scale": 4.0
}'
```
If successful, following output is like:
```text
============ Serving Benchmark Result ============
Successful requests: 3
Failed requests: 0
Maximum request concurrency: 1
Benchmark duration (s): 8.42
Request throughput (req/s): 0.36
Peak concurrent requests: 2.00
-------------------Peak Memory--------------------
Mean PEAK_MEMORY_MB (MB): 58832.00
Median PEAK_MEMORY_MB (MB): 58832.00
P99 PEAK_MEMORY_MB (MB): 58832.00
----------------End-to-end Latency----------------
Mean E2EL (ms): 2798.16
Median E2EL (ms): 2799.83
P99 E2EL (ms): 2800.03
================== Image Result ==================
Total images generated: 3
Image throughput (img/s): 0.36
Average pixels per image: 1048576.00
Mean denoise step latency (ms): 136.59
---------------- Image Generation ----------------
Mean IMAGE_GENERATION (ms): 2731.84
Median IMAGE_GENERATION (ms): 2732.02
P99 IMAGE_GENERATION (ms): 2734.65
==================================================
```
Use the same pattern with `--endpoint /v1/images/edits` or `--endpoint /v1/videos` (and model-specific `--extra-body` as needed). Pure image/video runs omit the Text Result section when there is no generated text.
`/v1/images/edits` defaults to **non-streaming JSON** (`stream=false`) so single-stage edit models are not rejected by the server. For multi-stage pipelines that need SSE (AR TTFT / image chunks), pass `"stream": true` in `--extra-body`.
`/v1/videos` is an async job API: the client creates a job, then polls until `completed`/`failed`. Measured **e2el** therefore includes client poll sleep and any overshoot after the job actually finishes. Tune polling via `--extra-body`:
- `poll_interval_s` (default `2.0`): sleep between status polls
- `poll_timeout_s` (default `21600`, i.e. 6 hours): give up waiting for the job
**VIDEO_RTF** prefers server-reported generation time when available:
- `video_rtf = video_generation_time_ms / 1000 / video_duration`, where `video_generation_time_ms` comes from response `stage_durations` (or `inference_time_s` as fallback)
- If generation time is missing, falls back to `e2el / video_duration` (this fallback *does* include poll overhead)
### Multi-Stage Benchmark
<details class="admonition abstract" markdown="1">