[Profile] Adding metrics for Diffusion/DiT Single diffusion Pipeline (#668)
Signed-off-by: Chen Yang <2082464740@qq.com> Signed-off-by: erfgss <97771661+erfgss@users.noreply.github.com>
This commit is contained in:
1 parent
411fe05bd4
commit
b7fcc9d07e
10 files changed
+132
-18
No files matched your search
@@ -69,8 +69,20 @@ With `--log-stats` enabled, the server will output detailed metrics logs after e
|
||||
| rx_decode_time_ms | 111.865 | 31.706 |
|
||||
| in_flight_time_ms | 2.015 | 2.819 |
|
||||
|
||||
### DiffusionStats
|
||||
|
||||
These logs include:
|
||||
| Field | Value |
|
||||
|---------------------------------|------------|
|
||||
| batch_id | 1 |
|
||||
| batch_size | 1 |
|
||||
| diffusion_engine_exec_time_ms | 16,363.419 |
|
||||
| diffusion_engine_total_time_ms | 16,234.849 |
|
||||
| image_num | 1.000 |
|
||||
| num_inference_steps | 50.000 |
|
||||
| postprocess_time_ms | 67.685 |
|
||||
| preprocess_time_ms | 60.106 |
|
||||
| resolution | 640.000 |
|
||||
| stage_gen_time_ms | 16,363.714 |
|
||||
|
||||
- **Overall summary**: total requests, wall time, average tokens/sec, etc.
|
||||
|
||||
@@ -80,6 +92,9 @@ These logs include:
|
||||
|
||||
- **Transfer table**: data transfer and timing for each edge.
|
||||
|
||||
- **DiffusionStats**:preprocessing time, model execution time, postprocessing time, generated images, resolution, and inference steps.
|
||||
|
||||
|
||||
You can use these logs to monitor system health, debug performance, and analyze request-level metrics as described above.
|
||||
|
||||
|
||||
@@ -130,6 +145,11 @@ For **online inference** (serving mode), the summary is always per-request. `e2e
|
||||
| `image_num` | Number of images generated (for diffusion/image stages). |
|
||||
| `resolution` | Image resolution (for diffusion/image stages). |
|
||||
| `postprocess_time_ms` | Diffusion/image: post-processing time in ms. |
|
||||
| `preprocess_time_ms` | Diffusion/image: pre-processing time in ms (before model execution). |
|
||||
| `diffusion_engine_exec_time_ms` | Diffusion/image: model execution time in ms, excluding pre-processing and post-processing. |
|
||||
| `diffusion_engine_total_time_ms` | Diffusion/image: total diffusion engine time in ms, including pre-processing, model execution, and post-processing. |
|
||||
| `num_inference_steps` | Number of inference steps for diffusion generation. |
|
||||
|
||||
|
||||
---
|
||||
|
||||
@@ -143,6 +163,8 @@ For **online inference** (serving mode), the summary is always per-request. `e2e
|
||||
| `in_flight_time_ms` | In-flight time in ms. |
|
||||
|
||||
|
||||
|
||||
|
||||
### Expectation of the Numbers (Verification)
|
||||
|
||||
**Formulas:**
|
||||
@@ -171,3 +193,14 @@ For each edge:
|
||||
- 1->2: tx_time_ms (**18.790**) + rx_decode_time_ms (**31.706**) + in_flight_time_ms (**2.819**) = **53.315**
|
||||
|
||||
192.581 + 53.315 = **245.896** = transfers_total_time_ms, which matches the calculation (difference is due to rounding)
|
||||
|
||||
**diffusion_engine_total_time_ms**
|
||||
|
||||
- preprocess_time_ms: Time spent on input preprocessing
|
||||
- diffusion_engine_exec_time_ms: Pure model execution time
|
||||
- postprocess_time_ms: Time spent on output postprocessing
|
||||
- diffusion_engine_total_time_ms: Total time spent in the diffusion engine
|
||||
|
||||
so `diffusion_engine_total_time_ms` = `diffusion_engine_exec_time_ms` + `preprocess_time_ms` + `postprocess_time_ms`
|
||||
= **16,363.419** + **60.106** + **67.685**
|
||||
= **16,491.210**
|
||||
Reference in new issue
Block a user