[Feat] DiffusionEngine Support async batch infer (#2729)

Signed-off-by: Semmer <semmer@live.cn>
Signed-off-by: jader <yjader@foxmail.com>
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: jader <yjader@foxmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
This commit is contained in:
authored and GitHub committed 2026-05-06 09:47:13 +12:00
1 parent 2b85474c23
commit e49fbd8a2d
32 files changed
+1951 -190

No files matched your search

+3 -3
View File
@@ -201,7 +201,7 @@ class _BaseScheduler(SchedulerInterface):
- **Shared cleanup logic**: Request-id registration, finish handling, and state removal are centralized instead of duplicated in each policy.
- **Current constraint**: `max_num_running_reqs` remains `1` because the current engine path is still synchronous request-mode execution.
- **Current constraint boundary**: `_BaseScheduler` derives `max_num_running_reqs` from `max_num_seqs`, but request-mode diffusion is still clamped back to `1` by the engine. The step-wise path can keep this above `1` for compatible-request batching.
#### 2.4 Current `RequestScheduler` Policy
@@ -217,7 +217,7 @@ class RequestScheduler(_BaseScheduler):
- **FIFO request scheduling**: Waiting requests are promoted in queue order.
- **Single-request admission**: The current policy only admits one active request at a time.
- **Single-request admission**: `RequestScheduler` still admits one active request at a time because request-mode execution completes a whole request per dispatch.
- **Executor result feedback**: `update_from_output()` converts executor output into `FINISHED_COMPLETED` or `FINISHED_ERROR` and returns finished scheduler ids.
@@ -237,7 +237,7 @@ while True:
- **No scheduler-owned IPC**: Scheduler no longer talks to workers directly.
- **Conservative concurrency**: The current request-mode implementation still allows only one active request at a time.
- **Split concurrency model**: Request-mode diffusion remains single-active-request, while the step-wise path can keep multiple compatible requests running and advance them independently between denoise steps.
---