[Refact] Refactor diffusion output streaming (#5166)
Signed-off-by: bjf-frz <frz123db@gmail.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
7d5376f215
commit
be667a8067
27 files changed
+628
-394
No files matched your search
@@ -32,7 +32,7 @@ Current engine/runtime limitations:
|
||||
|
||||
- Continuous batching with `max_num_seqs > 1` is experimental and documented in
|
||||
[Continuous Batching for Step-Wise Diffusion](diffusion_continuous_batching.md).
|
||||
Keep `max_num_seqs=1` if you want the older conservative behavior.
|
||||
Keep `max_num_seqs=1` for the conservative single-request step path.
|
||||
- `cache_backend` is not supported in step mode.
|
||||
- Request-mode extras such as KV transfer are not wired into step mode yet.
|
||||
- Unsupported pipelines now fail early during model loading instead of failing on the first request.
|
||||
|
||||
Reference in new issue
Block a user