[Refact] Refactor diffusion output streaming (#5166)
Signed-off-by: bjf-frz <frz123db@gmail.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
7d5376f215
commit
be667a8067
27 files changed
+628
-394
No files matched your search
@@ -139,7 +139,8 @@ compatibility gating and runner-side `StepInputBatch` packing.
|
||||
|
||||
## Current Limitations
|
||||
|
||||
- Experimental feature; use `max_num_seqs=1` for the older conservative path.
|
||||
- Experimental feature; use `max_num_seqs=1` for the conservative single-request
|
||||
step path.
|
||||
- Only native pipelines that already support `step_execution=True`.
|
||||
- Only homogeneous batches keyed by `StepBatchSamplingParamsKey` are supported.
|
||||
- `cache_backend`, KV transfer, and other request-mode extras are not wired
|
||||
|
||||
@@ -32,7 +32,7 @@ Current engine/runtime limitations:
|
||||
|
||||
- Continuous batching with `max_num_seqs > 1` is experimental and documented in
|
||||
[Continuous Batching for Step-Wise Diffusion](diffusion_continuous_batching.md).
|
||||
Keep `max_num_seqs=1` if you want the older conservative behavior.
|
||||
Keep `max_num_seqs=1` for the conservative single-request step path.
|
||||
- `cache_backend` is not supported in step mode.
|
||||
- Request-mode extras such as KV transfer are not wired into step mode yet.
|
||||
- Unsupported pipelines now fail early during model loading instead of failing on the first request.
|
||||
|
||||
Reference in new issue
Block a user