[Refact] Refactor diffusion output streaming (#5166)
Signed-off-by: bjf-frz <frz123db@gmail.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
7d5376f215
commit
be667a8067
27 files changed
+628
-394
No files matched your search
@@ -377,7 +377,12 @@ Default: `"0"`
|
||||
|
||||
#### `engine_args.max_num_seqs`
|
||||
|
||||
The maximum number of sequences for concurrent processing in this stage. For LLM stages, this controls the vLLM scheduler's maximum concurrent sequences. For all stage types, this also controls how many tasks can be batched together in the task processing loop.
|
||||
The maximum number of sequences for concurrent processing in this stage. For
|
||||
LLM stages, this controls the vLLM scheduler's maximum concurrent sequences.
|
||||
For diffusion stages, this controls scheduler wave capacity in both request
|
||||
batch mode and step batch mode. `max_num_seqs: 1` is the serial/conservative
|
||||
path; values above `1` allow compatible requests to batch when the selected
|
||||
diffusion pipeline supports that batching mode.
|
||||
|
||||
Default: `1`
|
||||
|
||||
|
||||
Reference in new issue
Block a user