[Refact] Refactor diffusion output streaming (#5166)
Signed-off-by: bjf-frz <frz123db@gmail.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
7d5376f215
commit
be667a8067
27 files changed
+628
-394
No files matched your search
@@ -39,6 +39,11 @@ vllm serve Qwen/Qwen-Image --omni \
|
||||
For serving, `--step-execution` enables the step-wise runtime. Continuous
|
||||
batching only becomes relevant when `--max-num-seqs > 1`.
|
||||
|
||||
`max_num_seqs` controls the step scheduler capacity:
|
||||
|
||||
- `max_num_seqs=1` runs one request through the step-wise state machine.
|
||||
- `max_num_seqs>1` allows compatible active requests to share step waves.
|
||||
|
||||
## Supported Pipelines
|
||||
|
||||
| Pipeline | Example models | Step execution |
|
||||
@@ -51,7 +56,7 @@ batching only becomes relevant when `--max-num-seqs > 1`.
|
||||
the step-wise path can batch
|
||||
compatible requests together. This is experimental. Requests with
|
||||
incompatible sampling parameters are intentionally kept in separate batches,
|
||||
and `max_num_seqs=1` remains the conservative default.
|
||||
and `max_num_seqs=1` remains the conservative single-request step path.
|
||||
|
||||
## Current Limitations
|
||||
|
||||
|
||||
Reference in new issue
Block a user