[Core][Diffusion] Refactor prompt updates into a modality-ready interaction framework (#6294)
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
This commit is contained in:
1 parent
eeee257913
commit
2df95de421
18 files changed
+1079
-362
No files matched your search
@@ -8,7 +8,7 @@ are exposed.
|
||||
## Choose a Mode
|
||||
|
||||
| Goal | CLI configuration |
|
||||
|---|---|
|
||||
| --- | --- |
|
||||
| Serial request execution | `--max-num-seqs 1` |
|
||||
| Fused request-level batching | `--max-num-seqs N` |
|
||||
| Single-request step execution | `--step-execution --max-num-seqs 1` |
|
||||
@@ -177,8 +177,10 @@ wait
|
||||
The scheduler may batch these requests when their sampling parameters are
|
||||
compatible. See the
|
||||
[Image Generation API](../../serving/image_generation_api.md) for response
|
||||
formats and additional client examples. Streaming models can use
|
||||
model-specific streaming endpoints documented by their serving guide.
|
||||
formats and additional client examples.
|
||||
|
||||
Streaming models can use model-specific streaming endpoints documented at
|
||||
[Streaming Video Output API](../../serving/streaming_video_output_api.md).
|
||||
|
||||
## Python API
|
||||
|
||||
@@ -241,7 +243,7 @@ For step execution, set `step_execution: true` and remove
|
||||
## CLI Reference
|
||||
|
||||
| Flag | Default | Effect |
|
||||
|---|---:|---|
|
||||
| --- | ---: | --- |
|
||||
| `--step-execution` | disabled | Select step-wise scheduling |
|
||||
| `--max-num-seqs` | `1` for diffusion stages | Set request- or step-scheduler capacity |
|
||||
| `--request-batch-max-wait-ms` | `0` | Wait for burst coalescing in request mode |
|
||||
|
||||
Reference in new issue
Block a user