[Doc][skip ci] Add design doc for scheduler-managed diffusion paged KV cache (#6891)
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
This commit is contained in:
1 parent
0570b0da7e
commit
303edff930
7 files changed
+582
-1
No files matched your search
@@ -108,7 +108,12 @@ set `DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA` or configure
|
||||
`diffusion_attention_config.default.backend=TORCH_SDPA` before using
|
||||
`--max-num-seqs >1`. See the
|
||||
[HunyuanImage-3.0 recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/Tencent/HunyuanImage-3.0-Instruct.md)
|
||||
for its validated configuration. Helios supports single-request step execution only: use
|
||||
for its validated configuration. For HunyuanImage3, this step-execution
|
||||
support applies to the `dense_legacy` path; `paged_scheduler` currently
|
||||
supports request-level execution only. See the
|
||||
[Scheduler-Managed Paged KV Cache guide](paged_kv_cache.md) for its required
|
||||
backend and configuration. Helios supports single-request step
|
||||
execution only: use
|
||||
`--step-execution --max-num-seqs 1` for Helios. MiniMax H3 supports step-wise
|
||||
continuous batching by packing co-batched requests into one sequence that keeps
|
||||
a separate attention document per request; that layout needs a backend which
|
||||
|
||||
Reference in new issue
Block a user