[Doc][skip ci] Add design doc for scheduler-managed diffusion paged KV cache (#6891)

Signed-off-by: Acerak01-fy <wfy2003324@163.com>

docs/.nav.yml

@@ -80,6 +80,8 @@ nav:
       - Cache Acceleration:
         - TeaCache: user_guide/diffusion/cache_acceleration/teacache.md
         - Cache-DiT: user_guide/diffusion/cache_acceleration/cache_dit.md
+      - KV Cache:
+        - Scheduler-Managed Paged KV Cache: user_guide/diffusion/paged_kv_cache.md
       - Parallelism:
         - Overview: user_guide/diffusion/parallelism/overview.md
         - CFG Parallel: user_guide/diffusion/parallelism/cfg_parallel.md
@@ -153,6 +155,7 @@ nav:
         - Attention Optimization:
           - Backend Selection: design/feature/attention_backend_selection.md
           - Skip-Softmax: design/feature/skip_softmax.md
+          - Scheduler-Managed Paged KV Cache: design/feature/diffusion_paged_kv_cache.md
         - CPU Offloading:
           - Overview: design/feature/offloader/README.md
           - Model-Level: design/feature/offloader/module_offload.md

docs/design/feature/diffusion_paged_kv_cache.md

@@ -0,0 +1,346 @@
+# Scheduler-Managed Paged KV Cache
+
+This document describes the design and lifecycle of Scheduler-managed paged KV cache for diffusion DiT stages.
+
+Paged KV cache is a scheduler-managed KV-cache and memory-management
+mechanism. It is separate from attention backend selection: native attention
+backends such as FlashAttention or Ascend FIA perform the attention computation,
+while paged KV provides the block layout and metadata consumed by those
+backends.
+
+The current implementation requires the `FLASH_ATTN` selector; other
+attention backends do not currently support `paged_scheduler`.
+
+For operator-facing configuration and examples, see the
+[Paged KV Cache user guide](../../user_guide/diffusion/paged_kv_cache.md).
+
+## Table of Contents
+
+1. [Overview](#overview)
+2. [Motivation](#motivation)
+3. [Architecture](#architecture)
+4. [Execution Modes](#execution-modes)
+5. [Design](#design)
+6. [Model and Platform Integration](#model-and-platform-integration)
+7. [Configuration and Compatibility](#configuration-and-compatibility)
+8. [Limitations](#limitations)
+9. [Related Files](#related-files)
+
+## Overview
+
+Scheduler-managed paged KV provides a page-based memory and data-movement
+foundation for diffusion DiT stages. It makes KV capacity explicit, preserves
+stable prefixes across denoising steps, and defines page-granular destinations
+for future AR-to-DiT transfer. HunyuanImage-3.0 is the current request-mode
+reference on CUDA and Ascend NPU; the supported boundaries and follow-up work
+are described below.
+
+## Motivation
+
+Paged KV is introduced for three related goals:
+
+1. **Memory management:** make diffusion KV capacity visible to the Scheduler
+   so logical requests can reserve and release fixed-size blocks while the
+   Worker owns one reusable physical pool. This improves capacity planning and
+   memory utilization, and can increase request concurrency and throughput when
+   the dense path would otherwise reserve separate full tensors.
+2. **Transfer optimization:** give a DiT stage stable destination page slots so
+   a future AR-to-DiT connector can transfer missing KV blocks directly,
+   instead of gathering a complete tensor into a temporary receiver buffer.
+3. **Prefix reuse:** keep the stable prompt/reference prefix resident while
+   only the timestep/image target is rewritten at each denoising step. The same
+   page-level contract also provides the foundation for canonical prefix-cache
+   reuse when a model supplies cacheable block identities.
+
+Request-local memory management and prefix reuse are implemented by the local
+request path. Cross-stage transfer and cross-request canonical prefix caching
+use the same page interfaces but require model/connector integration that is
+outside the current Hunyuan implementation.
+
+The legacy dense path keeps complete model-owned tensors for each request. The
+paged path instead maps requests to Scheduler blocks and reuses a Worker page
+pool, making capacity and fragmentation explicit. This can improve HBM
+utilization and concurrency, although it does not guarantee lower latency for
+every shape.
+
+## Architecture
+
+The control plane describes and reserves logical sequences. The data plane
+installs the resulting snapshot on each Worker and runs the native attention
+backend.
+
+```mermaid
+flowchart LR
+    subgraph CP[Control plane]
+        R[Diffusion request] --> P[Model preprocessing<br/>layout, CFG rows, spans]
+        P --> S[Diffusion Scheduler<br/>logical sequences and blocks]
+        S --> M[DiffusionKVMetadata]
+    end
+    M -->|immutable snapshot| X[Executor / RPC]
+    subgraph DP[Data plane: each Worker rank]
+        W[Model runner<br/>physical pages and block tables]
+        W --> A[Paged attention adapter]
+        A --> C[CUDA FlashAttention / FA3]
+        A --> N[Ascend FIA]
+        W --> O[Denoising output]
+    end
+    X --> W
+```
+
+### Ownership boundary
+
+| Component | Owns | Does not own |
+| --- | --- | --- |
+| Model preprocessing | Lengths, positions, CFG count, and attention spans | Physical blocks or Worker state |
+| Diffusion Scheduler | Request lifecycle, logical sequences, admission, and release | Device tensors or native kernel metadata |
+| Executor/RPC | Transport of the immutable allocation snapshot | Allocation decisions or page contents |
+| Worker/model runner | Rank-local KV tensors, rows, block tables, slots, and active metadata | Scheduler block lifetime |
+| Paged attention adapter | Layout translation, slot metadata, and backend dispatch | Capacity management or request admission |
+| Platform backend | Native page geometry, metadata construction, and kernel selection | Model-specific token semantics |
+
+`DiffusionKVCacheManager` is the request-facing Scheduler facade: it groups and
+atomically allocates the internal sequences for one request, then publishes
+`DiffusionKVMetadata` over the Worker-owned physical pool.
+
+### Request lifecycle
+
+1. **Admission:** preprocessing creates one logical sequence per CFG branch;
+   the Scheduler reserves all required blocks atomically.
+2. **Activation:** the Worker validates the metadata generation, binds rows,
+   stages block tables, and installs native attention metadata.
+3. **Execution:** the first denoising step writes the complete sequence once;
+   later steps write only the changing target span and read retained pages.
+4. **Release:** completion, cancellation, errors, and reinitialization release
+   Scheduler blocks and clear Worker rows.
+
+Dense requests do not install paged metadata. Paged requests reject legacy
+dense KV payloads or stale/incomplete native metadata instead of silently
+falling back to dense execution.
+
+### AR-to-DiT transfer contract
+
+When an AR stage and a DiT stage are deployed separately, the previous
+model-owned transfer commonly gathers and flattens a complete KV tensor into a
+temporary receiver buffer before the DiT model can use it.
+
+The Scheduler-managed contract gives the DiT Worker a destination page layout
+before transfer. The current feature implements this DiT-side destination
+contract (reservation, layout, and generation/readiness metadata); it does not
+execute connector transfers or import AR KV. A future connector can target the
+reserved slots directly:
+
+```mermaid
+flowchart LR
+    A[AR Worker<br/>source pages] --> X[Connector<br/>missing blocks]
+    S[DiT Scheduler<br/>reserve destination blocks] --> D[DiT Worker<br/>destination pages]
+    X --> D
+    D --> F[Native DiT attention]
+```
+
+The diagram is a target connector boundary, not an enabled runtime path in this
+feature. The expected benefits of a future connector using this contract are:
+
+| Concern | Previous model-owned transfer | Page-native destination contract |
+| --- | --- | --- |
+| Transfer unit | Packed or complete KV tensor | Individual missing page blocks |
+| Receiver storage | Temporary contiguous tensor, then model cache | Scheduler-reserved Worker pages |
+| Data movement | Gather/flatten plus another copy or concatenation | Direct write to destination slots when a connector is enabled |
+| Readiness | Implicit runner-side synchronization | Explicit generation and ready metadata |
+| Denoising reuse | Reassemble the stable prefix in the model | Native attention reads resident pages |
+| Cleanup | Transfer and model-cache lifetimes are coupled | Source lease, destination reservation, and request release are explicit |
+
+Reserving destination pages up front supplies stable physical targets and an
+explicit generation/ready state, so a future connector can move only missing
+blocks and pass the same metadata to native attention. Imported AR KV is not
+enabled for `paged_scheduler` in this PR; DreamZero and LingBot-World use the
+separate `ar_diffusion_kv` contract.
+
+## Execution Modes
+
+Paged KV is a cache feature; request batching and step execution are separate
+engine modes. The terminology follows the
+[diffusion continuous batching design](diffusion_continuous_batching.md):
+
+| Execution path | Configuration | Scheduler batch unit | Hunyuan `paged_scheduler` status |
+| --- | --- | --- | --- |
+| Request execution | `step_execution=false`, `max_num_seqs=1` | One complete denoising request | Current supported path |
+| Request-level batching | `step_execution=false`, `max_num_seqs>1` | Multiple compatible public requests in one forward | Not supported by the current `paged_scheduler` integration |
+| Step execution | `step_execution=true`, `max_num_seqs=1` | One request advances one denoising step per scheduler tick | Paged path rejects it |
+| Step continuous batching | `step_execution=true`, `max_num_seqs>1` | Compatible active step states advance together | Separate feature; paged Hunyuan is not implemented |
+
+`max_num_seqs` limits public requests. CFG branches are internal rows of one
+request and are controlled by `diffusion_kv_max_rows_per_request`: a request
+without CFG uses one row, while the standard positive/negative CFG request uses
+two. Atomic CFG admission does not mean that CFG is required.
+
+## Design
+
+### Startup and cache sizing
+
+Before real requests are admitted, the engine and Workers:
+
+1. Resolve `diffusion_kv_mode` and prepare a maximum-shape profile request.
+2. Register paged attention layers and collect a native `KVCacheSpec` per KV
+   group.
+3. Run the marked memory profile to determine non-KV memory. This probe is not
+   a latency sample. On NPU, it may use SDPA only when MindIE-SD is
+   unavailable; this is the sole intentional fallback and applies only to this
+   startup profile.
+4. Build the native cache configuration and resolve block geometry.
+5. Allocate rank-local physical pages and create the Scheduler facade over the
+   same geometry.
+
+`kv_cache_memory_bytes` is a physical KV-pool budget per Worker rank, not a
+request token count. When omitted, the normal utilization path sizes the pool
+after subtracting profiled non-KV memory. Reserved pool memory can exceed the
+live payload because it also reflects capacity and fragmentation.
+
+### Logical layout
+
+Each sequence has a stable prefix and a changing target:
+
+```text
+|-------------------------- allocated seq_len --------------------------|
+|---------------- prefix_len ----------------|---- target_len ----|unused|
+             retained across steps                 rewritten each step
+```
+
+The runner exposes these boundaries to the adapter:
+
+| Phase | `query_len` | `seq_len` | `kv_start_pos` |
+| --- | ---: | ---: | ---: |
+| First denoising/prefill | `seq_len` | `seq_len` | `0` |
+| Later denoising steps | `target_len` | `prefix_len + target_len` | `prefix_len` |
+
+For `block_size`, token position `p` maps to the native slot
+`block_id(p // block_size) * block_size + p % block_size`. The Scheduler owns
+the block IDs; the Worker turns them into rank-local tables and slots.
+
+### Dense and paged representation
+
+Both modes implement Hunyuan's mixed causal/full attention. They differ in KV
+representation and backend input, not in the attention result they describe:
+
+| Aspect | `dense_legacy` | `paged_scheduler` |
+| --- | --- | --- |
+| KV owner | Model-owned contiguous cache | Worker-owned physical pages |
+| Attention input | Contiguous K/V with a dense mask and/or span metadata | Block tables, slots, lengths, and `full_attn_spans` |
+| Prefix reuse | Model-owned prefix tensor; cache scope depends on the dense path | Stable prefix pages are retained across denoising steps; canonical cross-request reuse requires cache identities |
+| Scheduler state | No diffusion page reservation | Logical sequences, admission, generation, and release |
+
+The current implementation guarantees prefix reuse between the first and later
+denoising steps. Native vLLM cross-request prefix caching is not enabled by
+this Hunyuan integration: the diffusion KV manager uses
+`enable_caching=False`, and Hunyuan does not publish canonical block hashes.
+
+### Attention execution
+
+The two paths preserve the same mixed causal/full spans. In the paged path,
+the adapter converts each row into aligned native segments:
+
+| Segment | Query input | KV visibility | Causal flag |
+| --- | --- | --- | --- |
+| Causal `[s, e)` | `Q[s:e]` | `K[:e]`, `V[:e]` | `true` |
+| Full `[a, b)` | Query overlap | `K[:b]`, `V[:b]` | `false` |
+
+The K/V update is deliberately outside the segment loop:
+
+```text
+Q/K/V projection
+    -> write the current K/V span once per layer
+    -> run all causal/full native segments
+    -> restore output order
+```
+
+CUDA keeps cache-update ownership in its native paged-attention contract.
+Ascend prewrites the normal-layout K/V span once, then FIA segment calls read
+the persistent pages without receiving K/V again. Output is restored to the
+original token order before projection, residual, and MLP layers consume it.
+
+Dense does not imply one universal kernel call. Hunyuan's CUDA dense path can
+also use the shared piecewise FlashAttention helper for aligned regions. In
+the paged path, the same regions are described as native segments that read
+persistent pages. The two modes can therefore have different kernel names and
+call counts while preserving the same mask semantics.
+
+### Piecewise planning
+
+The common planner supports both row layouts:
+
+| Row layout | Preparation and output handling |
+| --- | --- |
+| Homogeneous ranges, such as CFG rows from one request | Keep the batched layout, use contiguous views/direct output-buffer writes, and avoid the large indexed output scatter |
+| Heterogeneous lengths or offsets | Gather valid tokens with `index_select`, run native segments, and scatter them back with `index_copy_` |
+
+The homogeneous fast path applies to rows in one attention invocation; it does
+not enable arbitrary public-request batching or change Scheduler allocation.
+
+## Model and Platform Integration
+
+### HunyuanImage-3.0
+
+Hunyuan is the reference integration because its self-attention mixes causal
+and full regions and its generated-image region changes across denoising
+steps. It creates one logical row per conditional or unconditional CFG branch
+and supports strict Ulysses SP in request mode. Its runner owns row creation,
+phase activation, and page metadata; the model boundary supplies token layout
+and attention spans without allocating Scheduler blocks.
+
+### Platform matrix
+
+| Model/integration | Platform | Execution | Backend | Status |
+| --- | --- | --- | --- | --- |
+| HunyuanImage-3.0 | NVIDIA CUDA | Request-level | `FLASH_ATTN` -> native FlashAttention/FA3 | Implemented and validated |
+| HunyuanImage-3.0 | Ascend NPU | Request-level | `FLASH_ATTN` -> Ascend FIA | Implemented and validated |
+| DreamZero / LingBot-World | Platform-specific | Separate `ar_diffusion_kv` contract | Imported AR-KV path | Not `paged_scheduler` |
+| Other diffusion models | Any | N/A | N/A | No paged integration yet |
+
+The logical `FLASH_ATTN` selector is the only current paged-KV capability
+advertised by `FlashAttentionBackend`. Other diffusion backends do not
+transparently fall back to dense attention for a paged request.
+
+## Configuration and Compatibility
+
+`paged_scheduler` requires a native cache configuration, a positive row limit,
+and a prepared memory-profile request. The main fields are:
+
+| Field | Meaning |
+| --- | --- |
+| `diffusion_kv_mode` | `dense_legacy` (default) or `paged_scheduler` |
+| `diffusion_kv_max_rows_per_request` | Worker row capacity for one public request, including CFG branches |
+| `kv_cache_memory_bytes` | Optional explicit physical KV-pool budget per Worker rank |
+| `gpu_memory_utilization` | Automatic pool sizing when no byte budget is supplied |
+| `diffusion_attention_backend` | Must resolve to `FLASH_ATTN` for the current paged implementation |
+
+The user guide contains the stage YAML and `vllm serve --omni` example.
+Use request execution for the current Hunyuan integration; the unsupported
+batching and step modes are summarized in [Execution Modes](#execution-modes).
+
+The current paged KV format is unquantized BF16 with identity Q/K/V scales.
+Model weight quantization, CPU offload, layerwise offload, and distributed
+layerwise offload (DLO) have not been validated with this path. Leave those
+options disabled for a validated paged setup; use `dense_legacy` when they are
+required.
+
+## Limitations
+
+- Hunyuan `paged_scheduler` is request-level only; request batching and step
+  execution are not implemented (see [Execution Modes](#execution-modes)).
+- Only strict Ulysses SP and the current two-branch CFG layout are supported;
+  Ring, AllGather-KV, and independent Hunyuan KV contexts are not implemented.
+- Imported AR-to-DiT page transfer and cross-request canonical prefix caching
+  are follow-up integrations; `ar_diffusion_kv` is a separate contract.
+- Quantized KV and CPU/DLO offload combinations are untested.
+- For a formal `paged_scheduler` request, missing native metadata or backend
+  support is an explicit error, not a dense or SDPA fallback. The marked
+  startup memory profile is the only intentional exception.
+
+## Related Files
+
+- Scheduler and allocation: [`diffusion_kv/manager.py`](gh-file:vllm_omni/diffusion/diffusion_kv/manager.py), [`diffusion_kv/request.py`](gh-file:vllm_omni/diffusion/diffusion_kv/request.py), [`diffusion_kv/metadata.py`](gh-file:vllm_omni/diffusion/diffusion_kv/metadata.py)
+- Scheduler and runtime: [`sched/base_scheduler.py`](gh-file:vllm_omni/diffusion/sched/base_scheduler.py), [`diffusion_kv/config.py`](gh-file:vllm_omni/diffusion/diffusion_kv/config.py), [`forward_context.py`](gh-file:vllm_omni/diffusion/forward_context.py), [`vllm_config.py`](gh-file:vllm_omni/diffusion/vllm_config.py)
+- Executor boundary: [`executor/abstract.py`](gh-file:vllm_omni/diffusion/executor/abstract.py), [`executor/uniproc_executor.py`](gh-file:vllm_omni/diffusion/executor/uniproc_executor.py)
+- Worker data plane: [`diffusion_kv/initialization.py`](gh-file:vllm_omni/diffusion/diffusion_kv/initialization.py), [`diffusion_kv/model_runner_backend.py`](gh-file:vllm_omni/diffusion/diffusion_kv/model_runner_backend.py), [`worker/diffusion_model_runner.py`](gh-file:vllm_omni/diffusion/worker/diffusion_model_runner.py)
+- Attention adapter and planner: [`diffusion_kv/paged_attention_adapter.py`](gh-file:vllm_omni/diffusion/diffusion_kv/paged_attention_adapter.py), [`attention/layer.py`](gh-file:vllm_omni/diffusion/attention/layer.py), [`attention/backends/flash_attn.py`](gh-file:vllm_omni/diffusion/attention/backends/flash_attn.py), [`attention/backends/utils/piecewise_attn.py`](gh-file:vllm_omni/diffusion/attention/backends/utils/piecewise_attn.py)
+- Model boundary: [`models/hunyuan_image3/request_layout.py`](gh-file:vllm_omni/diffusion/models/hunyuan_image3/request_layout.py), [`models/hunyuan_image3/pipeline_hunyuan_image3.py`](gh-file:vllm_omni/diffusion/models/hunyuan_image3/pipeline_hunyuan_image3.py), [`models/hunyuan_image3/hunyuan_image3_transformer.py`](gh-file:vllm_omni/diffusion/models/hunyuan_image3/hunyuan_image3_transformer.py)
+- Platform hooks: [`platforms/interface.py`](gh-file:vllm_omni/platforms/interface.py), [`platforms/cuda/platform.py`](gh-file:vllm_omni/platforms/cuda/platform.py), [`platforms/npu/platform.py`](gh-file:vllm_omni/platforms/npu/platform.py)

docs/design/index.md

@@ -57,6 +57,10 @@ implementation contract; it is not, by itself, a general support claim.
 - [Tensor Parallel](feature/tensor_parallel.md)
 - [VAE Patch Parallelism](feature/vae_parallel.md)
 
+#### KV cache and memory management
+
+- [Scheduler-Managed Paged KV Cache for Diffusion DiT Stages](feature/diffusion_paged_kv_cache.md)
+
 #### Attention optimization
 
 The [Diffusion Attention Backends](../user_guide/diffusion/attention_backends.md)

docs/user_guide/diffusion/execution_modes.md

@@ -108,7 +108,12 @@ set `DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA` or configure
 `diffusion_attention_config.default.backend=TORCH_SDPA` before using
 `--max-num-seqs >1`. See the
 [HunyuanImage-3.0 recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/Tencent/HunyuanImage-3.0-Instruct.md)
-for its validated configuration. Helios supports single-request step execution only: use
+for its validated configuration. For HunyuanImage3, this step-execution
+support applies to the `dense_legacy` path; `paged_scheduler` currently
+supports request-level execution only. See the
+[Scheduler-Managed Paged KV Cache guide](paged_kv_cache.md) for its required
+backend and configuration. Helios supports single-request step
+execution only: use
 `--step-execution --max-num-seqs 1` for Helios. MiniMax H3 supports step-wise
 continuous batching by packing co-batched requests into one sequence that keeps
 a separate attention document per request; that layout needs a backend which

docs/user_guide/diffusion/paged_kv_cache.md

@@ -0,0 +1,172 @@
+# Scheduler-Managed Paged KV Cache
+
+Scheduler-managed paged KV cache stores diffusion DiT key/value tensors in
+Worker-owned pages. It is useful when a denoising request should keep its
+stable prompt/reference prefix in device memory while rewriting only the
+changing image span on later steps. The current HunyuanImage-3.0 integration
+is request-level only; it does not silently change an unsupported request to
+dense attention.
+
+Paging changes KV storage and scheduling ownership; it does not change the
+model's causal/full attention semantics.
+
+## Choose a mode
+
+| Need | Mode |
+| --- | --- |
+| Broadest compatibility or step execution | `dense_legacy` (default) |
+| HunyuanImage-3.0 request-mode serving with Scheduler-managed KV pages | `paged_scheduler` |
+
+Use `dense_legacy` when the model, attention backend, or offload combination
+has not been validated for paged KV. HunyuanImage-3.0 paged KV currently accepts
+request-level execution only; set `step_execution: false`.
+
+## Supported scope
+
+| Model | Platform | Execution | Backend | Status |
+| --- | --- | --- | --- | --- |
+| HunyuanImage-3.0 | NVIDIA CUDA | Request-level | `FLASH_ATTN` (native FlashAttention/FA3) | Implemented and validated |
+| HunyuanImage-3.0 | Ascend NPU | Request-level | `FLASH_ATTN` (Ascend FIA) | Implemented and validated |
+| Other diffusion models | Any | N/A | Any | Not integrated or tested |
+
+Only the logical `FLASH_ATTN` backend currently advertises paged-KV support.
+`TORCH_SDPA`, `CUDNN_ATTN`, `FLASHINFER_ATTN`, SAGE, Hub, and other diffusion
+backends do not transparently fall back to dense execution for a paged request.
+
+Strict Ulysses SP and two-branch CFG parallel are supported in HunyuanImage-3.0
+request mode when the selected device topology has been validated. Ring SP,
+AllGather-KV SP, Hunyuan paged step execution, and independent public-request
+batching are not supported by this path. DreamZero and LingBot-World use the
+separate `ar_diffusion_kv` imported-AR-KV contract.
+
+## Configure
+
+The diffusion KV fields are stage settings. The two
+`diffusion_kv_*` fields are diffusion-specific engine fields and are forwarded
+through `engine_extras`; `kv_cache_memory_bytes` is a standard stage field and
+is placed directly under the stage.
+
+Set `pipeline: hunyuan_image3_dit` to select the standalone DiT pipeline,
+where stage `0` is the diffusion stage.
+
+```yaml
+pipeline: hunyuan_image3_dit
+
+stages:
+  - stage_id: 0
+    max_num_seqs: 1
+    enforce_eager: true
+    diffusion_attention_backend: FLASH_ATTN
+    step_execution: false
+    kv_cache_memory_bytes: 536870912  # 512 MiB per Worker/rank
+    engine_extras:
+      diffusion_kv_mode: paged_scheduler
+      diffusion_kv_max_rows_per_request: 2
+```
+
+| Field | Required | Meaning |
+| --- | --- | --- |
+| `diffusion_kv_mode` | Yes | `paged_scheduler` enables Scheduler-managed pages; `dense_legacy` is the default |
+| `diffusion_kv_max_rows_per_request` | Yes for paged | Positive Worker row limit, including all internal CFG branches; use `1` without CFG and `2` for standard CFG |
+| `kv_cache_memory_bytes` | Optional | Explicit physical KV-pool budget per Worker/rank; omit it to use automatic sizing |
+| `gpu_memory_utilization` | Optional | Automatic pool-sizing fraction when no explicit byte budget is set |
+| `max_num_seqs` | Recommended `1` | Public request capacity; it is separate from the internal CFG row limit |
+| `step_execution` | Must be `false` | Hunyuan paged KV is request-level only |
+| `diffusion_attention_backend` | `FLASH_ATTN` | Required logical backend for the current paged implementation |
+
+`kv_cache_memory_bytes` describes pool capacity, not live payload. Reserved
+memory can therefore exceed the memory currently used by active requests.
+The Scheduler allocates all rows belonging to one request together: a request
+with CFG disabled has one row, while standard two-branch CFG has two rows. CFG
+rows are internal branches, not independent public requests.
+
+Paged KV is currently unquantized BF16. `diffusion_kv_cache_dtype` does not
+enable quantized paged KV, and model weight quantization has not been validated
+in combination with this mode. CPU offload, layerwise offload, and distributed
+layerwise offload (DLO) are currently untested with paged KV. Leave these
+options disabled for paged KV; use `dense_legacy` when offload is required.
+
+## Hunyuan request-mode example
+
+Create a small deploy overlay (for example,
+`hunyuan_image3_paged.yaml`) with the stage settings above, then start the
+validated Hunyuan request-mode path:
+
+```bash
+vllm serve tencent/HunyuanImage-3.0-Instruct \
+  --omni \
+  --trust-remote-code \
+  --port 8091 \
+  --deploy-config /path/to/hunyuan_image3_paged.yaml
+```
+
+The base Hunyuan DiT deploy file can be reused and overridden from the command
+line when a separate overlay is inconvenient:
+
+```bash
+vllm serve tencent/HunyuanImage-3.0-Instruct \
+  --omni \
+  --trust-remote-code \
+  --deploy-config vllm_omni/deploy/hunyuan_image3_dit.yaml \
+  --stage-overrides \
+  '{"0":{"diffusion_kv_mode":"paged_scheduler","diffusion_kv_max_rows_per_request":2,"kv_cache_memory_bytes":536870912,"step_execution":false,"diffusion_attention_backend":"FLASH_ATTN"}}'
+```
+
+After the server is ready, send the normal Hunyuan image-generation request:
+
+```bash
+curl -s http://localhost:8091/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "modalities": ["image"],
+    "messages": [{"role": "user", "content": "A red ceramic vase on a wooden table"}],
+    "extra_body": {
+      "height": 1024,
+      "width": 1024,
+      "num_inference_steps": 20,
+      "guidance_scale": 5.0,
+      "seed": 1234
+    }
+  }' | jq -r '.choices[0].message.content[0].image_url.url' \
+    | cut -d',' -f2- | base64 -d > hunyuan_image3_paged.png
+```
+
+The same configuration is shown in the
+[HunyuanImage-3.0 recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/Tencent/HunyuanImage-3.0-Instruct.md).
+
+Confirm from startup and worker logs that the resolved backend is
+`FLASH_ATTN` and that the paged native path is active. A paged request must
+fail explicitly if its Worker metadata or native backend is unavailable; it
+must not silently switch to dense attention.
+
+## Parallelism
+
+Hunyuan request-mode paged KV can be combined with tensor parallelism, strict
+Ulysses SP, or two-way CFG parallel when the selected topology has been
+validated. Configure the topology in the stage's existing `parallel_config`:
+
+| Setting | User-visible effect | Paged-KV requirement |
+| --- | --- | --- |
+| `tensor_parallel_size` | Shards model weights across ranks | Device count must match the stage topology |
+| `sequence_parallel_size` + strict Ulysses | Shards the sequence across ranks | Ring and AllGather-KV modes are not supported |
+| `cfg_parallel_size: 2` | Places the positive and negative branches on separate ranks | Set `diffusion_kv_max_rows_per_request` to at least `2` |
+
+`max_num_seqs` controls public request concurrency and is independent of the
+internal CFG row limit. Keep `max_num_seqs: 1` for the current Hunyuan paged
+path; increasing it does not enable independent request batching.
+
+## Troubleshooting
+
+- `paged_scheduler requires diffusion_kv_max_rows_per_request`: add a positive
+  stage value and include all CFG rows.
+- Unsupported backend error: select `FLASH_ATTN`; the other attention backends
+  currently do not implement the paged-KV capability.
+- Step-mode rejection: remove `step_execution: true` and use request-level
+  execution, or select `dense_legacy` for step execution.
+- Imported AR-KV rejection: `paged_scheduler` does not consume the
+  `ar_diffusion_kv` contract used by DreamZero or LingBot-World.
+- Missing image output: include `"modalities": ["image"]` when using the chat
+  completions endpoint, and decode the returned data URI as shown above.
+
+For the ownership and lifecycle contract, see the
+[Scheduler-managed paged KV design](../../design/feature/diffusion_paged_kv_cache.md).

docs/user_guide/diffusion_features.md

@@ -248,6 +248,7 @@ The Diffusion Acceleration navigation groups the remaining guides as follows:
 | Compatibility | [Feature Compatibility](feature_compatibility.md) |
 | CPU offloading | [CPU Offloading](diffusion/cpu_offload.md) |
 | Cache acceleration | [TeaCache](diffusion/cache_acceleration/teacache.md), [Cache-DiT](diffusion/cache_acceleration/cache_dit.md) |
+| KV cache paging | [Scheduler-Managed Paged KV Cache](diffusion/paged_kv_cache.md) |
 | Parallelism | [Parallelism Overview](diffusion/parallelism/overview.md) |
 | Attention | [Attention Backends](diffusion/attention_backends.md) |
 | Compilation | [Regional Compilation](diffusion/regional_compilation.md) |

recipes/Tencent/HunyuanImage-3.0-Instruct.md

@@ -297,6 +297,56 @@ of:
   the TP=4 configuration before increasing batch size. GPU memory utilization
   is not a useful primary tuning knob for this DiT-only recipe.
 
+### Scheduler-managed paged KV (request mode)
+
+HunyuanImage-3 can use Scheduler-managed paged KV in request-level execution.
+The current paged implementation uses unquantized BF16 KV and requires the
+logical `FLASH_ATTN` backend. The following command reuses the standard DiT
+deploy file and supplies the paged-only stage settings with
+`--stage-overrides`:
+
+For the mode-selection guide, configuration field reference, and parallelism
+limits, see the [Scheduler-Managed Paged KV Cache user guide](../../docs/user_guide/diffusion/paged_kv_cache.md).
+
+```bash
+vllm serve tencent/HunyuanImage-3.0-Instruct \
+  --omni \
+  --trust-remote-code \
+  --port 8091 \
+  --deploy-config vllm_omni/deploy/hunyuan_image3_dit.yaml \
+  --stage-overrides \
+  '{"0":{"diffusion_kv_mode":"paged_scheduler","diffusion_kv_max_rows_per_request":2,"kv_cache_memory_bytes":536870912,"step_execution":false,"diffusion_attention_backend":"FLASH_ATTN"}}'
+```
+
+Here `kv_cache_memory_bytes` reserves a 512 MiB physical KV pool per Worker
+and rank, and `diffusion_kv_max_rows_per_request=2` reserves the two internal
+rows used by standard positive/negative CFG. Use `1` when CFG is disabled.
+Keep `max_num_seqs=1`: this paged Hunyuan path does not currently batch
+independent public requests.
+
+Generate an image after `/health` reports ready:
+
+```bash
+curl -s http://localhost:8091/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "modalities": ["image"],
+    "messages": [{"role": "user", "content": "A red ceramic vase on a wooden table"}],
+    "extra_body": {
+      "height": 1024,
+      "width": 1024,
+      "num_inference_steps": 20,
+      "guidance_scale": 5.0,
+      "seed": 1234
+    }
+  }' | jq -r '.choices[0].message.content[0].image_url.url' \
+    | cut -d',' -f2- | base64 -d > hunyuan_image3_paged.png
+```
+
+This mode is also implemented on Ascend NPU, where `FLASH_ATTN` resolves to
+Ascend FIA. Hunyuan paged step execution, quantized paged KV, and CPU/DLO
+offload combinations are not validated; use `dense_legacy` for those cases.
+
 ### 2x B200 ModelOpt mixed FP8/NVFP4
 
 #### Environment