[Perf] Support continuous batching for MammothModa2 (#7954)

Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: nagisa <1434936049@qq.com>
This commit is contained in:
hurukawa authored and GitHub committed 2026-10-05 02:22:37 +08:00
1 parent c9c478dd07
commit b9cf74352a
5 files changed
+684 -72

No files matched your search

+18 -6
View File
@@ -18,12 +18,24 @@ The generic example formats the AR prompt, drives the AR → DiT stage pipeline,
and forwards MammothModa2-specific generation parameters through the
pipeline-declared `extra_body` contract.
MammothModa2's DiT stage runs in the shared diffusion runtime in request mode.
The first integration intentionally supports one request and one image per
forward only (`max_num_seqs: 1`, `num_outputs_per_prompt: 1`). Request-level
batching, step execution, continuous batching, compilation, quantization,
parallelism, and offload are not enabled by this recipe. TeaCache acceleration
is supported for the DiT stage.
MammothModa2's DiT stage runs in the shared diffusion runtime. The default
`mammoth_moda2.yaml` uses request-level batching with up to eight compatible
requests (`max_num_seqs: 8`) and one image per request
(`num_outputs_per_prompt: 1`). TeaCache and Cache-DiT are optional alternative
cache backends in request mode. Enable TeaCache with `--cache-backend tea_cache`
as shown below, or enable Cache-DiT using the commented settings in that YAML.
Compilation, DiT quantization, parallelism, and offload are not enabled by
these presets.
To use step execution and continuous batching, select
[`mammoth_moda2_step.yaml`](../../vllm_omni/deploy/mammoth_moda2_step.yaml)
with `--deploy-config vllm_omni/deploy/mammoth_moda2_step.yaml` in the
text-to-image commands below. This preset inherits the same stage placement
and capacity, enables step execution for Stage 1, and disables request-batch
admission waiting and diffusion cache acceleration. Compatible requests can
join between denoising steps and finish independently, including requests
with different inference-step counts. Step mode cannot be combined with
TeaCache, Cache-DiT, or other diffusion cache backends.
Image size, seed, guidance, and denoising steps use the standard diffusion
request fields. `cfg_range` remains a MammothModa2-specific `extra_body`