[Model] MammothModa2: enable Cache-DiT acceleration for the DiT stage (#7291)
Signed-off-by: shihongzhi <shi65881583@gmail.com>
This commit is contained in:
1 parent
0b5f8832b9
commit
c11596c637
10 files changed
+507
-14
No files matched your search
@@ -72,6 +72,39 @@ omni = Omni(
|
||||
|
||||
## Example Script
|
||||
|
||||
### Multi-Stage Models (MammothModa2)
|
||||
|
||||
MammothModa2 is a multi-stage AR→DiT pipeline; the diffusion runner owns the
|
||||
Cache-DiT lifecycle of its DiT stage. Configure it on the **DiT stage entry**
|
||||
of the deploy YAML so the acceleration only applies to the denoising stage:
|
||||
|
||||
```yaml
|
||||
# deploy YAML (see vllm_omni/deploy/mammoth_moda2.yaml)
|
||||
stages:
|
||||
- stage_id: 1
|
||||
# ... other DiT stage settings ...
|
||||
cache_backend: cache_dit
|
||||
cache_config:
|
||||
Fn_compute_blocks: 1
|
||||
Bn_compute_blocks: 0
|
||||
max_warmup_steps: 4
|
||||
residual_diff_threshold: 0.24
|
||||
max_continuous_cached_steps: 3
|
||||
enable_cache_dit_summary: true # log skip-ratio stats per request
|
||||
```
|
||||
|
||||
Behavior notes:
|
||||
|
||||
- Only the repeated main-layer stack is cached; the Q-Former refiners always run.
|
||||
- Sequential-CFG parity requires the unconditional pass on every denoise step,
|
||||
so the `cfg_range` skip optimization is not used: outside the interval CFG is
|
||||
neutralized with `scale=1.0` instead of skipping the uncond forward.
|
||||
- Requests with `text_guidance_scale = 1.0` run with cache hooks disabled.
|
||||
- `residual_diff_threshold` defaults to `0.24` (measured ~1.7-2.2x at 50 steps,
|
||||
PSNR 27-34 dB vs uncached). For 20-step generation — especially with a
|
||||
partial `cfg_range` — use `0.12` for noticeably closer parity at a modest
|
||||
speed cost.
|
||||
|
||||
### Offline Inference
|
||||
|
||||
Use the example script under `examples/offline_inference/text_to_image`:
|
||||
|
||||
@@ -130,7 +130,7 @@ The following tables show which models support each feature:
|
||||
| **Krea 2** | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ (decode) | ❌ | ❌ |
|
||||
| **LongCat-Image** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
|
||||
| **LongCat-Image-Edit** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
|
||||
| **MammothModa2(T2I)** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
|
||||
| **MammothModa2(T2I)** | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
|
||||
| **Nextstep_1(T2I)** | ❓ | ❓ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
|
||||
| **OmniGen2** | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
|
||||
| **Ovis-Image** | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
|
||||
@@ -154,6 +154,7 @@ The following tables show which models support each feature:
|
||||
> 4. Krea 2 currently supports single-GPU inference plus LoRA, Cache-DiT, HSDP, CPU/layerwise offload, and VAE-patch-parallel (decode). TP/SP/CFG-Parallel are not yet wired. The few-step distilled (Turbo) checkpoint uses `is_distilled=true` (fixed timestep shift `mu=1.15`); generate at 2048x2048 by default with `num_inference_steps≈8` and `guidance_scale=0`. The Raw checkpoint uses 1024x1024, `num_inference_steps=28`, and `guidance_scale=4.5`.
|
||||
> 5. HunyuanImage3 supports step execution. Multi-request step execution requires `TORCH_SDPA`; see [Diffusion Execution Modes](diffusion/execution_modes.md#step-execution).
|
||||
> 6. BAGEL step execution supports image generation with `bagel.yaml`, `bagel_think.yaml`, and `bagel_single_stage.yaml`; two-stage Thinker execution and explicit single-stage text output remain on their existing complete-request paths. Image requests require `num_inference_steps >= 2`. BAGEL step execution cannot currently be combined with sequence parallelism or a diffusion cache backend; see [Diffusion Execution Modes](diffusion/execution_modes.md#step-execution).
|
||||
> 7. MammothModa2 runs its DiT stage on the diffusion runner (`StageExecutionType.DIFFUSION`); Cache-DiT is enabled through the standard diffusion-stage knobs on the stage entry of the deploy YAML (`cache_backend: cache_dit`, plus optional `cache_config` / `enable_cache_dit_summary`). The runner installs the backend at startup and the pipeline adopts it per request. Only the repeated main-layer stack is cached; requests with `text_guidance_scale = 1.0` bypass the cache hooks.
|
||||
|
||||
### VideoGen
|
||||
|
||||
|
||||
Reference in new issue
Block a user