diff --git a/docs/design/skip_softmax_sage_trtllm_gen.md b/docs/design/skip_softmax_sage_trtllm_gen.md index b7fc173a0..f783973b5 100644 --- a/docs/design/skip_softmax_sage_trtllm_gen.md +++ b/docs/design/skip_softmax_sage_trtllm_gen.md @@ -44,12 +44,14 @@ class TrtllmGenConfig: # mirrors TRT-LLM SkipSoftmaxAttentionCon disabled_until_timestep: float = 0.0 # keep initial (noisy) steps dense; per-model preset ``` -| CLI flag | env var | default | meaning | -|----------|---------|:-------:|---------| -| `--diffusion-attention-backend trtllm-gen` | `DIFFUSION_ATTENTION_BACKEND=trtllm-gen` | — | select backend (**BF16 dense**) | -| `--trtllm-gen-sage` | `TRTLLM_GEN_SAGE=1` | **off** | FP8-SAGE | -| `--trtllm-gen-target-sparsity ` | `TRTLLM_GEN_TARGET_SPARSITY` | unset (no skip) | skip amount; needs calibrated ckpt | -| `--trtllm-gen-disabled-until-timestep ` | `TRTLLM_GEN_DISABLED_UNTIL_TIMESTEP` | per-model preset | keep initial noisy steps dense (fidelity guard); normalized denoise t 1→0 | +| CLI flag | default | meaning | +|----------|:-------:|---------| +| `--diffusion-attention-backend trtllm-gen` | — | BF16 dense | +| `--trtllm-gen-sage` | off | FP8-SAGE | +| `--trtllm-gen-target-sparsity ` | unset | skip amount (needs calibrated ckpt) | +| `--trtllm-gen-disabled-until-timestep ` | preset | keep initial steps dense (t 1→0) | + +Env var = flag in UPPER_SNAKE (e.g. `TRTLLM_GEN_SAGE=1`). **Maps to upstream:**