anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

docs: simplify upstream-mapping table

Browse Source
This commit is contained in:
lishunyang12 committed 2026-07-15 09:58:29 +00:00
1 parent ef426192b5
commit c74176c065
1 file changed
+5 -5
Download Patch File Download Diff File Expand all files Collapse all files
docs/design/skip_softmax_sage_trtllm_gen.md
+5 -5
View File
@@ -53,11 +53,11 @@ class TrtllmGenConfig: # mirrors TRT-LLM SkipSoftmaxAttentionCon
**Maps to upstream:**
| vLLM-Omni knob | TRT-LLM / ModelOpt | FlashInfer kernel arg |
|----------------|--------------------|-----------------------|
| `sage` | SAGE (separate axis; not in `SkipSoftmaxAttentionConfig`) | `sage_block_sizes=(q,k,v)` + `bmm1_scale`/`bmm2_scale` |
| `target_sparsity` | ModelOpt calibration input → config file maps to `threshold_scale_factor` | `skip_softmax_threshold_scale_factor` (threshold = factor / seqlen) |
| `disabled_until_timestep` | — (DiT-only; not in TRT-LLM's LLM config) | host-side gate, no kernel arg |
| vLLM-Omni | TRT-LLM / ModelOpt | FlashInfer |
|-----------|--------------------|------------|
| `sage` | — | `sage_block_sizes` + `bmm1/bmm2_scale` |
| `target_sparsity` | `threshold_scale_factor` (via calibration) | `skip_softmax_threshold_scale_factor` |
| `disabled_until_timestep` | — (DiT-only) | host gate |
---