[Feature] gen-only FP8 Quantization Support for SenseNova-U1 (#3943)

Signed-off-by: leohuang257 <masugchds@gmail.com>
This commit is contained in:
Guanzhe (Leo) Huang authored and GitHub committed 2026-06-05 02:51:11 +00:00
1 parent 95d56cf062
commit b63b36ded3
4 files changed
+318 -1

No files matched your search

+11
View File
@@ -52,6 +52,17 @@ guide. FP8 on Ampere may use a weight-only path where available.
Audio encoder, vision encoder, talker, and code2wav stay in BF16 unless a
model-specific guide says otherwise.
### MoT Diffusion Model (SenseNova-U1)
| Model | HF models | Scope | Status | Notes |
|-------|-----------|-------|--------|-------|
| SenseNova-U1 | `SenseNova/SenseNova-U1-8B-MoT` | Gen-path only (`*_mot_gen` layers) | Validated | Understanding-path layers stay in BF16; no `ignored_layers` needed |
SenseNova-U1 uses Mixture-of-Tokenizers (MoT) with duplicate weights per
layer: understanding path (`qkv_proj`, `mlp`) and generation path
(`qkv_proj_mot_gen`, `mlp_mot_gen`). When `quantization="fp8"` is set,
the pipeline automatically applies FP8 only to gen-path GEMMs.
### Multi-Stage Diffusion Model (BAGEL, GLM-Image)
| Model | Scope | Status | Notes |