[Attention] Refine TRTLLM attention support for MiniMax H3 (#5779)
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com> Signed-off-by: lishunyang12 <lishunyang03@gmail.com> Signed-off-by: lishunyang12 <lishunyang12@163.com> Co-authored-by: lishunyang12 <lishunyang03@gmail.com> Co-authored-by: lishunyang12 <lishunyang12@163.com>
This commit is contained in:
11 files changed
+370
-33
No files matched your search
@@ -1434,6 +1434,10 @@ class SkipSoftmaxSpec:
|
||||
if self.target_sparsity is not None and self.threshold is not None:
|
||||
raise ValueError("skip_softmax: set either target_sparsity or threshold, not both.")
|
||||
|
||||
@property
|
||||
def enabled(self) -> bool:
|
||||
return self.threshold is not None or self.target_sparsity is not None
|
||||
|
||||
|
||||
@dataclass
|
||||
class AttnQuantSpec:
|
||||
|
||||
Reference in new issue
Block a user