[Doc][NPU] Keep AllGather for MiniMax-H3 INT8 DLO (#7444)
Signed-off-by: KrystalRay <keeleiray@gmail.com> Co-authored-by: KrystalRay <keeleiray@gmail.com>
This commit is contained in:
1 parent
e284d907b5
commit
6505577459
1 file changed
+3
-2
@@ -252,8 +252,9 @@ throughput guarantee.
|
||||
supports the H3 native `tile` mode only.
|
||||
- RainFusion block-sparse attention and INT8 quantization are validated for
|
||||
T2VA only; use the BF16 dense configuration for FL2VA and Ref2VA.
|
||||
- Online quantization cannot be combined with distributed layerwise offload
|
||||
while AllGather is enabled; pass `--dlo-no-use-allgather` in that case.
|
||||
- Online INT8 quantization can be combined with distributed layerwise offload
|
||||
while AllGather is enabled. Keep AllGather enabled for SP>1; use
|
||||
`--dlo-no-use-allgather` only when collective communication is unavailable.
|
||||
|
||||
## Additional resources
|
||||
|
||||
|
||||
Reference in new issue
Block a user