[Bugfix][Ascend NPU][Diffusion] Load over-wide unquantized fallback weights straight into host memory under DLOFix/mxfp online offload after quant (#7009)

Signed-off-by: brandneway <gyuan4892@gmail.com>
This commit is contained in:
ooooooye authored and GitHub committed 2026-09-18 07:36:48 +00:00
1 parent 5d3e6dc1b4
commit 823af92347
7 files changed
+561 -43

No files matched your search

@@ -353,9 +353,14 @@ modes can resolve the same `None` value to different schedules.
Cache-DiT on any selected component. Prompt-embedding cache is also
incompatible when the text encoder uses `allgather`. Rank-local cache hits
could otherwise make ranks enter different weight collectives.
- Per-tensor online FP8 linears use the ordinary loader and can run with either
DiT transfer path. With DiT `allgather`, every rank temporarily materializes
the complete FP8 model in host memory before DLO retains only its shard.
- Per-tensor online FP8, INT8, and MXFP8 linears use the ordinary loader and
can run with either DiT transfer path. With DiT `allgather`, every rank
temporarily materializes the complete quantized model in host memory before
DLO retains only its shard. Layers too wide for the NPU quantization kernel
(`npu_quant_matmul` rejects an output dimension past 65535) stay
unquantized and load straight into host memory under offload-after-quant —
their runtime layout is a plain contiguous bf16 weight, the same layout DLO
shards on the ordinary path, so they are allowed under `allgather` too.
Other online quantization methods require rank-local transfer for the
affected component until their runtime layouts are validated.
- Resident leading layers require DiT `rank-local` transfer and a model