Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com> Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com> Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
1.5 KiB
1.5 KiB
title, kind, status, owners, primary_code_paths, related_code_paths, depends_on, validation_paths, upstream_refs, last_reviewed
| title | kind | status | owners | primary_code_paths | related_code_paths | depends_on | validation_paths | upstream_refs | last_reviewed | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Diffusion Parallelism | module | draft |
|
|
|
|
|
|
2026-07-16 |
Diffusion parallelism
Parallelism owns diffusion rank topology, process groups, tensor and sequence sharding, collectives, and distributed execution.
Candidate invariants
PARALLEL-INV-001: Topology has one source of truth
Rule: Rank coordinates, group membership, and parallel dimensions MUST be derived from validated configuration.
PARALLEL-INV-002: Sharding contracts are explicit
Rule: Every distributed boundary MUST define tensor shape, sharded dimension, placement, dtype, producer, and consumer.
PARALLEL-INV-003: Collectives are symmetric
Rule: Process-group members MUST invoke compatible collectives in the same logical order.
PARALLEL-INV-004: Single-rank behavior remains valid
Rule: Distributed implementations SHOULD preserve a supported single-rank path without distributed initialization.
Safe-change guide
Test topology validation, single-rank execution, affected parallel modes, combined modes, invalid shapes, and orderly teardown.