Files
vllm-omni/docs/design/module/diffusion/parallelism.md
T
2026-08-10 19:25:03 +08:00

1.5 KiB

title, kind, status, owners, primary_code_paths, related_code_paths, depends_on, validation_paths, upstream_refs, last_reviewed
title kind status owners primary_code_paths related_code_paths depends_on validation_paths upstream_refs last_reviewed
Diffusion Parallelism module draft
@princepride
@yuanheng-zhao
@xuechendi
@Bounty-hunter
vllm_omni/diffusion/distributed/**
vllm_omni/diffusion/attention/parallel/**
vllm_omni/distributed/**
vllm_omni/config/composable_parallel/**
diffusion_runtime.md
../vllm_omni_config.md
tests/diffusion/distributed/**
tests/diffusion/attention/**
torch.distributed
2026-07-16

Diffusion parallelism

Parallelism owns diffusion rank topology, process groups, tensor and sequence sharding, collectives, and distributed execution.

Candidate invariants

PARALLEL-INV-001: Topology has one source of truth

Rule: Rank coordinates, group membership, and parallel dimensions MUST be derived from validated configuration.

PARALLEL-INV-002: Sharding contracts are explicit

Rule: Every distributed boundary MUST define tensor shape, sharded dimension, placement, dtype, producer, and consumer.

PARALLEL-INV-003: Collectives are symmetric

Rule: Process-group members MUST invoke compatible collectives in the same logical order.

PARALLEL-INV-004: Single-rank behavior remains valid

Rule: Distributed implementations SHOULD preserve a supported single-rank path without distributed initialization.

Safe-change guide

Test topology validation, single-rank execution, affected parallel modes, combined modes, invalid shapes, and orderly teardown.