docs: reorganize module design documents
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
238fc0a609
commit
80fd96c1da
23 files changed
+952
-1537
No files matched your search
@@ -0,0 +1,51 @@
|
||||
---
|
||||
title: Diffusion Continuous Batching
|
||||
kind: module
|
||||
status: draft
|
||||
owners:
|
||||
- "@Isotr0py"
|
||||
- "@princepride"
|
||||
primary_code_paths:
|
||||
- vllm_omni/diffusion/sched/**
|
||||
related_code_paths:
|
||||
- vllm_omni/diffusion/executor/**
|
||||
depends_on:
|
||||
- diffusion_runtime.md
|
||||
validation_paths:
|
||||
- tests/diffusion/batching/**
|
||||
upstream_refs: []
|
||||
last_reviewed: 2026-07-16
|
||||
---
|
||||
|
||||
# Diffusion continuous batching
|
||||
|
||||
Continuous batching defines when diffusion requests are compatible, how they
|
||||
share an execution step, and how each request advances independently.
|
||||
|
||||
## Candidate invariants
|
||||
|
||||
### BATCH-INV-001: Compatibility is explicit
|
||||
|
||||
**Rule:** Requests MAY share a batch only when all properties required by the
|
||||
selected pipeline, scheduler step, shape, precision, and execution features are
|
||||
compatible.
|
||||
|
||||
### BATCH-INV-002: Per-request state remains isolated
|
||||
|
||||
**Rule:** Batch assembly MUST preserve each request's random generator, progress,
|
||||
conditioning, outputs, cancellation, and errors.
|
||||
|
||||
### BATCH-INV-003: Admission is bounded
|
||||
|
||||
**Rule:** Admission MUST respect configured capacity and MUST NOT rely on an
|
||||
unbounded waiting or active-request collection.
|
||||
|
||||
### BATCH-INV-004: Batch order is not completion order
|
||||
|
||||
**Rule:** Output association MUST use stable request identity rather than batch
|
||||
position across execution steps.
|
||||
|
||||
## Safe-change guide
|
||||
|
||||
Test heterogeneous requests, partial completion, cancellation, deterministic
|
||||
seeds, capacity limits, and batch sizes of one and many.
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
title: Diffusion Model Integration
|
||||
kind: module
|
||||
status: draft
|
||||
owners:
|
||||
- "@Isotr0py"
|
||||
- "@wtomin"
|
||||
- "@ZJY0516"
|
||||
primary_code_paths:
|
||||
- vllm_omni/diffusion/models/**
|
||||
- vllm_omni/diffusion/model_loader/**
|
||||
related_code_paths:
|
||||
- vllm_omni/diffusion/layers/**
|
||||
- vllm_omni/diffusion/lora/**
|
||||
- vllm_omni/diffusion/utils/**
|
||||
depends_on:
|
||||
- diffusion_runtime.md
|
||||
- ../input_output_modality_contracts.md
|
||||
validation_paths:
|
||||
- tests/diffusion/models/**
|
||||
- tests/diffusion/model_loader/**
|
||||
- tests/diffusion/layers/**
|
||||
- tests/diffusion/lora/**
|
||||
upstream_refs:
|
||||
- diffusers.DiffusionPipeline
|
||||
last_reviewed: 2026-07-16
|
||||
---
|
||||
|
||||
# Diffusion model integration
|
||||
|
||||
Diffusion model integration owns pipeline contracts, registration, checkpoint
|
||||
loading, adapters, shared layers, and model-specific processing.
|
||||
|
||||
## Candidate invariants
|
||||
|
||||
### DIFF-MODEL-INV-001: Pipelines implement one runtime contract
|
||||
|
||||
**Rule:** A pipeline MUST declare its supported modalities, configuration,
|
||||
inputs, outputs, loading path, and runtime capabilities.
|
||||
|
||||
### DIFF-MODEL-INV-002: Registration is the selection boundary
|
||||
|
||||
**Rule:** Runtime code MUST select model implementations through the registry or
|
||||
loader contract rather than scattered model-name conditionals.
|
||||
|
||||
### DIFF-MODEL-INV-003: Model code does not schedule requests
|
||||
|
||||
**Rule:** Pipeline code MUST NOT own admission, batching, cancellation, or
|
||||
cross-stage routing.
|
||||
|
||||
### DIFF-MODEL-INV-004: Shared behavior stays shared
|
||||
|
||||
**Rule:** Model directories SHOULD contain only genuine model differences.
|
||||
|
||||
## Safe-change guide
|
||||
|
||||
Test registry selection, checkpoint loading, minimal inference, input and output
|
||||
contracts, and every declared optional capability.
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
title: Diffusion Runtime
|
||||
kind: module
|
||||
status: draft
|
||||
owners:
|
||||
- "@Isotr0py"
|
||||
- "@princepride"
|
||||
- "@SamitHuang"
|
||||
primary_code_paths:
|
||||
- vllm_omni/diffusion/sched/**
|
||||
- vllm_omni/diffusion/executor/**
|
||||
- vllm_omni/diffusion/worker/**
|
||||
related_code_paths:
|
||||
- vllm_omni/diffusion/hooks/**
|
||||
- vllm_omni/diffusion/postprocess/**
|
||||
depends_on:
|
||||
- ../engine_orchestration.md
|
||||
- ../input_output_modality_contracts.md
|
||||
- diffusion_model_integration.md
|
||||
validation_paths:
|
||||
- tests/diffusion/diffusion_backend/**
|
||||
- tests/diffusion/hooks/**
|
||||
upstream_refs:
|
||||
- diffusers.DiffusionPipeline
|
||||
last_reviewed: 2026-07-16
|
||||
---
|
||||
|
||||
# Diffusion runtime
|
||||
|
||||
The diffusion runtime owns request admission, scheduling, execution, progress,
|
||||
output, cancellation, and cleanup inside a diffusion stage.
|
||||
|
||||
## Candidate invariants
|
||||
|
||||
### DIFF-RUNTIME-INV-001: One lifecycle owner
|
||||
|
||||
**Rule:** Every admitted request MUST have exactly one scheduler-owned lifecycle
|
||||
until completion, cancellation, or failure.
|
||||
|
||||
### DIFF-RUNTIME-INV-002: Execution follows scheduler output
|
||||
|
||||
**Rule:** Executors and workers MUST execute scheduler decisions without
|
||||
admitting, reordering, or forwarding requests independently.
|
||||
|
||||
### DIFF-RUNTIME-INV-003: Terminal cleanup is complete
|
||||
|
||||
**Rule:** Every terminal path MUST release request state, temporary tensors,
|
||||
hooks, and runtime-owned resources.
|
||||
|
||||
### DIFF-RUNTIME-INV-004: Optional features use runtime hooks
|
||||
|
||||
**Rule:** Cache, profiling, offload, and parallel features SHOULD integrate at
|
||||
defined hooks instead of creating another request lifecycle.
|
||||
|
||||
## Safe-change guide
|
||||
|
||||
Test success, cancellation, failure, shutdown, multiple requests, and resource
|
||||
cleanup.
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
title: Diffusion Modules
|
||||
kind: module-index
|
||||
status: draft
|
||||
owners:
|
||||
- "@Isotr0py"
|
||||
- "@princepride"
|
||||
- "@SamitHuang"
|
||||
- "@wtomin"
|
||||
- "@ZJY0516"
|
||||
- "@RuixiangMa"
|
||||
- "@david6666666"
|
||||
- "@xuechendi"
|
||||
primary_code_paths:
|
||||
- vllm_omni/diffusion/**
|
||||
related_code_paths:
|
||||
- vllm_omni/platforms/**
|
||||
depends_on:
|
||||
- ../engine_orchestration.md
|
||||
- ../input_output_modality_contracts.md
|
||||
validation_paths:
|
||||
- tests/diffusion/**
|
||||
upstream_refs:
|
||||
- diffusers
|
||||
last_reviewed: 2026-07-16
|
||||
---
|
||||
|
||||
# Diffusion modules
|
||||
|
||||
Diffusion is a module family containing runtime, model integration, batching,
|
||||
distributed execution, and memory-management subsystems.
|
||||
|
||||
## Module documents
|
||||
|
||||
- [Diffusion runtime](diffusion_runtime.md)
|
||||
- [Diffusion model integration](diffusion_model_integration.md)
|
||||
- [Continuous batching](continuous_batching.md)
|
||||
- [Parallelism](parallelism.md)
|
||||
- [Offloader](offloader.md)
|
||||
|
||||
Cache, quantization, profiling, and benchmarking follow their top-level module
|
||||
documents.
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
title: Diffusion Offloader
|
||||
kind: module
|
||||
status: draft
|
||||
owners:
|
||||
- "@yuanheng-zhao"
|
||||
- "@Isotr0py"
|
||||
primary_code_paths:
|
||||
- vllm_omni/diffusion/offloader/**
|
||||
related_code_paths:
|
||||
- vllm_omni/diffusion/models/**
|
||||
- vllm_omni/diffusion/worker/**
|
||||
depends_on:
|
||||
- diffusion_runtime.md
|
||||
- ../execution_platforms.md
|
||||
validation_paths:
|
||||
- tests/diffusion/offloader/**
|
||||
upstream_refs:
|
||||
- torch.nn.Module.to
|
||||
last_reviewed: 2026-07-16
|
||||
---
|
||||
|
||||
# Diffusion offloader
|
||||
|
||||
The offloader owns component residency, transfer scheduling, memory accounting,
|
||||
prefetch, and teardown for diffusion execution.
|
||||
|
||||
## Candidate invariants
|
||||
|
||||
### OFFLOAD-INV-001: Residency is explicit
|
||||
|
||||
**Rule:** Each managed component MUST have one known residency state and one
|
||||
owner responsible for transitions.
|
||||
|
||||
### OFFLOAD-INV-002: Use follows readiness
|
||||
|
||||
**Rule:** Runtime execution MUST NOT consume a component until its transfer to
|
||||
the required device is complete.
|
||||
|
||||
### OFFLOAD-INV-003: Transfers preserve model state
|
||||
|
||||
**Rule:** Offloading MUST preserve parameter and buffer identity, dtype, device,
|
||||
and correctness required by the selected execution mode.
|
||||
|
||||
### OFFLOAD-INV-004: Memory is bounded and released
|
||||
|
||||
**Rule:** Retained host and device copies MUST respect configured limits and
|
||||
provide deterministic teardown.
|
||||
|
||||
## Safe-change guide
|
||||
|
||||
Test enabled and disabled paths, repeated execution, asynchronous transfer,
|
||||
memory limits, failure, teardown, and numerical equivalence.
|
||||
@@ -0,0 +1,56 @@
|
||||
---
|
||||
title: Diffusion Parallelism
|
||||
kind: module
|
||||
status: draft
|
||||
owners:
|
||||
- "@princepride"
|
||||
- "@yuanheng-zhao"
|
||||
- "@xuechendi"
|
||||
primary_code_paths:
|
||||
- vllm_omni/diffusion/distributed/**
|
||||
- vllm_omni/diffusion/attention/parallel/**
|
||||
related_code_paths:
|
||||
- vllm_omni/distributed/**
|
||||
- vllm_omni/config/composable_parallel/**
|
||||
depends_on:
|
||||
- diffusion_runtime.md
|
||||
- ../vllm_omni_config.md
|
||||
validation_paths:
|
||||
- tests/diffusion/distributed/**
|
||||
- tests/diffusion/attention/**
|
||||
upstream_refs:
|
||||
- torch.distributed
|
||||
last_reviewed: 2026-07-16
|
||||
---
|
||||
|
||||
# Diffusion parallelism
|
||||
|
||||
Parallelism owns diffusion rank topology, process groups, tensor and sequence
|
||||
sharding, collectives, and distributed execution.
|
||||
|
||||
## Candidate invariants
|
||||
|
||||
### PARALLEL-INV-001: Topology has one source of truth
|
||||
|
||||
**Rule:** Rank coordinates, group membership, and parallel dimensions MUST be
|
||||
derived from validated configuration.
|
||||
|
||||
### PARALLEL-INV-002: Sharding contracts are explicit
|
||||
|
||||
**Rule:** Every distributed boundary MUST define tensor shape, sharded
|
||||
dimension, placement, dtype, producer, and consumer.
|
||||
|
||||
### PARALLEL-INV-003: Collectives are symmetric
|
||||
|
||||
**Rule:** Process-group members MUST invoke compatible collectives in the same
|
||||
logical order.
|
||||
|
||||
### PARALLEL-INV-004: Single-rank behavior remains valid
|
||||
|
||||
**Rule:** Distributed implementations SHOULD preserve a supported single-rank
|
||||
path without distributed initialization.
|
||||
|
||||
## Safe-change guide
|
||||
|
||||
Test topology validation, single-rank execution, affected parallel modes,
|
||||
combined modes, invalid shapes, and orderly teardown.
|
||||
Reference in new issue
Block a user