3.9 KiB
Design Documents
This section contains design documents and architecture specifications for vLLM-Omni. The sidebar groups documents by the system concern they describe; retired module pages are preserved in the legacy archive instead of remaining in the active navigation.
Architecture Documents
Feature Design Documents
For user-facing configuration and current compatibility, see the Features overview. A design document defines an implementation contract; it is not, by itself, a general support claim.
Runtime and stage execution
- Disaggregated Inference
- Async Chunk
- Async Diffusion Output
- Async Omni Output Materialization
- Automatic Prefix Caching in Omni Models
Communication
OmniConnector implementations
- Mooncake Store Connector
- Mooncake Transfer Engine Connector
- Mori Transfer Engine Connector
- Shared Memory Connector
- Yuanrong Store Connector
- Yuanrong Transfer Engine Connector
Quantization
Diffusion acceleration
Parallelism
- CFG-Parallel
- Expert Parallel
- Hybrid Sharded Data Parallel (HSDP)
- Pipeline Parallel
- Sequence Parallel
- Tensor Parallel
- VAE Parallel
Attention optimization
The Diffusion Attention Backends guide lists all selectable backends and their platform defaults. Skip-Softmax is the backend-related optimization with a standalone feature-design contract:
Infrastructure and Performance
Module Design Documents
- Entrypoints and Serving Boundaries
- vLLM-Omni Configuration
- Input, Output, and Modality Contracts
- Error Classification, Propagation, and Rendering
- Engine Orchestration
- Stage Runtime and Replica Lifecycle
- OmniConnector
- Model Integration
- Autoregressive Runtime
- Diffusion
- Execution Platforms
- Cache Management
- Quantization
- Observability
- Profiling
- Benchmarking
The pre-#5137 pages are preserved in the legacy module archive for historical reference and are not active design contracts.