vllm-omni/docs/design/index.md

Design Documents

This section contains design documents and architecture specifications for vLLM-Omni. The sidebar groups documents by the system concern they describe; retired module pages are preserved in the legacy archive instead of remaining in the active navigation.

Architecture Documents

• Architecture Overview

Feature Design Documents

Runtime and stage execution

• Disaggregated Inference
• Async Chunk
• Async Diffusion Output
• Async Omni Output Materialization
• Automatic Prefix Caching in Omni Models

Communication

OmniConnector implementations

• Mooncake Store Connector
• Mooncake Transfer Engine Connector
• Mori Transfer Engine Connector
• Shared Memory Connector
• Yuanrong Store Connector
• Yuanrong Transfer Engine Connector

Diffusion acceleration

Parallelism

• CFG-Parallel
• Expert Parallel
• Hybrid Sharded Data Parallel (HSDP)
• Pipeline Parallel
• Sequence Parallel
• Tensor Parallel
• VAE Patch Parallelism

Attention Backends

• Skip-Softmax

Quantization

• Quantization

• Cache-DiT

• TeaCache

• Diffusion Continuous Batching

• Distributed Layerwise Offload

Infrastructure and Performance

• Prometheus Metrics
• Speech Generation Performance Optimizations

Module Design Documents

• Entrypoints and Serving Boundaries
• vLLM-Omni Configuration
• Input, Output, and Modality Contracts
• Error Classification, Propagation, and Rendering
• Engine Orchestration
• Stage Runtime and Replica Lifecycle
• OmniConnector
• Model Integration
• Autoregressive Runtime
• Diffusion
  • Overview
  • Runtime
  • Model Integration
  • Continuous Batching
  • Parallelism
  • Offloader
• Execution Platforms
• Cache Management
• Quantization
• Observability
• Profiling
• Benchmarking

The pre-#5137 pages are preserved in the legacy module archive for historical reference and are not active design contracts.