vllm-omni/docs/design/index.md

Design Documents

This section contains design documents and architecture specifications for vLLM-Omni. The sidebar groups documents by the system concern they describe, while the source paths remain stable so existing links continue to work.

Architecture Documents

• Architecture Overview

Feature Design Documents

Runtime and stage execution

• Disaggregated Inference
• Async Chunk
• Async Diffusion Output
• Async Omni Output Materialization
• Automatic Prefix Caching in Omni Models

Communication

OmniConnector implementations

• Mooncake Store Connector
• Mooncake Transfer Engine Connector
• Mori Transfer Engine Connector
• Shared Memory Connector
• Yuanrong Store Connector
• Yuanrong Transfer Engine Connector

Diffusion acceleration

Parallelism

• CFG-Parallel
• Expert Parallel
• Hybrid Sharded Data Parallel (HSDP)
• Pipeline Parallel
• Sequence Parallel
• Tensor Parallel
• VAE Patch Parallelism

Attention Backends

• Skip-Softmax

Quantization

• Quantization

• Cache-DiT

• TeaCache

• Diffusion Continuous Batching

• Distributed Layerwise Offload

Infrastructure and Performance

• Prometheus Metrics
• Speech Generation Performance Optimizations

Module Design Documents

Runtime Modules

• AR Module
• DIT Module

Orchestration

• Entrypoint Module
• AsyncOmni Architecture (Qwen3-Omni Example)