vllm-omni/docs/design/index.md

Design Documents

This section contains design documents and architecture specifications for vLLM-Omni.

Architecture Documents

• Architecture Overview

Feature Design Documents

• Disaggregated Inference
• Ray-based Execution
• Adding Step Execution Support for Diffusion Pipelines
• Request-Level Batching for Diffusion
• Continuous Batching for Step-Wise Diffusion

Infrastructure Design Documents

• Prometheus Metrics

Module Design Documents

• Entrypoints
• vLLM-Omni Configuration
• Input, Output, and Modality Contracts
• Engine Orchestration
• OmniConnector
• Model Integration
• Autoregressive Runtime
• Diffusion
  • Overview
  • Runtime
  • Model Integration
  • Continuous Batching
  • Parallelism
  • Offloader
• Execution Platforms
• Cache Management
• Quantization
• Observability
• Profiling
• Benchmarking