diff --git a/docs/.nav.yml b/docs/.nav.yml index ac5ee2b28..3f99c1616 100644 --- a/docs/.nav.yml +++ b/docs/.nav.yml @@ -101,31 +101,50 @@ nav: - CI: contributing/ci - Design Documents: - design/index.md - - design/architecture_overview.md + - Architecture Documents: + - Architecture Overview: design/architecture_overview.md - Feature Design: - - design/feature/disaggregated_inference.md - - design/feature/ray_based_execution.md - - design/feature/omni_connectors/ - - design/feature/prefix_caching.md - - design/feature/cfg_parallel.md - - design/feature/expert_parallel.md - - design/feature/sequence_parallel.md - - design/feature/tensor_parallel.md - - design/feature/vae_parallel.md - - design/feature/hsdp.md - - design/feature/cache_dit.md - - design/feature/teacache.md - - design/feature/skip_softmax.md - - design/feature/async_chunk.md - - design/feature/omni_async_output_materialization.md - - design/feature/vae_parallel.md - - design/feature/diffusion_continuous_batching.md - - design/feature/distributed_layerwise_offload.md + - Runtime and Stage Execution: + - Disaggregated Inference: design/feature/disaggregated_inference.md + - Ray-based Execution: design/feature/ray_based_execution.md + - Async Chunk: design/feature/async_chunk.md + - Async Diffusion Output: design/feature/async_diffusion_output.md + - Async Omni Output Materialization: design/feature/omni_async_output_materialization.md + - Automatic Prefix Caching: design/feature/prefix_caching.md + - Communication: + - OmniConnector Implementations: + - Mooncake Store Connector: design/feature/omni_connectors/mooncake_store_connector.md + - Mooncake Transfer Engine Connector: design/feature/omni_connectors/mooncake_transfer_engine_connector.md + - Mori Transfer Engine Connector: design/feature/omni_connectors/mori_transfer_engine_connector.md + - Shared Memory Connector: design/feature/omni_connectors/shared_memory_connector.md + - Yuanrong Store Connector: design/feature/omni_connectors/yuanrong_connector.md + - Yuanrong Transfer Engine Connector: design/feature/omni_connectors/yuanrong_transfer_engine_connector.md + - Diffusion Acceleration: + - Parallelism: + - CFG-Parallel: design/feature/cfg_parallel.md + - Expert Parallel: design/feature/expert_parallel.md + - Hybrid Sharded Data Parallel (HSDP): design/feature/hsdp.md + - Pipeline Parallel: design/feature/pipeline_parallel.md + - Sequence Parallel: design/feature/sequence_parallel.md + - Tensor Parallel: design/feature/tensor_parallel.md + - VAE Patch Parallelism: design/feature/vae_parallel.md + - Attention Backends: + - Skip-Softmax: design/feature/skip_softmax.md + - Quantization: user_guide/quantization/overview.md + - Cache-DiT: design/feature/cache_dit.md + - TeaCache: design/feature/teacache.md + - Diffusion Continuous Batching: design/feature/diffusion_continuous_batching.md + - Distributed Layerwise Offload: design/feature/distributed_layerwise_offload.md + - Infrastructure and Performance: + - Prometheus Metrics: design/metrics.md + - Speech Generation Performance Optimizations: design/qwen3_omni_tts_performance_optimization.md - Module Design: - - design/module/ar_module.md - - design/module/dit_module.md - - design/module/entrypoint_module.md - - design/module/async_omni_architecture.md + - Runtime Modules: + - AR Module: design/module/ar_module.md + - DIT Module: design/module/dit_module.md + - Orchestration: + - Entrypoint Module: design/module/entrypoint_module.md + - AsyncOmni Architecture: design/module/async_omni_architecture.md - Docs Guide: contributing/DOCS_GUIDE.md - API Reference: - api/README.md diff --git a/docs/design/index.md b/docs/design/index.md index e8df11b77..57fd9ba4c 100644 --- a/docs/design/index.md +++ b/docs/design/index.md @@ -1,6 +1,8 @@ # Design Documents -This section contains design documents and architecture specifications for vLLM-Omni. +This section contains design documents and architecture specifications for +vLLM-Omni. The sidebar groups documents by the system concern they describe, +while the source paths remain stable so existing links continue to work. ## Architecture Documents @@ -8,18 +10,64 @@ This section contains design documents and architecture specifications for vLLM- ## Feature Design Documents +### Runtime and stage execution + - [Disaggregated Inference](feature/disaggregated_inference.md) - [Ray-based Execution](feature/ray_based_execution.md) +- [Async Chunk](feature/async_chunk.md) +- [Async Diffusion Output](feature/async_diffusion_output.md) - [Async Omni Output Materialization](feature/omni_async_output_materialization.md) +- [Automatic Prefix Caching in Omni Models](feature/prefix_caching.md) + +### Communication + +#### OmniConnector implementations + +- [Mooncake Store Connector](feature/omni_connectors/mooncake_store_connector.md) +- [Mooncake Transfer Engine Connector](feature/omni_connectors/mooncake_transfer_engine_connector.md) +- [Mori Transfer Engine Connector](feature/omni_connectors/mori_transfer_engine_connector.md) +- [Shared Memory Connector](feature/omni_connectors/shared_memory_connector.md) +- [Yuanrong Store Connector](feature/omni_connectors/yuanrong_connector.md) +- [Yuanrong Transfer Engine Connector](feature/omni_connectors/yuanrong_transfer_engine_connector.md) + +### Diffusion acceleration + +#### Parallelism + +- [CFG-Parallel](feature/cfg_parallel.md) +- [Expert Parallel](feature/expert_parallel.md) +- [Hybrid Sharded Data Parallel (HSDP)](feature/hsdp.md) +- [Pipeline Parallel](feature/pipeline_parallel.md) +- [Sequence Parallel](feature/sequence_parallel.md) +- [Tensor Parallel](feature/tensor_parallel.md) +- [VAE Patch Parallelism](feature/vae_parallel.md) + +#### Attention Backends + +- [Skip-Softmax](feature/skip_softmax.md) + +#### Quantization + +- [Quantization Overview](../user_guide/quantization/overview.md) + +- [Cache-DiT](feature/cache_dit.md) +- [TeaCache](feature/teacache.md) - [Diffusion Continuous Batching](feature/diffusion_continuous_batching.md) - [Distributed Layerwise Offload](feature/distributed_layerwise_offload.md) -## Infrastructure Design Documents +## Infrastructure and Performance - [Prometheus Metrics](metrics.md) +- [Speech Generation Performance Optimizations](qwen3_omni_tts_performance_optimization.md) ## Module Design Documents +### Runtime Modules + - [AR Module](module/ar_module.md) - [DIT Module](module/dit_module.md) + +### Orchestration + - [Entrypoint Module](module/entrypoint_module.md) +- [AsyncOmni Architecture (Qwen3-Omni Example)](module/async_omni_architecture.md)