Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ac67e8874d | ||
|
|
dfe6e13cfc | ||
|
|
9d80efcbe5 | ||
|
|
ddbbfd29e5 |
No files matched your search
+41
-22
@@ -101,30 +101,49 @@ nav:
|
||||
- CI: contributing/ci
|
||||
- Design Documents:
|
||||
- design/index.md
|
||||
- design/architecture_overview.md
|
||||
- Architecture Documents:
|
||||
- Architecture Overview: design/architecture_overview.md
|
||||
- Feature Design:
|
||||
- design/feature/disaggregated_inference.md
|
||||
- design/feature/ray_based_execution.md
|
||||
- design/feature/omni_connectors/
|
||||
- design/feature/prefix_caching.md
|
||||
- design/feature/cfg_parallel.md
|
||||
- design/feature/expert_parallel.md
|
||||
- design/feature/sequence_parallel.md
|
||||
- design/feature/tensor_parallel.md
|
||||
- design/feature/vae_parallel.md
|
||||
- design/feature/hsdp.md
|
||||
- design/feature/cache_dit.md
|
||||
- design/feature/teacache.md
|
||||
- design/feature/skip_softmax.md
|
||||
- design/feature/async_chunk.md
|
||||
- design/feature/omni_async_output_materialization.md
|
||||
- design/feature/vae_parallel.md
|
||||
- design/feature/diffusion_continuous_batching.md
|
||||
- Runtime and Stage Execution:
|
||||
- Disaggregated Inference: design/feature/disaggregated_inference.md
|
||||
- Ray-based Execution: design/feature/ray_based_execution.md
|
||||
- Async Chunk: design/feature/async_chunk.md
|
||||
- Async Diffusion Output: design/feature/async_diffusion_output.md
|
||||
- Async Omni Output Materialization: design/feature/omni_async_output_materialization.md
|
||||
- Automatic Prefix Caching: design/feature/prefix_caching.md
|
||||
- Communication:
|
||||
- OmniConnector Implementations:
|
||||
- Mooncake Store Connector: design/feature/omni_connectors/mooncake_store_connector.md
|
||||
- Mooncake Transfer Engine Connector: design/feature/omni_connectors/mooncake_transfer_engine_connector.md
|
||||
- Mori Transfer Engine Connector: design/feature/omni_connectors/mori_transfer_engine_connector.md
|
||||
- Shared Memory Connector: design/feature/omni_connectors/shared_memory_connector.md
|
||||
- Yuanrong Store Connector: design/feature/omni_connectors/yuanrong_connector.md
|
||||
- Yuanrong Transfer Engine Connector: design/feature/omni_connectors/yuanrong_transfer_engine_connector.md
|
||||
- Diffusion Acceleration:
|
||||
- Parallelism:
|
||||
- CFG-Parallel: design/feature/cfg_parallel.md
|
||||
- Expert Parallel: design/feature/expert_parallel.md
|
||||
- Hybrid Sharded Data Parallel (HSDP): design/feature/hsdp.md
|
||||
- Pipeline Parallel: design/feature/pipeline_parallel.md
|
||||
- Sequence Parallel: design/feature/sequence_parallel.md
|
||||
- Tensor Parallel: design/feature/tensor_parallel.md
|
||||
- VAE Patch Parallelism: design/feature/vae_parallel.md
|
||||
- Attention Backends:
|
||||
- Skip-Softmax: design/feature/skip_softmax.md
|
||||
- Quantization: user_guide/quantization/overview.md
|
||||
- Cache-DiT: design/feature/cache_dit.md
|
||||
- TeaCache: design/feature/teacache.md
|
||||
- Diffusion Continuous Batching: design/feature/diffusion_continuous_batching.md
|
||||
- Infrastructure and Performance:
|
||||
- Prometheus Metrics: design/metrics.md
|
||||
- Speech Generation Performance Optimizations: design/qwen3_omni_tts_performance_optimization.md
|
||||
- Module Design:
|
||||
- design/module/ar_module.md
|
||||
- design/module/dit_module.md
|
||||
- design/module/entrypoint_module.md
|
||||
- design/module/async_omni_architecture.md
|
||||
- Runtime Modules:
|
||||
- AR Module: design/module/ar_module.md
|
||||
- DIT Module: design/module/dit_module.md
|
||||
- Orchestration:
|
||||
- Entrypoint Module: design/module/entrypoint_module.md
|
||||
- AsyncOmni Architecture: design/module/async_omni_architecture.md
|
||||
- Docs Guide: contributing/DOCS_GUIDE.md
|
||||
- API Reference:
|
||||
- api/README.md
|
||||
|
||||
+50
-2
@@ -1,6 +1,8 @@
|
||||
# Design Documents
|
||||
|
||||
This section contains design documents and architecture specifications for vLLM-Omni.
|
||||
This section contains design documents and architecture specifications for
|
||||
vLLM-Omni. The sidebar groups documents by the system concern they describe,
|
||||
while the source paths remain stable so existing links continue to work.
|
||||
|
||||
## Architecture Documents
|
||||
|
||||
@@ -8,17 +10,63 @@ This section contains design documents and architecture specifications for vLLM-
|
||||
|
||||
## Feature Design Documents
|
||||
|
||||
### Runtime and stage execution
|
||||
|
||||
- [Disaggregated Inference](feature/disaggregated_inference.md)
|
||||
- [Ray-based Execution](feature/ray_based_execution.md)
|
||||
- [Async Chunk](feature/async_chunk.md)
|
||||
- [Async Diffusion Output](feature/async_diffusion_output.md)
|
||||
- [Async Omni Output Materialization](feature/omni_async_output_materialization.md)
|
||||
- [Automatic Prefix Caching in Omni Models](feature/prefix_caching.md)
|
||||
|
||||
### Communication
|
||||
|
||||
#### OmniConnector implementations
|
||||
|
||||
- [Mooncake Store Connector](feature/omni_connectors/mooncake_store_connector.md)
|
||||
- [Mooncake Transfer Engine Connector](feature/omni_connectors/mooncake_transfer_engine_connector.md)
|
||||
- [Mori Transfer Engine Connector](feature/omni_connectors/mori_transfer_engine_connector.md)
|
||||
- [Shared Memory Connector](feature/omni_connectors/shared_memory_connector.md)
|
||||
- [Yuanrong Store Connector](feature/omni_connectors/yuanrong_connector.md)
|
||||
- [Yuanrong Transfer Engine Connector](feature/omni_connectors/yuanrong_transfer_engine_connector.md)
|
||||
|
||||
### Diffusion acceleration
|
||||
|
||||
#### Parallelism
|
||||
|
||||
- [CFG-Parallel](feature/cfg_parallel.md)
|
||||
- [Expert Parallel](feature/expert_parallel.md)
|
||||
- [Hybrid Sharded Data Parallel (HSDP)](feature/hsdp.md)
|
||||
- [Pipeline Parallel](feature/pipeline_parallel.md)
|
||||
- [Sequence Parallel](feature/sequence_parallel.md)
|
||||
- [Tensor Parallel](feature/tensor_parallel.md)
|
||||
- [VAE Patch Parallelism](feature/vae_parallel.md)
|
||||
|
||||
#### Attention Backends
|
||||
|
||||
- [Skip-Softmax](feature/skip_softmax.md)
|
||||
|
||||
#### Quantization
|
||||
|
||||
- [Quantization Overview](../user_guide/quantization/overview.md)
|
||||
|
||||
- [Cache-DiT](feature/cache_dit.md)
|
||||
- [TeaCache](feature/teacache.md)
|
||||
- [Diffusion Continuous Batching](feature/diffusion_continuous_batching.md)
|
||||
|
||||
## Infrastructure Design Documents
|
||||
## Infrastructure and Performance
|
||||
|
||||
- [Prometheus Metrics](metrics.md)
|
||||
- [Speech Generation Performance Optimizations](qwen3_omni_tts_performance_optimization.md)
|
||||
|
||||
## Module Design Documents
|
||||
|
||||
### Runtime Modules
|
||||
|
||||
- [AR Module](module/ar_module.md)
|
||||
- [DIT Module](module/dit_module.md)
|
||||
|
||||
### Orchestration
|
||||
|
||||
- [Entrypoint Module](module/entrypoint_module.md)
|
||||
- [AsyncOmni Architecture (Qwen3-Omni Example)](module/async_omni_architecture.md)
|
||||
Reference in new issue
Block a user