SYLARandHongsheng Liu
0e44fc7406
Signed-off-by: lishunyang12 <lishunyang12@163.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
190 lines
9.8 KiB
YAML
190 lines
9.8 KiB
YAML
nav:
|
|
- Home: README.md
|
|
- User Guide:
|
|
- Getting Started:
|
|
- getting_started/quickstart.md
|
|
- getting_started/installation/*
|
|
- Serving:
|
|
- OpenAI-Compatible API:
|
|
- Chat Completions API: serving/chat_completions_api.md
|
|
- Image Generation: serving/image_generation_api.md
|
|
- Image Edit: serving/image_edit_api.md
|
|
- Text to Speech: serving/speech_api.md
|
|
- Videos: serving/videos_api.md
|
|
- Audio Generation: serving/audio_generate_api.md
|
|
- Streaming Video Input: serving/video_stream_api.md
|
|
- Examples:
|
|
- examples/README.md
|
|
- Offline Inference:
|
|
- Image-To-Image: user_guide/examples/offline_inference/image_to_image.md
|
|
- Image-To-Video: user_guide/examples/offline_inference/image_to_video.md
|
|
- Speech-To-Video: user_guide/examples/offline_inference/speech_to_video.md
|
|
- Text-To-Audio: user_guide/examples/offline_inference/text_to_audio.md
|
|
- Text-To-Image: user_guide/examples/offline_inference/text_to_image.md
|
|
- Text-To-Speech: user_guide/examples/offline_inference/text_to_speech.md
|
|
- Text-To-Video: user_guide/examples/offline_inference/text_to_video.md
|
|
- X-To-Text: user_guide/examples/offline_inference/x_to_text.md
|
|
- Online Serving:
|
|
- Image-To-Image: user_guide/examples/online_serving/image_to_image.md
|
|
- Image-To-Video: user_guide/examples/online_serving/image_to_video.md
|
|
- Speech-To-Video: user_guide/examples/online_serving/speech_to_video.md
|
|
- Text-To-Image: user_guide/examples/online_serving/text_to_image.md
|
|
- Text-To-Speech: user_guide/examples/online_serving/text_to_speech.md
|
|
- Text-To-Video: user_guide/examples/online_serving/text_to_video.md
|
|
- General:
|
|
- usage/*
|
|
- Configuration:
|
|
- configuration/README.md
|
|
- configuration/*
|
|
- Models:
|
|
- models/supported_models.md
|
|
- Features:
|
|
- Overview: features/README.md
|
|
- Runtime and Stage Execution:
|
|
- Execution Modes and Streaming: user_guide/diffusion/execution_modes.md
|
|
- Prefill-Decode Disaggregation (experimental): features/pd_disaggregation.md
|
|
- Sleep Mode: features/sleep_mode.md
|
|
- Quantization:
|
|
- Overview: user_guide/quantization/overview.md
|
|
- Online Quantization: user_guide/quantization/online.md
|
|
- Quantized KV Cache: user_guide/quantization/quantized_kvcache.md
|
|
- FP8 W8A8: user_guide/quantization/fp8.md
|
|
- Int8 W8A8: user_guide/quantization/int8.md
|
|
- BitsAndBytes W4: user_guide/quantization/bitsandbytes.md
|
|
- MXFP8 W8A8: user_guide/quantization/mxfp8.md
|
|
- MXFP4 W4A4: user_guide/quantization/mxfp4.md
|
|
- SVDQuant W4A4: user_guide/quantization/svdquant.md
|
|
- ModelOpt: user_guide/quantization/modelopt.md
|
|
- AutoRound: user_guide/quantization/autoround.md
|
|
- msModelSlim: user_guide/quantization/msmodelslim.md
|
|
- Diffusion Acceleration:
|
|
- Overview and Compatibility:
|
|
- Overview: user_guide/diffusion_features.md
|
|
- Feature Compatibility: user_guide/feature_compatibility.md
|
|
- CPU Offloading:
|
|
- Overview: user_guide/diffusion/cpu_offload.md
|
|
- Model-Level: user_guide/diffusion/offloader/cpu_offload.md
|
|
- Layerwise: user_guide/diffusion/offloader/layerwise_offload.md
|
|
- Distributed Layerwise: user_guide/diffusion/offloader/distributed_layerwise_offload.md
|
|
- Cache Acceleration:
|
|
- TeaCache: user_guide/diffusion/cache_acceleration/teacache.md
|
|
- Cache-DiT: user_guide/diffusion/cache_acceleration/cache_dit.md
|
|
- Parallelism:
|
|
- Overview: user_guide/diffusion/parallelism/overview.md
|
|
- CFG Parallel: user_guide/diffusion/parallelism/cfg_parallel.md
|
|
- Expert Parallel: user_guide/diffusion/parallelism/expert_parallel.md
|
|
- Hybrid Sharded Data Parallel: user_guide/diffusion/parallelism/hsdp.md
|
|
- Sequence Parallel: user_guide/diffusion/parallelism/sequence_parallel.md
|
|
- Tensor Parallel: user_guide/diffusion/parallelism/tensor_parallel.md
|
|
- VAE Parallelism: user_guide/diffusion/parallelism/vae_parallelism.md
|
|
- Attention Backends:
|
|
- Overview: user_guide/diffusion/attention_backends.md
|
|
- Dense Backends: user_guide/diffusion/attention_backends/dense_backends.md
|
|
- TRTLLM: user_guide/diffusion/attention_backends/trtllm.md
|
|
- SageAttention: user_guide/diffusion/attention_backends/sage.md
|
|
- Hugging Face Hub: user_guide/diffusion/attention_backends/huggingface_hub.md
|
|
- RainFusion: user_guide/diffusion/attention_backends/rainfusion.md
|
|
- Regional Compilation: user_guide/diffusion/regional_compilation.md
|
|
- Frame Interpolation: user_guide/diffusion/frame_interpolation.md
|
|
- Startup and Loading: user_guide/diffusion/startup_and_loading.md
|
|
- LoRA: user_guide/diffusion/lora.md
|
|
- Experimental:
|
|
- Session State Manager: features/session_state_manager.md
|
|
- Integrations:
|
|
- Custom Diffusion Pipeline: features/custom_pipeline.md
|
|
- ComfyUI: features/comfyui.md
|
|
- Fault Tolerance:
|
|
- Failure Modes: user_guide/fault_injection_reliability_matrix.md
|
|
- Developer Guide:
|
|
- General:
|
|
- contributing/README.md
|
|
- glob: contributing/*
|
|
flatten_single_child_sections: true
|
|
- Model Implementation:
|
|
- contributing/model/README.md
|
|
- contributing/model/adding_omni_model.md
|
|
- contributing/model/adding_tts_model.md
|
|
- contributing/model/adding_diffusion_model.md
|
|
- CI: contributing/ci
|
|
- Design Documents:
|
|
- design/index.md
|
|
- Architecture Documents:
|
|
- Architecture Overview: design/architecture_overview.md
|
|
- Feature Design:
|
|
- Runtime and Stage Execution:
|
|
- Disaggregated Inference: design/feature/disaggregated_inference.md
|
|
- Host Weight Runtime: design/feature/host_weight_runtime.md
|
|
- Async Chunk: design/feature/async_chunk.md
|
|
- Async Diffusion Output: design/feature/async_diffusion_output.md
|
|
- Async Omni Output Materialization: design/feature/omni_async_output_materialization.md
|
|
- Automatic Prefix Caching: design/feature/prefix_caching.md
|
|
- Realtime AR-Diffusion: design/feature/realtime_ar_diffusion.md
|
|
- Communication:
|
|
- OmniConnector Implementations:
|
|
- Mooncake Store Connector: design/feature/omni_connectors/mooncake_store_connector.md
|
|
- Mooncake Transfer Engine Connector: design/feature/omni_connectors/mooncake_transfer_engine_connector.md
|
|
- Mori Transfer Engine Connector: design/feature/omni_connectors/mori_transfer_engine_connector.md
|
|
- Shared Memory Connector: design/feature/omni_connectors/shared_memory_connector.md
|
|
- Yuanrong Store Connector: design/feature/omni_connectors/yuanrong_connector.md
|
|
- Yuanrong Transfer Engine Connector: design/feature/omni_connectors/yuanrong_transfer_engine_connector.md
|
|
- Quantization: design/feature/quantization.md
|
|
- Diffusion Acceleration:
|
|
- Parallelism:
|
|
- CFG-Parallel: design/feature/cfg_parallel.md
|
|
- Expert Parallel: design/feature/expert_parallel.md
|
|
- Hybrid Sharded Data Parallel (HSDP): design/feature/hsdp.md
|
|
- Pipeline Parallel: design/feature/pipeline_parallel.md
|
|
- Sequence Parallel: design/feature/sequence_parallel.md
|
|
- Tensor Parallel: design/feature/tensor_parallel.md
|
|
- VAE Patch Parallelism: design/feature/vae_parallel.md
|
|
- Attention Optimization:
|
|
- Backend Selection: design/feature/attention_backend_selection.md
|
|
- Skip-Softmax: design/feature/skip_softmax.md
|
|
- CPU Offloading:
|
|
- Overview: design/feature/offloader/README.md
|
|
- Model-Level: design/feature/offloader/cpu_offload.md
|
|
- Layerwise: design/feature/offloader/layerwise_offload.md
|
|
- Distributed Layerwise: design/feature/offloader/distributed_layerwise_offload.md
|
|
- Cache-DiT: design/feature/cache_dit.md
|
|
- TeaCache: design/feature/teacache.md
|
|
- Diffusion Continuous Batching: design/feature/diffusion_continuous_batching.md
|
|
- Infrastructure and Performance:
|
|
- Prometheus Metrics: design/metrics.md
|
|
- Speech Generation Performance Optimizations: design/qwen3_omni_tts_performance_optimization.md
|
|
- Module Design:
|
|
- Entrypoints: design/module/entrypoints.md
|
|
- vLLM-Omni Configuration: design/module/vllm_omni_config.md
|
|
- Input, Output, and Modality Contracts: design/module/input_output_modality_contracts.md
|
|
- Error Contracts: design/module/error_contracts.md
|
|
- Engine Orchestration: design/module/engine_orchestration.md
|
|
- Stage Runtime: design/module/stage_runtime.md
|
|
- OmniConnector: design/module/omni_connector.md
|
|
- Model Integration: design/module/model_integration.md
|
|
- Autoregressive Runtime: design/module/ar_runtime.md
|
|
- Diffusion:
|
|
- Overview: design/module/diffusion/index.md
|
|
- Runtime: design/module/diffusion/diffusion_runtime.md
|
|
- Model Integration: design/module/diffusion/diffusion_model_integration.md
|
|
- Continuous Batching: design/module/diffusion/continuous_batching.md
|
|
- Parallelism: design/module/diffusion/parallelism.md
|
|
- Offloader: design/module/diffusion/offloader.md
|
|
- Execution Platforms: design/module/execution_platforms.md
|
|
- Cache Management: design/module/cache_management.md
|
|
- Host Weight Runtime: design/module/host_weight_runtime.md
|
|
- Quantization: design/module/quantization.md
|
|
- Observability: design/module/observability.md
|
|
- Profiling: design/module/profiling.md
|
|
- Benchmarking: design/module/benchmarking.md
|
|
- Docs Guide: contributing/DOCS_GUIDE.md
|
|
- API Reference:
|
|
- api/README.md
|
|
- api/vllm_omni
|
|
- CLI Reference: cli
|
|
- Community:
|
|
- Governance: community/governance.md
|
|
- Contact Us: community/contact_us.md
|
|
- Community Meetings: community/meetings.md
|
|
- Slack: https://slack.vllm.ai
|
|
- Blog: https://blog.vllm.ai
|
|
- Forum: https://discuss.vllm.ai
|