diff --git a/docs/.nav.yml b/docs/.nav.yml index d6789a6cf..56733c434 100644 --- a/docs/.nav.yml +++ b/docs/.nav.yml @@ -44,23 +44,23 @@ nav: - Runtime and Stage Execution: - Execution Modes and Streaming: user_guide/diffusion/execution_modes.md - Sleep Mode: features/sleep_mode.md + - Quantization: + - Overview: user_guide/quantization/overview.md + - Online Quantization: user_guide/quantization/online.md + - Quantized KV Cache: user_guide/quantization/quantized_kvcache.md + - FP8 W8A8: user_guide/quantization/fp8.md + - Int8 W8A8: user_guide/quantization/int8.md + - BitsAndBytes W4: user_guide/quantization/bitsandbytes.md + - MXFP8 W8A8: user_guide/quantization/mxfp8.md + - MXFP4 W4A4: user_guide/quantization/mxfp4.md + - ModelOpt: user_guide/quantization/modelopt.md + - GGUF: user_guide/quantization/gguf.md + - AutoRound: user_guide/quantization/autoround.md + - msModelSlim: user_guide/quantization/msmodelslim.md - Diffusion Acceleration: - Overview and Compatibility: - Overview: user_guide/diffusion_features.md - Feature Compatibility: user_guide/feature_compatibility.md - - Quantization: - - Overview: user_guide/quantization/overview.md - - Online Quantization: user_guide/quantization/online.md - - Quantized KV Cache: user_guide/quantization/quantized_kvcache.md - - FP8 W8A8: user_guide/quantization/fp8.md - - Int8 W8A8: user_guide/quantization/int8.md - - BitsAndBytes W4: user_guide/quantization/bitsandbytes.md - - MXFP8 W8A8: user_guide/quantization/mxfp8.md - - MXFP4 W4A4: user_guide/quantization/mxfp4.md - - ModelOpt: user_guide/quantization/modelopt.md - - GGUF: user_guide/quantization/gguf.md - - AutoRound: user_guide/quantization/autoround.md - - msModelSlim: user_guide/quantization/msmodelslim.md - CPU Offloading: user_guide/diffusion/cpu_offload.md - Cache Acceleration: - TeaCache: user_guide/diffusion/cache_acceleration/teacache.md @@ -115,6 +115,7 @@ nav: - Shared Memory Connector: design/feature/omni_connectors/shared_memory_connector.md - Yuanrong Store Connector: design/feature/omni_connectors/yuanrong_connector.md - Yuanrong Transfer Engine Connector: design/feature/omni_connectors/yuanrong_transfer_engine_connector.md + - Quantization: design/feature/quantization.md - Diffusion Acceleration: - Parallelism: - CFG-Parallel: design/feature/cfg_parallel.md @@ -126,7 +127,6 @@ nav: - VAE Patch Parallelism: design/feature/vae_parallel.md - Attention Backends: - Skip-Softmax: design/feature/skip_softmax.md - - Quantization: design/feature/quantization.md - Cache-DiT: design/feature/cache_dit.md - TeaCache: design/feature/teacache.md - Diffusion Continuous Batching: design/feature/diffusion_continuous_batching.md diff --git a/docs/design/index.md b/docs/design/index.md index 14dd3edb5..e00c31c3d 100644 --- a/docs/design/index.md +++ b/docs/design/index.md @@ -34,6 +34,10 @@ implementation contract; it is not, by itself, a general support claim. - [Yuanrong Store Connector](feature/omni_connectors/yuanrong_connector.md) - [Yuanrong Transfer Engine Connector](feature/omni_connectors/yuanrong_transfer_engine_connector.md) +### Quantization + +- [Quantization](feature/quantization.md) + ### Diffusion acceleration #### Parallelism @@ -50,10 +54,6 @@ implementation contract; it is not, by itself, a general support claim. - [Skip-Softmax](feature/skip_softmax.md) -#### Quantization - -- [Quantization](feature/quantization.md) - - [Cache-DiT](feature/cache_dit.md) - [TeaCache](feature/teacache.md) - [Diffusion Continuous Batching](feature/diffusion_continuous_batching.md) diff --git a/docs/features/README.md b/docs/features/README.md index 22e9b9582..0dee558a4 100644 --- a/docs/features/README.md +++ b/docs/features/README.md @@ -35,12 +35,19 @@ features yet: design-level contract until its user-facing configuration and compatibility surface is consolidated. +## Quantization + +Quantization is a cross-model feature rather than a diffusion-only +optimization. The unified [`quantization_config` guide](../user_guide/quantization/overview.md) +covers diffusion-only models, multi-stage omni/TTS models, and multi-stage +diffusion models. Its [design contract](../design/feature/quantization.md) +defines the shared configuration and backend extension points. + ## Diffusion Acceleration | Goal | User guide | Related design contract | | --- | --- | --- | | Compare acceleration methods and supported combinations | [Overview](../user_guide/diffusion_features.md), [Feature Compatibility](../user_guide/feature_compatibility.md) | [Diffusion acceleration designs](../design/index.md#diffusion-acceleration) | -| Reduce model memory with lower-precision weights | [Quantization](../user_guide/quantization/overview.md) | [Quantization](../design/feature/quantization.md) | | Move weights between host and device memory | [CPU Offloading](../user_guide/diffusion/cpu_offload.md) | [Distributed Layerwise Offload](../design/feature/distributed_layerwise_offload.md) | | Reuse denoising computation | [Cache Acceleration](../user_guide/diffusion/cache_acceleration/cache_dit.md) | [Cache-DiT](../design/feature/cache_dit.md), [TeaCache](../design/feature/teacache.md) | | Distribute diffusion work across devices | [Parallelism](../user_guide/diffusion/parallelism/overview.md) | [Parallelism designs](../design/index.md#parallelism) | diff --git a/docs/user_guide/diffusion_features.md b/docs/user_guide/diffusion_features.md index c810d582f..7c664e88b 100644 --- a/docs/user_guide/diffusion_features.md +++ b/docs/user_guide/diffusion_features.md @@ -227,34 +227,23 @@ links to this section continue to work. ## Learn More -**Cache Acceleration:** +The Diffusion Acceleration navigation groups the remaining guides as follows: -- **[TeaCache Configuration Guide](diffusion/cache_acceleration/teacache.md)** - Parameter tuning, performance tips, troubleshooting -- **[Cache-DiT Advanced Guide](diffusion/cache_acceleration/cache_dit.md)** - DBCache, TaylorSeer, SCM techniques and optimization +| Area | Guide | +| --- | --- | +| Compatibility | [Feature Compatibility](feature_compatibility.md) | +| CPU offloading | [CPU Offloading](diffusion/cpu_offload.md) | +| Cache acceleration | [TeaCache](diffusion/cache_acceleration/teacache.md), [Cache-DiT](diffusion/cache_acceleration/cache_dit.md) | +| Parallelism | [Parallelism Overview](diffusion/parallelism/overview.md) | +| Attention | [Attention Backends](diffusion/attention_backends.md) | +| Compilation | [Regional Compilation](diffusion/regional_compilation.md) | +| Video extension | [Frame Interpolation](diffusion/frame_interpolation.md) | +| Startup | [Startup and Loading](diffusion/startup_and_loading.md) | +| Adapters | [LoRA](diffusion/lora.md) | -**Parallelism Methods:** +Related cross-model and runtime features are documented separately: -- **[Parallelism Overview](diffusion/parallelism/overview.md)** - Tensor Parallelism, Sequence Parallelism, CFG Parallelism, Pipeline Parallelism, HSDP, and Expert Parallelism - -**Memory Optimization:** - -- **[CPU Offload Guide](diffusion/cpu_offload.md)** - Offload model components to CPU, reduce GPU memory usage -- **[VAE Parallelism Guide](diffusion/parallelism/vae_parallelism.md)** - Distribute VAE decode work across GPUs for high-resolution images and videos -- **[Quantization](quantization/overview.md)** - Quantization methods for diffusion, multi-stage omni/TTS, and multi-stage diffusion models - -**Extensions:** - -- **[LoRA Inference Guide](diffusion/lora.md)** - Low-Rank Adaptation for style customization and fine-tuning -- **[Frame Interpolation Guide](diffusion/frame_interpolation.md)** - Worker-side post-generation video frame interpolation for smoother motion - -**Execution Modes:** - -- **[Diffusion Execution Modes](diffusion/execution_modes.md)** - Configure request batching, step execution, continuous batching, and streaming output - -**Startup Optimization:** - -- **[Multi-Thread Weight Loading](#multi-thread-weight-loading)** - Speed up model startup by loading safetensors shards in parallel - -**Advanced Topics:** - -- **[Feature Compatibility](feature_compatibility.md)** - How to combine multiple features for maximum performance +- [Quantization](quantization/overview.md) covers diffusion-only models, + multi-stage omni/TTS models, and multi-stage diffusion models. +- [Execution Modes and Streaming](diffusion/execution_modes.md) covers the + diffusion runtime, including batching, step execution, and streaming output.