[Doc] Misc documentation polishing (#98)

Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>

.github/ISSUE_TEMPLATE/config.yml

@@ -2,4 +2,4 @@ blank_issues_enabled: false
 contact_links:
   - name: Questions
     url: https://github.com/vllm-project/vllm-omni/discussions
-    about: Ask questions and discuss with other vLLM-omni community members
+    about: Ask questions and discuss with other vLLM-Omni community members

.gitignore

@@ -175,7 +175,7 @@ tests/
 #  *.iws
 #  .idea/
 
-# vLLM-omni specific
+# vLLM-Omni specific
 # Model files and checkpoints
 checkpoints/
 *.bin

README.md

@@ -22,7 +22,7 @@ Easy, fast, and cheap omni-modality model serving for everyone
 
 ## About
 
-[vLLM](https://github.com/vllm-project/vllm) was originally designed to support large language models for text-based autoregressive generation tasks. vLLM-Omni extends its support for omni-modality model inference and serving:
+[vLLM](https://github.com/vllm-project/vllm) was originally designed to support large language models for text-based autoregressive generation tasks. vLLM-Omni is a framework that extends its support for omni-modality model inference and serving:
 
 - **Omni-modality**: Text, image, video, and audio data processing
 - **Non-autoregressive Architectures**: extend the AR support of vLLM to Diffusion Transformers (DiT) and other parallel generation models
@@ -34,24 +34,24 @@ Easy, fast, and cheap omni-modality model serving for everyone
   </picture>
 </p>
 
-vLLM-omni is fast with:
+vLLM-Omni is fast with:
 
-- Seamless AR support for efficient KV Cache management with vLLM
-- Pipelined stage execution overlapping
-- Fully disaggregation based on omniConnector and dynamic resource allocation across stages
+- State-of-the-art AR support by leveraging efficient KV cache management from vLLM
+- Pipelined stage execution overlapping for high throughput performance
+- Fully disaggregation based on OmniConnector and dynamic resource allocation across stages
 
-vLLM-omni is flexible and easy to use with:
+vLLM-Omni is flexible and easy to use with:
 
+- Heterogeneous pipeline abstraction to manage complex model workflows
 - Seamless integration with popular Hugging Face models
-- High-throughput serving with various decoding algorithms, including *parallel sampling*, *beam search*, and more
 - Tensor, pipeline, data and expert parallelism support for distributed inference
 - Streaming outputs
 - OpenAI-compatible API server
 
-vLLM seamlessly supports most popular open-source models on HuggingFace, including:
+vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, including:
 
-- Multi-modality generation models (e.g. Qwen-image)
-- Omni-modality models (e.g. Qwen-omni)
+- Omni-modality models (e.g. Qwen-Omni)
+- Multi-modality generation models (e.g. Qwen-Image)
 
 ## Getting Started
 
@@ -66,7 +66,7 @@ Visit our documentation to learn more.
 ## Contributing
 
 We welcome and value any contributions and collaborations.
-Please check out [Contributing to vLLM-omni](https://vllm-omni.readthedocs.io/en/latest/contributing/) for how to get involved.
+Please check out [Contributing to vLLM-Omni](https://vllm-omni.readthedocs.io/en/latest/contributing/) for how to get involved.
 
 ## Weekly Meeting
 

docs/.nav.yml

@@ -8,8 +8,9 @@ nav:
       - examples/README.md
       - Offline Inference:
         - Qwen2.5-Omni: user_guide/examples/offline_inference/qwen2_5_omni.md
+      - Online Serving:
+        - Qwen2.5-Omni: user_guide/examples/online_serving/qwen2_5_omni.md
         - Qwen2.5-Image: user_guide/examples/offline_inference/qwen_image.md
-      - Online Serving: user_guide/examples/online_serving/README.md
     - General:
       - usage/*
     - Configuration:
@@ -17,16 +18,20 @@ nav:
       - configuration/*
     - Models:
       - models/supported_models.md
-    - Features: features
   - Developer Guide:
-    - contributing/README.md
     - General:
+      - contributing/README.md
       - glob: contributing/*
         flatten_single_child_sections: true
     - Model Implementation:
       - contributing/model/README.md
     - CI: contributing/ci
-    - Design Documents: design
+    - Design Documents:
+      - design/index.md
+      - design/architecture_overview.md
+      - design/vllm_omni_design.md
+      - design/mrs_design.md
+      - design/api_design_doc.md
     - Docs Guide: contributing/DOCS_GUIDE.md
   - API Reference:
     - api/README.md
@@ -34,6 +39,6 @@ nav:
   - CLI Reference: cli
   - Community:
     - community/*
+    - Slack: https://slack.vllm.ai
     - Blog: https://blog.vllm.ai
     - Forum: https://discuss.vllm.ai
-    - Slack: https://slack.vllm.ai

docs/README.md

@@ -26,7 +26,7 @@ Easy, fast, and cheap omni-modality model serving for everyone
 
 ## About
 
-[vLLM](https://github.com/vllm-project/vllm) was originally designed to support large language models for text-based autoregressive generation tasks. vLLM-Omni extends its support for omni-modality model inference and serving:
+[vLLM](https://github.com/vllm-project/vllm) was originally designed to support large language models for text-based autoregressive generation tasks. vLLM-Omni is a framework that extends its support for omni-modality model inference and serving:
 
 - **Omni-modality**: Text, image, video, and audio data processing
 - **Non-autoregressive Architectures**: extend the AR support of vLLM to Diffusion Transformers (DiT) and other parallel generation models
@@ -38,23 +38,24 @@ Easy, fast, and cheap omni-modality model serving for everyone
   </picture>
 </p>
 
-vLLM-omni is fast with:
+vLLM-Omni is fast with:
 
-- Seamless AR support for efficient KV Cache management with vLLM
-- Pipelined stage execution overlapping
-- Fully disaggregation based on omniConnector and dynamic resource allocation across stages
+- State-of-the-art AR support by leveraging efficient KV cache management from vLLM
+- Pipelined stage execution overlapping for high throughput performance
+- Fully disaggregation based on OmniConnector and dynamic resource allocation across stages
 
-vLLM-omni is flexible and easy to use with:
+vLLM-Omni is flexible and easy to use with:
 
+- Heterogeneous pipeline abstraction to manage complex model workflows
 - Seamless integration with popular Hugging Face models
-- High-throughput serving with various decoding algorithms, including *parallel sampling*, *beam search*, and more
 - Tensor, pipeline, data and expert parallelism support for distributed inference
 - Streaming outputs
 - OpenAI-compatible API server
 
 vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, including:
 
-- Multi-modality generation models (e.g. Qwen-image)
-- Omni-modality models (e.g. Qwen-omni)
+- Omni-modality models (e.g. Qwen2.5-Omni, Qwen3-Omni)
+- Multi-modality generation models (e.g. Qwen-Image)
 
 For more information, checkout the following:
+- [Placeholder for vllm-omni blogpost]()

docs/api/README.md

@@ -2,7 +2,7 @@
 
 ## Entry Points
 
-Main entry points for vLLM-omni inference and serving.
+Main entry points for vLLM-Omni inference and serving.
 
 - [vllm_omni.entrypoints.async_omni_llm.AsyncOmniLLM][]
 - [vllm_omni.entrypoints.async_omni_llm.AsyncOmniStageLLM][]

docs/community/contact_us.md

@@ -1,6 +1,5 @@
 # Contact Us
 
 - For technical questions and feature requests, please use GitHub [Issues](https://github.com/vllm-project/vllm-omni/issues)
-- For discussing with fellow users, please use the [vLLM Forum](https://discuss.vllm.ai/)
-- For coordinating contributions and development, please use [Slack](https://slack.vllm.ai/)
+- For coordinating contributions and development and discussing with other users and developers, please join `sig-omni` channel in our [Slack](https://slack.vllm.ai/) or use the [vLLM Forum](https://discuss.vllm.ai/)
 - For security disclosures, please use GitHub's [Security Advisories](https://github.com/vllm-project/vllm/security/advisories) feature

docs/configuration/stage_configs.md

@@ -1,10 +1,10 @@
 # Stage configs for vLLM-Omni
 
-In vLLM-Omni, it divides models in vLLM-omni into multiple stages, which are processed by different LLMEngines, DiffusionEngines or other kinds of engines. According to different types of stages, such as AR stage or DiT stage, each stage can choose corresponding schedulers, model workers to load into Engines with a plug-in way.
+In vLLM-Omni, the target model is separated into multiple stages, which are processed by different LLMEngines, DiffusionEngines or other types of engines. Depending on different types of stages, such as Autoregressive (AR) stage or Diffusion transformer (DiT) stage, each can choose corresponding schedulers, model workers to load with the Engines in a plug-in fashion.
 
-Therefore, the stage configs for a model have several main functions:
+Therefore, as a core part of vLLM-Omni, the stage configs for a model have several main functions:
 
-- Claim partition of stages and corresponding class implementation in the folder model_executor/models.
+- Claim partition of stages and their corresponding class implementation in `model_executor/models`.
 - The disaggregated configuration for each stage and the communication topology among them.
 - Engine arguments for each engine within the stage.
 - Input and output dependencies for each stage.
@@ -22,6 +22,8 @@ For online serving:
 ```bash
 vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091 --stage-configs-path /path/to/stage_configs_file
 ```
+!!! important
+    We are actively iterating on the definition of stage configs, and we welcome all feedbacks from both community users and developers to help us shape the development!
 
 Below is a specific example of stage_configs.yaml in Qwen2.5-omni.
 ```python

docs/contributing/DOCS_GUIDE.md

@@ -1,6 +1,6 @@
 # Documentation Build Guide
 
-This directory contains the source files for the vLLM-omni documentation.
+This directory contains the source files for the vLLM-Omni documentation.
 
 ## Building Documentation Locally
 
@@ -39,7 +39,7 @@ Example docstring:
 
 ```python
 class Omni:
-    """Main entry point for vLLM-omni inference.
+    """Main entry point for vLLM-Omni inference.
 
     This class provides a high-level interface for running multi-modal
     inference with non-autoregressive models.

docs/contributing/README.md

@@ -1,64 +1,42 @@
-# Contributing to vLLM-omni
+# Contributing to vLLM-Omni
 
-Thank you for your interest in contributing to vLLM-omni! This document provides guidelines and instructions for contributing.
+Thank you for your interest in contributing to vLLM-Omni! This document provides guidelines and instructions for contributing.
 
 ## Getting Started
 
-### Development Environment for vLLM
-
-vLLM-omni uses `uv` as the environment manager, to create and manage Python environments. Please follow the documentation to install `uv`. After installing `uv`, you can create a new Python environment using the following commands:
+vLLM-Omni uses `uv` as the environment manager, to create and manage Python environments. Please follow the documentation to install `uv`. After installing `uv`, you can create a new Python environment using the following commands:
 
 ```bash
 uv venv --python 3.12 --seed
 source .venv/bin/activate
 ```
+### Development Environment for vLLM
 
-Install vLLM from source:
-
-```bash
-git clone https://github.com/vllm-project/vllm.git
-cd vllm
-git checkout v0.11.0
-```
-Set up environment variables to get pre-built wheels. If there are internet problems, just download the whl file manually. And set VLLM_PRECOMPILED_WHEEL_LOCATION as your local absolute path of whl file.
-```bash
-export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.11.0/vllm-0.11.0-cp38-abi3-manylinux1_x86_64.whl
-```
-Install vllm with command below (If you have no existing PyTorch).
-```bash
-uv pip install --editable .
-```
-
-Install vllm with command below (If you already have PyTorch).
-```bash
-python use_existing_torch.py
-uv pip install -r requirements/build.txt
-uv pip install --no-build-isolation --editable .
-```
+vLLM-Omni is built on top of the stable release 0.11.0 of vLLM. You may install `vllm==0.11.0` directly or install the library from source if you need to check, modify or debug with source code of vLLM.
 
 ### Development Environment for vLLM-Omni
 
-Install vLLM-omni from source:
+Install vLLM-Omni from source with development dependencies:
 
 ```bash
 git clone https://github.com/vllm-project/vllm-omni.git
 cd vllm_omni
-uv pip install -e .
+uv pip install -e ".[dev]"
 ```
 
 !!! tip
-    vLLM-omni is compatible with Python versions 3.10 to 3.12. However, we recommend developing with Python 3.12 to minimize the chance of your local environment clashing with our CI environment.
+    vLLM-Omni is compatible with Python versions 3.10 to 3.12. However, we recommend developing with Python 3.12 to minimize the chance of your local environment clashing with our CI environment.
 
 ### Linting
 
-vLLM-omni uses `pre-commit` to lint and format the codebase. See [pre-commit documentation](https://pre-commit.com/#usage) if `pre-commit` is new to you. Setting up `pre-commit` is as easy as:
+vLLM-Omni uses `pre-commit` to lint and format the codebase. See [pre-commit documentation](https://pre-commit.com/#usage) if `pre-commit` is new to you. Setting up `pre-commit` is as easy as:
 
 ```bash
 uv pip install pre-commit
 pre-commit install
 ```
 
-vLLM-omni's `pre-commit` hooks will now run automatically every time you commit.
+vLLM-Omni's `pre-commit` hooks will now run automatically every time you commit.
 
 !!! tip
     You can manually run the `pre-commit` hooks using:
@@ -94,20 +72,14 @@ For additional features and advanced configurations, refer to the:
 
 ### Testing
 
-vLLM-omni uses `pytest` to test the codebase.
+vLLM-Omni uses `pytest` to test the codebase.
 
 ```bash
-# Install the test dependencies used in CI (CUDA only)
-uv pip install -e ".[dev]" --torch-backend=auto
-
-# Install some common test dependencies (hardware agnostic)
-uv pip install pytest pytest-asyncio
-
 # Run all tests
 pytest tests/
 
 # Run tests for a single test file with detailed output
-pytest -s -v tests/test_logger.py
+pytest -s -v tests/test_omni_llm.py
 ```
 
 !!! warning
@@ -122,7 +94,7 @@ If you encounter a bug or have a feature request, please search existing issues
 
 ## Pull Requests & Code Reviews
 
-Thank you for your contribution to vLLM-omni! Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM-omni maintain the code quality and improve the efficiency of the review process.
+Thank you for your contribution to vLLM-Omni! Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM-Omni maintain the code quality and improve the efficiency of the review process.
 
 ### DCO and Signed-off-by
 
@@ -144,9 +116,9 @@ Only specific types of PRs will be reviewed. The PR title is prefixed appropriat
 - `[CI/Build]` for build or continuous integration improvements.
 - `[Doc]` for documentation fixes and improvements.
 - `[Model]` for adding a new model or improving an existing model. Model name should appear in the title.
-- `[Frontend]` For changes on the vLLM-omni frontend (e.g., OpenAI API server, `OmniLLM` class, etc.)
+- `[Frontend]` For changes on the vLLM-Omni frontend (e.g., OpenAI API server, `OmniLLM` class, etc.)
 - `[Kernel]` for changes affecting CUDA kernels or other compute kernels.
-- `[Core]` for changes in the core vLLM-omni logic (e.g., `OmniProcessor`, `OmniScheduler`, etc.)
+- `[Core]` for changes in the core vLLM-Omni logic (e.g., `OmniProcessor`, `OmniScheduler`, etc.)
 - `[Hardware][Vendor]` for hardware-specific changes. Vendor name should appear in the prefix, such as [Ascend] for Ascend NPUs.
 - `[Misc]` for PRs that do not fit the above categories. Please use this sparingly.
 
@@ -161,7 +133,7 @@ The PR needs to meet the following code quality standards:
 - Pass all linter checks.
 - The code needs to be well-documented to ensure future contributors can easily understand the code.
 - Include sufficient tests to ensure the project stays correct and robust. This includes both unit tests and integration tests.
-- Please add documentation to `docs/` if the PR modifies the user-facing behaviors of vLLM-omni. It helps vLLM-omni users understand and utilize the new features or changes.
+- Please add documentation to `docs/` if the PR modifies the user-facing behaviors of vLLM-Omni. It helps vLLM-Omni users understand and utilize the new features or changes.
 
 ### Notes for Large Changes
 
@@ -169,13 +141,12 @@ Please keep the changes as concise as possible. For major architectural changes
 
 ### What to Expect for the Reviews
 
-The goal of the vLLM-omni team is to be a _transparent reviewing machine_. We would like to make the review process transparent and efficient and make sure no contributor feels confused or frustrated. However, the vLLM-omni team is small, so we need to prioritize some PRs over others. Here is what you can expect from the review process:
+The goal of the vLLM-Omni team is to be a _transparent reviewing machine_. We would like to make the review process transparent and efficient and make sure no contributor feels confused or frustrated. However, the vLLM-Omni team is small, so we need to prioritize some PRs over others. Here is what you can expect from the review process:
 
 - After the PR is submitted, the PR will be assigned to a reviewer. Every reviewer will pick up the PRs based on their expertise and availability.
-- After the PR is assigned, the reviewer will provide status updates every 2-3 days. If the PR is not reviewed within 7 days, please feel free to ping the reviewer or the vLLM-omni team.
+- After the PR is assigned, the reviewer will provide status updates every 2-3 days. If the PR is not reviewed within 7 days, please feel free to ping the reviewer or the vLLM-Omni team.
 - After the review, the reviewer will put an `action-required` label on the PR if there are changes required. The contributor should address the comments and ping the reviewer to re-review the PR.
 - Please respond to all comments within a reasonable time frame. If a comment isn't clear or you disagree with a suggestion, feel free to ask for clarification or discuss the suggestion.
-- Note that not all CI checks will be executed due to limited computational resources. The reviewer will add `ready` label to the PR when the PR is ready to merge or a full CI run is needed.
 
 ## Additional Resources
 
@@ -183,4 +154,4 @@ The goal of the vLLM-omni team is to be a _transparent reviewing machine_. We wo
 
 ## Thank You
 
-Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM-omni. All of your contributions help make vLLM-omni a great tool and community for everyone!
+Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM-Omni. All of your contributions help make vLLM-Omni a great tool and community for everyone!

docs/contributing/model/README.md

@@ -1 +1,2 @@
-# Summary
+# Adding a New Model
+Stay tuned for guidance on how to add a new model on vLLM-Omni!

docs/design/api_design_doc.md

@@ -1,6 +1,6 @@
 # API Design Documentation
 
-This document contains comprehensive API design documentation for all core modules in vLLM-omni. These templates provide a standardized structure for designing and implementing the core, engine, executor, and worker modules.
+This document contains comprehensive API design documentation for all core modules in vLLM-Omni. These templates provide a standardized structure for designing and implementing the core, engine, executor, and worker modules.
 
 ## 📋 Module API
 

docs/design/api_design_template.md

@@ -1,6 +1,6 @@
-# API Design Template for vLLM-omni Modules
+# API Design Template for vLLM-Omni Modules
 
-This template provides a standardized structure for designing APIs for core, engine, executor, and worker modules in vLLM-omni. Use this template to ensure consistency and completeness across all modules.
+This template provides a standardized structure for designing APIs for core, engine, executor, and worker modules in vLLM-Omni. Use this template to ensure consistency and completeness across all modules.
 
 ## Template Structure
 
@@ -44,7 +44,7 @@ This template provides a standardized structure for designing APIs for core, eng
 
 ### 1. Module Overview
 
-**Purpose**: The core module provides fundamental scheduling and caching functionality for vLLM-omni.
+**Purpose**: The core module provides fundamental scheduling and caching functionality for vLLM-Omni.
 
 **Responsibilities**:
 - Request scheduling and prioritization
@@ -333,4 +333,4 @@ print(f"Cache hit rate: {status['cache_hit_rate']:.2%}")
 6. Include error handling strategies
 7. Submit for review before implementation
 
-This template ensures consistency and completeness across all vLLM-omni modules.
+This template ensures consistency and completeness across all vLLM-Omni modules.

docs/design/architecture_overview.md

@@ -2,11 +2,11 @@
 
 # Introduction
 
-This document outlines the architectural design for vLLM-omni, a new repository developed under the vLLM project. vLLM-omni supports multi-modality models with non-autoregressive structures and non-textual outputs, extending vLLM beyond traditional text-based, autoregressive generation.
+This document outlines the architectural design for vLLM-Omni, a new repository developed under the vLLM project. vLLM-Omni supports multi-modality models with non-autoregressive structures and non-textual outputs, extending vLLM beyond traditional text-based, autoregressive generation.
 
 # Goals
 
-The primary goals for vLLM-omni are:
+The primary goals for vLLM-Omni are:
 
 * **Support for Non-textual Output:** Enables the integration, efficient processing and output of various data types, including but not limited to, images, audio, and video, alongside text.  
 * **Non-Autoregressive Structure:** Support model structure beyond autoregressive, especially Diffusion Transformer (DiT), which is widely used in visual and audio generation.  
@@ -57,12 +57,12 @@ According to analysis for current popular open-source models, most of them have
 
 # Architectural Overview
 
-vLLM-omni will import core modules from existing vLLM framework and introduce several new components and modifications to achieve its goals. The core architecture will be based on a modular design, allowing for independent development and integration of different modality handlers and output generators.  
+vLLM-Omni will import core modules from existing vLLM framework and introduce several new components and modifications to achieve its goals. The core architecture will be based on a modular design, allowing for independent development and integration of different modality handlers and output generators.  
 ![][image6]
 
 ## General ideas
 
-* It divides models in vLLM-omni into multiple stages, which are processed by different Engine Cores. Each stage can choose AR Engine Core for AR model part, and Diffusion Engine Core for diffusion model part.  
+* It divides models in vLLM-Omni into multiple stages, which are processed by different Engine Cores. Each stage can choose AR Engine Core for AR model part, and Diffusion Engine Core for diffusion model part.  
 * Designed a specialized diffusion engine core for DiT or dLLM.  
 * Plug-in with external DiT acceleration modules such as xDiT, Cache-DiT.  
 * Output linked with Higress API, Gradio for online demo, ComfyUI for less-code process.
@@ -89,32 +89,6 @@ vLLM-omni will import core modules from existing vLLM framework and introduce se
 
 ![][image11]
 
-# Roadmap
-
-* AsyncLLM:  
-- [ ] Initializer: resource allocation for different stages (2025 H2)  
-- [ ] Hidden state Output Processor (2025 H2)  
-- [ ] Multimodal Output Processor (2025 H2)  
-- [ ] Interface for API, Gradio, ComfyUI  
-* OmniEngine:  
-- [ ] Basic Diffusion Engine core: Diffusion step manager, Cache manager (2025 H2)  
-- [ ] Pipeline of different Engine Cores (2025 H2)  
-- [ ] Communication between Engine Cores via AsyncLLM (2025 H2)  
-* Model support (Included but not limited to):  
-- [ ] DiT: Qwen-Image (2025 H2), Flux-Kontext  
-- [ ] Unified-MLLM: BAGEL (2025 H2), OmniGen2, Janus-Pro  
-- [ ] Extensions of AR+DiT: Qwen-omni (2025 H2)  
-* Optimization  
-- [ ] Integration of other DiT acceleration modules, such as xDiT, Cache-DiT  
-- [ ] Disaggregation for AR and DiT  
-- [ ] Integration with vLLM's existing KV cache for potential future hybrid models.  
-- [ ] Support for streaming multi-modal inputs and outputs
-
-# Contact
-
-For any questions or feedback regarding this design document, please contact  
-Han Gao: [hgaoaf@connect.ust.hk](mailto:hgaoaf@connect.ust.hk),  
-Hongsheng Liu: [vllm-project@gmail.com](mailto:vllm-project@gmail.com)
 
 [image1]: <data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAALYAAAFwCAYAAAAVNoEZAABxw0lEQVR4XuydBVRcWb6ve90709OStnTScel4x93dIUaEuBJ3dzciRPDgEtzdCRpCBCIEEkIgQHCCBHe+t6uSvNvNnZ55r2/fng5zvrX+69TZp4qkan3nz2/vKup88sm/OWqXL3HxwnnOnj7JubOnuaB6jqtXLqJ28TxXRWmrX8Ha0gIzE0OuX1bFQFsTcxMDLMyMMDXS44bWVRxsLLG5aYKuxlVsLMxwc3bAWF8bNzdnHGwt8fb2wNXZHlcnW3HMDjsbC4Ju3cLO2hwvT3ecnexwsLcmIMAHHy83+X18PVxxtLMUt+1xcbTBw8URRwdr+f1tLM1wd3PB08NZlAs24ue4uTrKf0bD5yfxb8qWDWvZtnE9W0Vt37yBvTu3cWDPTlE7OHviCBfPnsTEwECIfg7Vk0e5fkkVoxtamBneQF1NlSsXTmMrZJaN6elcF7LexFaIZ6SngbOjvfy2l5DUSwhoa2mK9U1jfL09ibwTKaS3IPx2BA4ONnh7uuHr4yUX1d/Pi0B/X1xd7PHz8cTT3UUc85SfIMHBQUJ4a3EfH/mJ4OsjThoXB/njPNydJbEl3rFisTKrli5i9fLFrF21jC0bVdi2aR07tqzn+KF9nD91HF1NDU4eO8CZIwe5fO4UBjoacpEvnDnG5fMnsDQ1Rk/rOje0rwmxLeQd3chAC0d7W6zMjfBwc8JHiGlmpIu5oS6hISHcibgtOrE99+4+wE50dR8hraeHm1zOW4G+BAb4yjtyYICf/FiAED0wMICw0DDxODvR8QNEBeLv7yO6txPeotN7e7lLYku8Y8akccycOhGlGVNRVprJsoXzWLlEmbUrl7Jv5xbOHD8sBD/ApnWrOLZvN2ePH0FDRJVronMf3rdDiH1aiK4p9s+JE0AWS6xE59bA2EAHO6ub3DTRw1UI7Ovjg/oVEWV01Llz567ouH6i+wbz8FEMtiKaBPr7yzuvn6+nED9I3rH9/bwJDgqUSy3r1OHh4YSEBOPu7io6/W1CQ0PFSeAvxHeTd3B/X19JbIl3TB4zjOnjRzJz8jiUFCazYPZ0li1QYrXo4nt3bObE4QPs3LIRlRWLObh7h9jfj5rqGdGtj7N/11Yh9hk0r14W+yfQ0bgiZLZC8/pljPR1ROwwwdxYF2d7GyGfN1cunkJfiB0h4oe3lweRkfeIjn4iF9vf1wcnkbX9fL0ICb4ll1kutrgtkzdEyB4ZGUlQ0C35Y+/cuSPv3oHimOxkkHX2W6KjN3x+Ev+mjBzQmzGD+zJxxCAmjxrC9AmjUBZyL1dWEkJvEF17G6tEB1++aB47Nq0Xsm+Vy71PbHdv2yTP4LL9Q3t3onVdTYhsyCXV02LyqCuyuS762texlk0oXV05e/IQetrqBInuK5v8RUc/JiLiDvYiisgmf7Y2N0UssZJv1a9fFQJ7yiUPkXXrsBAePHiAn+jMsghy//590dlDRRTxk8cW2VhoWLAktsQ7Bv7UleH9egi5+8kFl3Xw+TOnsmTBLDGpXMsOMaFcunCuiChKbF27ml1CbtnEcvt6FbnY504eE517uxB+CxpXLwlxNbhwXjbhvCGXWFOMmZsZibztwMmj+7khsniA6Kwurs4iSoi87OKCuuj4urqa3DQ3xdHBVt6pXZwd0NJQJ1AmbGiw6NC3xYkQLbK6lxA6WC55yP8VWxZrbhEWHiKJLfGOPl06MKhnVwb06Cy2XZg0aqiIJJNYNHcG61cvl2dtZXF78byZrF22iHUrl7B53WqRwxcI6dfLO/WW9Wvk3f3i2VOonj7BeRFT9HW0ULtwRoh/FIMb2hgZ6otMLuvql9klos3COYri3xrOiEH96dujK907d+Snrp3o37c3Gzdu4MKFCxjqG7Bn9y7uRt6Vd+gHUQ9xFsKHiMmnbEy29ReRRbaaEhYWJo8qDZ+fxL8pfbt2pF+3TqJ+ZHCvLiKODGX2tAly8VRWLJFPJBcoKcpFX6E8j9WL300slyrPkXd0WdfeIE4A2UqKbHlQtpJy7tQxtNWvckZILYspOprX0dFSZ8+2DezcqMKsCaOZOGwQvTp1oHPbVnRo2YL2rZrz/TdN+O6rJrRp8YMQvA+q5y9w9NARbgUFEf3wEffuR2FrZyUXOiIiQnTyEPmkUSa2LNLcu/dAElviHb06taF357bIBB/cqzNjh/YXOXs0cxWnsWjebPlKyVwRTebNmILyLAWWzJ3NMuW5LJijIO/o61YvY/WShWxUWcmB3dvZLgQ/emAvZ04cZc/2zULmTaiKuLJv22ZmjxvJqH496dK6pagW9Gnfhp7tWtH5h+9p//23tPruG1qI+v6br/n2qy/47K9/odn337FGRQUHJ2fCbt/GyFhPxI4Q+eRRJrivr7d8AvngQZTo6tGS2BLv6P6jELtLO7nY/bv/yPCBPZkiBJylMIV5QuS5M6czR3EKStMnMU8m+5yZLFGazdwZU1mzfAmrly2UTyzXr1omjyMb1qxgn8jch/fvYdsGFXZsXMMusV0i7j+6V1eGdm3PGBF5pvXtztxBPZjetyujxL8/VPw/erRuzo9C8pZNheBNv+b7r7/iy88/o2PHjixcspTbtyO4IbK4bMnvzp13KySyzB0cIptYRktiS/wXnzdpzpdNmtGtQyt+6tiK/j06MXbEYCaPG8WMKROYOWUc0yaOY6ro4orittKMacxWmMysaeNZtmAei+fPEZ19FquWLmSdkHv5ovnyzL1dxJQ1Qvr1ohaOG8rxJTO4ukYJ/a1LODprDEsGdmdkux8Y2KopHb/+knZffk6LJp/xvRD5e3G7+Zdf0PKbr2gtJG8jZP/uqy9Zu24tqmdOEn77jrxjy96wkb1BExERyd2794m4c1cSW+Idn33ZlL99IaJA6x/o1KaZ6N7tGTmkP+NGDpXLPG38KLnkk8eNYPqkMXKpZ0ydIKQfK2LJHHkHV1aaIYReIM/jMtFlgss691KRzeeKn6G7YwVup3fgr3YAw83KLO/VkbX9OrNhYBfmd23JrC6tmNT+B7p89zVdRafu1uwbOjf9ivbffUUHsS/r4h3F/69Fs2YsWbhAZOn78owdGBgol1vWyWWTyYg7EZLYEu9o/cVXrO71E6aL53Nx1mT6iLw9sE9Phg/qx/jhQ+Q1doTYjhryXu6xTJNtx4scrjCVOSKiKImoIsvispJ19MULlEQGn8GsYf05vWI2+aGOZHqb4LJvNY+v7iP2+hEeXdpL2NF1uG9V5ti4nuwb1pUlPVoz88cWTGzblHGihrVuypA23zOsfXOGCPE7t/qBZt9+g7qmFmFh4XKxZXlb9kbPbdG1ZZ87afj8JP5NGd2mFfuHD8BCeQ43F8/hzMxxzB3al5H9fmJo/z4MH9CXEYMHMEpMKscNFxFl9AjGjxjERNHRFSbIJB/N9MljmDV9MjOnTUJh8nhmT5/CxNHDMNu/luI77pRGB5Dhb0t1zB2qnoRREuxEnpsx989sI3T/apzXKXBz6WRMl01Bf9F4VKcP5uj4Pmwd1p21g7ui3Lsj83p1oEerZrQV3fv7pt9y4uRxbgUHERIWRlh4BMFC9FvBoZLYEu/YObgfp0YO4ZLowtemjhViKXB1wVT2KIxBccRAxorOPVgIPnRQX8YMHSjv4GPeSz5p7EgmjhkhjywyoadPkmXxMfJMPmHoAOJstKhLjKLiaTiVCQ+pSY6jJj6K0jA3CjxNSdI/S4L2MR5c2sW9CzsIOqKC156lGC2bivr8MRye0I+dI3uyelBXlvXvxMA2zWnb7Ds+//QvjBkziiAxaQwJDRWTx3D5kqC/9Ja6xAcU27Rkcgvx67/59yzo2JrDo/thvVYZj51rCT20leCDm7DfvALN5XNRHj2UcQP7MKRPV9HJe4suLuvkQvZRw+SCTxg9XH57pDgZji6bSXVcJHWZidS+SaMuN43alGdysasfBlER6UNpsCNv/azJctQh1eoacdrHuX9hF67bFmK+SpFz04dwZHw/to3sxfqh3ZgssniXZl/LlwT/+p//ia+vn5g8+uPt7SX/YJSrq4sktsQ7un39Fd2/+ZoB337NpJZNWdezI/pKY3FSmU+o2mVCjc3xd/LCw9kHbSNrDl/QQFFpAaNHDGNov3dZfIzo3rLJ5tiR7/L4zJGDCBQ5ui4tntr8TOpF1WWnUJsUQ+2z+1RHBVF1L4DSUFeKAuzJdjEkzUaT+BtniLq8F+ftypitVOCsEPvoxH7sHt2XzUN7MLtHGwa0aEpLIfYXf/sMS0sLPDw8cHR0xN7ORl4Nn5/Evymd2rXgxzY/0EF0wj5dO3NE2xTj5CLMC+rQL6zDtKQe+/I6rEvrMS6owfBNFQY5leimlnLI0pOl2/aLDN6X0UPede9TG1dxS/sMiO5cHO5DcYgHed4OZNkZURbqSfWDYKrvBlN+25/iIDcK/Rx542JEpp0uTzWPEXl+Jw6b52O8fDrnFYZxcupg9o7pK7p2Dxb3aYdil5a0+eYrvvnicw4e3I/lTQsM9G+gr6eL3g1dSWyJd3T44Rt+bN2C+bsOc9w7ApMiuFkMN/JrsSyuwbm8FuuiWkzF/s3Caizf1mCZX4VpbgV6yW+5/iCRpVv3MkFMJBdMn0BehDf50SGQLaLH6xfUy+JHVCiVYZ6UeltRGexO9T2xHxFI0S2RtX0cyHMxI9P2Bo9Flw8/vQ3bjfMwWDqVc0LsE1MGsXd0H7aP6M6SPu2Z3bUVbb9pwjdffsnKlcvQ1rjO1SuXuHZVVpclsf8gNomSvdgfKuaXh//1qD3O4fLrKjRya9HJrUE3uxqr4mrcy2vwLKvDMKsKSyG0s9i3Lxad+20txqJj3xRlJeQ2yyhBNy6Dq0GPWLN8Kbn3g6jNEdEjL5uHphZcO3QRNyNHEnyCqY2NpfbFC2rjnlBxL0x0clkMuUmSmQZPdS9wR3U/t4Line truncated
 

docs/design/index.md

@@ -1,12 +1,13 @@
 # Design Documents
 
-This section contains design documents and architecture specifications for vLLM-omni.
+This section contains design documents and architecture specifications for vLLM-Omni.
 
 ## Architecture Documents
 
-- [vLLM-omni Software Design Document](vllm_omni_design.md)
 - [Architecture Overview](architecture_overview.md)
+- [vLLM-Omni Software Design Document](vllm_omni_design.md)
+- [Multi-Request Streaming](mrs_design.md)
 
 ## API Design Documentation
 
-- [vLLM-omni API Documentation](api_design_doc.md)
+- [vLLM-Omni API Design Documentation](api_design_doc.md)

docs/design/vllm_omni_design.md

@@ -2,7 +2,7 @@
 
 ## Overview
 
-vLLM-omni is a multi-modality extension for vLLM that supports non-autoregressive structures and non-textual outputs. This document outlines the key software abstractions, APIs, and dependencies for the system, designed to maximize reuse of vLLM's proven architecture.
+vLLM-Omni is a multi-modality extension for vLLM that supports non-autoregressive structures and non-textual outputs. This document outlines the key software abstractions, APIs, and dependencies for the system, designed to maximize reuse of vLLM's proven architecture.
 
 ## Architecture Principles
 
@@ -36,7 +36,7 @@ vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8000
 #### Design Architecture
 ```mermaid
 graph TD
-    A[vllm serve model --omni] --> B[vLLM-omni CLI Wrapper]
+    A[vllm serve model --omni] --> B[vLLM-Omni CLI Wrapper]
     B --> C{Detect --omni flag}
     C -->|Yes| D[Parse OmniConfig]
     C -->|No| E[Forward to vLLM CLI]
@@ -45,7 +45,7 @@ graph TD
     G --> H[Multi-stage Processing]
     E --> I[Standard vLLM Pipeline]
 
-    subgraph "vLLM-omni Components"
+    subgraph "vLLM-Omni Components"
         F
         G
         H
@@ -68,7 +68,7 @@ def main():
     """Main CLI entry point that intercepts vLLM commands"""
     # Check if --omni flag is present
     if "--omni" in sys.argv:
-        # Remove --omni flag and process with vLLM-omni
+        # Remove --omni flag and process with vLLM-Omni
         omni_args = [arg for arg in sys.argv[1:] if arg != "--omni"]
         omni_serve = OmniServeCommand()
         omni_serve.run(omni_args)

docs/features/README.md

@@ -1,3 +0,0 @@
-# Features
-
-vLLM-Omni introduces new features to support multimodal comprehension and generation models inference and serving.

docs/getting_started/installation/README.md

@@ -3,5 +3,5 @@
 vLLM supports the following hardware platforms:
 
 - [GPU](gpu.md)
-    - [NVIDIA CUDA](gpu.md#nvidia-cuda)
+    - [NVIDIA CUDA](gpu.md)
 - [Ascend NPU](npu.md)

docs/getting_started/installation/gpu.md

@@ -1,12 +1,6 @@
 # GPU
 
-vLLM-Omni is a Python library that supports the following GPU variants. Select your GPU type to see vendor specific instructions:
-
-[](){ #nvidia-cuda }
-
-=== "NVIDIA CUDA"
-
-    --8<-- "docs/getting_started/installation/gpu/cuda.inc.md:installation"
+vLLM-Omni is a Python library that supports the following GPU variants. The library itself mainly contains python implementations for framework and models.
 
 ## Requirements
 
@@ -14,7 +8,7 @@ vLLM-Omni is a Python library that supports the following GPU variants. Select y
 - Python: 3.12
 
 !!! note
-    vLLM-Omni does not support Windows natively.
+    vLLM-Omni is currently not natively supported on Windows.
 
 === "NVIDIA CUDA"
 
@@ -26,10 +20,6 @@ vLLM-Omni is a Python library that supports the following GPU variants. Select y
 
 --8<-- "docs/getting_started/installation/python_env_setup.inc.md"
 
-=== "NVIDIA CUDA"
-
-    --8<-- "docs/getting_started/installation/gpu/cuda.inc.md:set-up-using-python"
-
 ### Pre-built wheels
 
 === "NVIDIA CUDA"
@@ -47,3 +37,4 @@ vLLM-Omni is a Python library that supports the following GPU variants. Select y
 ## Set up using Docker
 
 ### Pre-built images
+To be released... Stay tuned!

docs/getting_started/installation/gpu/cuda.inc.md

@@ -1,8 +1,3 @@
-# --8<-- [start:installation]
-
-vLLM-Omni mainly contains python implementations for framework and models.
-
-# --8<-- [end:installation]
 # --8<-- [start:requirements]
 
 - GPU: compute capability 7.0 or higher (e.g., V100, T4, RTX20xx, A100, L4, H100, etc.)
@@ -21,14 +16,14 @@ Therefore, it is recommended to install vLLM and vLLM-Omni with a **fresh new**
 
 # --8<-- [start:pre-built-wheels]
 
-## Installation of vLLM
+#### Installation of vLLM
 
-Now we build it based on vLLM v0.11.0. Please install it with command below.
+vLLM-Omni is built based on vLLM v0.11.0. Please install it with command below.
 ```bash
 uv pip install vllm==0.11.0 --torch-backend=auto
 ```
 
-## Installation of vLLM-Omni
+#### Installation of vLLM-Omni
 
 ```bash
 uv pip install vllm-omni
@@ -38,15 +33,30 @@ uv pip install vllm-omni
 
 # --8<-- [start:build-wheel-from-source]
 
-## Installation of vLLM
-If you want to check or debug with source code of vLLM, install stable release version of vllm with 0.11.0 from source with pre-built wheel file.
+#### Installation of vLLM
+If you do not need to modify source code of vLLM, you can directly install the stable 0.11.0 release version of the library
+
+```bash
+uv pip install vllm==0.11.0 --torch-backend=auto
+```
+
+#### Installation of vLLM-Omni
+Install additional requirements for vLLM-Omni
+```bash
+git clone https://github.com/vllm-project/vllm-omni.git
+cd vllm_omni
+uv pip install -e .
+```
+
+<details><summary>(Optional) Installation of vLLM from source</summary>
+If you want to check, modify or debug with source code of vLLM, install the library from source with the following instructions:
 
 ```bash
 git clone https://github.com/vllm-project/vllm.git
 cd vllm
 git checkout v0.11.0
 ```
-Set up environment variables to get pre-built wheels. If there are internet problems, just download the whl file manually. And set VLLM_PRECOMPILED_WHEEL_LOCATION as your local absolute path of whl file.
+Set up environment variables to get pre-built wheels. If there are internet problems, just download the whl file manually. And set `VLLM_PRECOMPILED_WHEEL_LOCATION` as your local absolute path of whl file.
 ```bash
 export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.11.0/vllm-0.11.0-cp38-abi3-manylinux1_x86_64.whl
 ```
@@ -54,20 +64,12 @@ Install vllm with command below (If you have no existing PyTorch).
 ```bash
 uv pip install --editable .
 ```
-
 Install vllm with command below (If you already have PyTorch).
 ```bash
 python use_existing_torch.py
 uv pip install -r requirements/build.txt
 uv pip install --no-build-isolation --editable .
 ```
-
-## Installation of vLLM-omni
-Install additional requirements for vllm-omni
-```bash
-git clone https://github.com/vllm-project/vllm-omni.git
-cd vllm_omni
-uv pip install -e .
-```
+</details>
 
 # --8<-- [end:build-wheel-from-source]

docs/getting_started/quickstart.md

@@ -20,4 +20,4 @@ Please refer to [offline inference](../user_guide/examples/offline_inference/qwe
 
 ## Online Serving with OpenAI-Completions API
 
-Please refer to [online serving](../user_guide/examples/online_serving/README.md)
+Please refer to [online serving](../user_guide/examples/online_serving/qwen2_5_omni.md)

docs/mkdocs/hooks/generate_api_readme.py

@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM-omni project
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
 """
 Hook to automatically generate docs/api/README.md from the codebase.
 
@@ -20,7 +20,7 @@ API_README_PATH = ROOT_DIR / "docs" / "api" / "README.md"
 CATEGORIES = {
     "entrypoints": {
         "name": "Entry Points",
-        "description": "Main entry points for vLLM-omni inference and serving.",
+        "description": "Main entry points for vLLM-Omni inference and serving.",
     },
     "inputs": {
         "name": "Inputs",

docs/mkdocs/hooks/generate_examples.py

@@ -1,233 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM-omni project
-import itertools
-import logging
-from dataclasses import dataclass, field
-from pathlib import Path
-from typing import Literal
-
-import regex as re
-
-logger = logging.getLogger("mkdocs")
-
-ROOT_DIR = Path(__file__).parent.parent.parent.parent
-ROOT_DIR_RELATIVE = "../../../../.."
-EXAMPLE_DIR = ROOT_DIR / "examples"
-EXAMPLE_DOC_DIR = ROOT_DIR / "docs/user_guide/examples"
-
-
-def fix_case(text: str) -> str:
-    subs = {
-        "api": "API",
-        "cli": "CLI",
-        "cpu": "CPU",
-        "llm": "LLM",
-        "mae": "MAE",
-        "tpu": "TPU",
-        "gguf": "GGUF",
-        "lora": "LoRA",
-        "rlhf": "RLHF",
-        "vllm": "vLLM",
-        "omni": "Omni",
-        "openai": "OpenAI",
-        "lmcache": "LMCache",
-        "multilora": "MultiLoRA",
-        "mlpspeculator": "MLPSpeculator",
-        r"fp\d+": lambda x: x.group(0).upper(),  # e.g. fp16, fp32
-        r"int\d+": lambda x: x.group(0).upper(),  # e.g. int8, int16
-    }
-    for pattern, repl in subs.items():
-        text = re.sub(rf"\b{pattern}\b", repl, text, flags=re.IGNORECASE)
-    return text
-
-
-@dataclass
-class Example:
-    """
-    Example class for generating documentation content from a given path.
-
-    Attributes:
-        path (Path): The path to the main directory or file.
-        category (str): The category of the document.
-        main_file (Path): The main file in the directory.
-        other_files (list[Path]): list of other files in the directory.
-        title (str): The title of the document.
-
-    Methods:
-        __post_init__(): Initializes the main_file, other_files, and title attributes.
-        determine_main_file() -> Path: Determines the main file in the given path.
-        determine_other_files() -> list[Path]: Determines other files in the directory excluding the main file.
-        determine_title() -> str: Determines the title of the document.
-        generate() -> str: Generates the documentation content.
-    """  # noqa: E501
-
-    path: Path
-    category: str = None
-    main_file: Path = field(init=False)
-    other_files: list[Path] = field(init=False)
-    title: str = field(init=False)
-
-    def __post_init__(self):
-        self.main_file = self.determine_main_file()
-        self.other_files = self.determine_other_files()
-        self.title = self.determine_title()
-
-    @property
-    def is_code(self) -> bool:
-        return self.main_file.suffix != ".md"
-
-    def determine_main_file(self) -> Path:
-        """
-        Determines the main file in the given path.
-        If the path is a file, it returns the path itself. Otherwise, it searches
-        for Markdown files (*.md) in the directory and returns the first one found.
-        Returns:
-            Path: The main file path, either the original path if it's a file or the first
-            Markdown file found in the directory.
-        Raises:
-            IndexError: If no Markdown files are found in the directory.
-        """  # noqa: E501
-        return self.path if self.path.is_file() else list(self.path.glob("*.md")).pop()
-
-    def determine_other_files(self) -> list[Path]:
-        """
-        Determine other files in the directory excluding the main file.
-
-        This method checks if the given path is a file. If it is, it returns an empty list.
-        Otherwise, it recursively searches through the directory and returns a list of all
-        files that are not the main file.
-
-        Returns:
-            list[Path]: A list of Path objects representing the other files in the directory.
-        """  # noqa: E501
-        if self.path.is_file():
-            return []
-        # Text file extensions that can be safely included
-        text_extensions = {
-            ".py",
-            ".md",
-            ".sh",
-            ".yaml",
-            ".yml",
-            ".json",
-            ".txt",
-            ".toml",
-            ".cfg",
-            ".ini",
-        }
-        is_other_file = lambda file: (file.is_file() and file != self.main_file and file.suffix in text_extensions)  # noqa: E731
-        return [file for file in self.path.rglob("*") if is_other_file(file)]
-
-    def determine_title(self) -> str:
-        if not self.is_code:
-            # Specify encoding for building on Windows
-            with open(self.main_file, encoding="utf-8") as f:
-                first_line = f.readline().strip()
-            match = re.match(r"^#\s+(?P<title>.+)$", first_line)
-            if match:
-                return match.group("title")
-        return fix_case(self.path.stem.replace("_", " ").title())
-
-    def fix_relative_links(self, content: str) -> str:
-        """
-        Fix relative links in markdown content by converting them to gh-file
-        format.
-
-        Args:
-            content (str): The markdown content to process
-
-        Returns:
-            str: Content with relative links converted to gh-file format
-        """
-        # Regex to match markdown links [text](relative_path)
-        # This matches links that don't start with http, https, ftp, or #
-        link_pattern = r"\[([^\]]*)\]\((?!(?:https?|ftp)://|#)([^)]+)\)"
-
-        def replace_link(match):
-            link_text = match.group(1)
-            relative_path = match.group(2)
-
-            # Make relative to repo root
-            gh_file = (self.main_file.parent / relative_path).resolve()
-            gh_file = gh_file.relative_to(ROOT_DIR)
-
-            # Make GitHub URL
-            url = "https://github.com/vllm-project/vllm-omni/"
-            url += "tree/main" if self.path.is_dir() else "blob/main"
-            gh_url = f"{url}/{gh_file}"
-
-            return f"[{link_text}]({gh_url})"
-
-        return re.sub(link_pattern, replace_link, content)
-
-    def generate(self) -> str:
-        content = f"# {self.title}\n\n"
-        url = "https://github.com/vllm-project/vllm-omni/"
-        url += "tree/main" if self.path.is_dir() else "blob/main"
-        content += f"Source <{url}/{self.path.relative_to(ROOT_DIR)}>.\n\n"
-
-        # Use long code fence to avoid issues with
-        # included files containing code fences too
-        code_fence = "``````"
-
-        if self.is_code:
-            # Use relative path from ROOT_DIR for snippet inclusion
-            main_file_rel_path = self.main_file.relative_to(ROOT_DIR)
-            content += f'{code_fence}{self.main_file.suffix[1:]}\n--8<-- "{main_file_rel_path}"\n{code_fence}\n'
-        else:
-            with open(self.main_file, encoding="utf-8") as f:
-                # Skip the title from md snippets as it's been included above
-                main_content = f.readlines()[1:]
-            content += self.fix_relative_links("".join(main_content))
-        content += "\n"
-
-        if not self.other_files:
-            return content
-
-        content += "## Example materials\n\n"
-        for file in sorted(self.other_files):
-            # Use relative path from ROOT_DIR for snippet inclusion
-            file_rel_path = file.relative_to(ROOT_DIR)
-            content += f'??? abstract "{file.relative_to(self.path)}"\n'
-            if file.suffix != ".md":
-                content += f"    {code_fence}{file.suffix[1:]}\n"
-            content += f'    --8<-- "{file_rel_path}"\n'
-            if file.suffix != ".md":
-                content += f"    {code_fence}\n"
-
-        return content
-
-
-def on_startup(command: Literal["build", "gh-deploy", "serve"], dirty: bool):
-    logger.info("Generating example documentation")
-    logger.debug("Root directory: %s", ROOT_DIR.resolve())
-    logger.debug("Example directory: %s", EXAMPLE_DIR.resolve())
-    logger.debug("Example document directory: %s", EXAMPLE_DOC_DIR.resolve())
-
-    # Create the EXAMPLE_DOC_DIR if it doesn't exist
-    if not EXAMPLE_DOC_DIR.exists():
-        EXAMPLE_DOC_DIR.mkdir(parents=True)
-
-    categories = sorted(p for p in EXAMPLE_DIR.iterdir() if p.is_dir())
-
-    examples = []
-    glob_patterns = ["*.py", "*.md"]
-    # Find categorised examples
-    for category in categories:
-        globs = [category.glob(pattern) for pattern in glob_patterns]
-        for path in itertools.chain(*globs):
-            examples.append(Example(path, category.stem))
-        # Find examples in subdirectories
-        for path in category.glob("*/*.md"):
-            examples.append(Example(path.parent, category.stem))
-
-    # Generate the example documentation
-    for example in sorted(examples, key=lambda e: e.path.stem):
-        example_name = f"{example.path.stem}.md"
-        doc_path = EXAMPLE_DOC_DIR / example.category / example_name
-        if not doc_path.parent.exists():
-            doc_path.parent.mkdir(parents=True)
-        # Specify encoding for building on Windows
-        with open(doc_path, "w+", encoding="utf-8") as f:
-            f.write(example.generate())
-        logger.debug("Example generated: %s", doc_path.relative_to(ROOT_DIR))

docs/user_guide/examples/offline_inference/qwen2_5_omni.md

@@ -1,40 +1,55 @@
-# Offline Example of vLLM-omni for Qwen2.5-omni
+# Offline Inference Example of vLLM-Omni for Qwen2.5-Omni
 
 Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/qwen2_5_omni>.
 
 
 ## 🛠️ Installation
 
-Please refer to [README.md](https://github.com/vllm-project/vllm-omni/tree/main/README.md)
+Please refer to [installation](../../../getting_started/installation/README.md).
+
+## Run Offline inference with Qwen2.5-Omni.
+First, navigate to the example folder
+```bash
+cd examples/offline_inference/qwen2_5_omni
+```
+Inside the directory, `end2end.py` is a comprehensive demo suite for initializing a model instance of `Qwen/Qwen2.5-Omni-7B` and use it for various offline inference tasks.
+??? abstract "end2end.py"
+    ``````py
+    --8<-- "examples/offline_inference/qwen2_5_omni/end2end.py"
+    ``````
+
+Below we also provide simple bash scripts to execute this file.
+### Single Prompt
+```bash
+bash run_single_prompt.sh
+```
+??? abstract "run_single_prompt.sh"
+    ``````sh
+    --8<-- "examples/offline_inference/qwen2_5_omni/run_single_prompt.sh"
+    ``````
 
-## Run examples (Qwen2.5-omni)
 ### Multiple Prompts
-Download dataset from [seed_tts](https://drive.google.com/file/d/1GlSjVfSHkW3-leKKBlfrjuuTGqQ_xaLP/edit). To get the prompt, you can:
+Download dataset from [seed_tts](https://drive.google.com/file/d/1GlSjVfSHkW3-leKKBlfrjuuTGqQ_xaLP/edit) and extract the prompts with `extract_prompts.py`
+??? abstract "extract_prompts.py"
+    ``````py
+    --8<-- "examples/offline_inference/qwen2_5_omni/extract_prompts.py"
+    ``````
 ```bash
 tar -xf <Your Download Path>/seedtts_testset.tar
 cp seedtts_testset/en/meta.lst examples/offline_inference/qwen2_5_omni/meta.lst
 python3 examples/offline_inference/qwen2_5_omni/extract_prompts.py \
   --input examples/offline_inference/qwen2_5_omni/meta.lst \
   --output examples/offline_inference/qwen2_5_omni/top100.txt \
-  --topk 100
-```
-Get into the example folder
-```bash
-cd examples/offline_inference/qwen2_5_omni
+  --topk 10
 ```
 Then run the command below.
 ```bash
 bash run_multiple_prompts.sh
 ```
-### Single Prompts
-Get into the example folder
-```bash
-cd examples/offline_inference/qwen2_5_omni
-```
-Then run the command below.
-```bash
-bash run_single_prompt.sh
-```
+??? abstract "run_multiple_prompts.sh"
+    ``````sh
+    --8<-- "examples/offline_inference/qwen2_5_omni/run_multiple_prompts.sh"
+    ``````
 
 ### FAQ
 
@@ -43,22 +58,3 @@ If you encounter error about backend of librosa, try to install ffmpeg with comm
 sudo apt update
 sudo apt install ffmpeg
 ```
-
-## Example materials
-
-??? abstract "end2end.py"
-    ``````py
-    --8<-- "examples/offline_inference/qwen2_5_omni/end2end.py"
-    ``````
-??? abstract "extract_prompts.py"
-    ``````py
-    --8<-- "examples/offline_inference/qwen2_5_omni/extract_prompts.py"
-    ``````
-??? abstract "run_multiple_prompts.sh"
-    ``````sh
-    --8<-- "examples/offline_inference/qwen2_5_omni/run_multiple_prompts.sh"
-    ``````
-??? abstract "run_single_prompt.sh"
-    ``````sh
-    --8<-- "examples/offline_inference/qwen2_5_omni/run_single_prompt.sh"
-    ``````

docs/user_guide/examples/online_serving/README.md

@@ -1,80 +0,0 @@
-# Online serving Example of vLLM-omni for Qwen2.5-omni
-
-Source <https://github.com/vllm-project/vllm-omni/blob/main/examples/online_serving/README.md>.
-
-
-## 🛠️ Installation
-
-Please refer to [README.md](https://github.com/vllm-project/vllm-omni/blob/main/README.md)
-
-## Run examples (Qwen2.5-omni)
-
-Launch the server
-```bash
-vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091
-```
-
-If you have custom stage configs file, launch the server with command below
-```bash
-vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091 --stage-configs-path /path/to/stage_configs_file
-```
-
-Get into the example folder
-```bash
-cd examples/online_serving
-```
-
-Send request via python
-```bash
-python openai_chat_completion_client_for_multimodal_generation.py --query-type mixed_modalities
-```
-
-Send request via curl
-```bash
-bash run_curl_multimodal_generation.sh mixed_modalities
-```
-
-### FAQ
-
-If you encounter error about backend of librosa, try to install ffmpeg with command below.
-```
-sudo apt update
-sudo apt install ffmpeg
-```
-
-## Run Local Web UI Demo
-
-This Web UI demo allows users to interact with the model through a web browser.
-
-### Running Gradio Demo
-
-Once vllm and vllm-omni are installed, you can launch the web service built on AsyncOmniLLM by
-
-```bash
-python gradio_demo.py  --model Qwen/Qwen2.5-Omni-7B --port 7861
-```
-
-Then open `http://localhost:7861/` on your local browser to interact with the web UI.
-
-
-### Options
-
-The gradio demo also supports running with an existing API server and can be customized with the following arguments.
-
-
-```bash
-python gradio_demo.py \
-    --model Qwen/Qwen2.5-Omni-7B \
-    --use-api-server \
-    --api-base http://localhost:8091/v1 \
-    --ip 127.0.0.1 \
-    --port 7861
-```
-
-- `--model`: Model name
-- `--use-api-server`: If set, connect to an existing vLLM HTTP API server instead of running AsyncOmniLLM locally.
-- `--api-base`: Base URL for vllm serve (only used when `use-api-server` is set, default: http://localhost:8091/v1)
-- `--ip`: Host/IP for Gradio server (default: 127.0.0.1)
-- `--port`: Port for Gradio server (default: 7861)
-- `--stage-configs-path`: Path to custom stage configs YAML file (optional)
-- `--share`: Share the Gradio demo publicly (creates a public link)

docs/user_guide/examples/online_serving/qwen2_5_omni.md

@@ -0,0 +1,80 @@
+# Online Serving Example of vLLM-Omni for Qwen2.5-Omni
+
+Source <https://github.com/vllm-project/vllm-omni/blob/main/examples/online_serving/README.md>.
+
+
+## 🛠️ Installation
+
+Please refer to [installation](../../../getting_started/installation/README.md).
+
+## Deploy Qwen/Qwen2.5-Omni-7B
+
+First launch the OpenAI-compatible inference server
+```bash
+vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091
+```
+If you have custom stage configs file, launch the server with command below
+```bash
+vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091 --stage-configs-path /path/to/stage_configs_file
+```
+## Query the model
+Navigate to the example folder
+```bash
+cd examples/online_serving
+```
+Query the model server with OpenAI Python API client:
+```bash
+python openai_chat_completion_client_for_multimodal_generation.py --query-type mixed_modalities
+```
+??? abstract "openai_chat_completion_client_for_multimodal_generation.py"
+    ``````py
+    --8<-- "examples/online_serving/openai_chat_completion_client_for_multimodal_generation.py"
+    ``````
+You can also query the model with `curl` command:
+```bash
+bash run_curl_multimodal_generation.sh mixed_modalities
+```
+??? abstract "run_curl_multimodal_generation.sh"
+    ``````py
+    --8<-- "examples/online_serving/run_curl_multimodal_generation.sh"
+    ``````
+
+### FAQ
+
+If you encounter error about backend of librosa, try to install ffmpeg with command below.
+```
+sudo apt update
+sudo apt install ffmpeg
+```
+
+## Run Local Web UI Demo
+
+You can also deploy a Gradio Web UI that allows users to interact with the model through a web browser. Below is an example on how to do so with `Qwen/Qwen2.5-Omni-7B`.
+
+### Running Gradio Demo
+
+Install gradio with `uv pip install "gradio>=5.49.1,<6.0.0"`, then you can launch the web service built on AsyncOmni by
+
+```bash
+python gradio_demo.py  --model Qwen/Qwen2.5-Omni-7B --port 7861
+```
+Now you can interact with model via the web UI at `http://localhost:7861/` on your local browser.
+
+
+### Options
+
+You can customize its basic launch parameters:
+
+```bash
+python gradio_demo.py \
+    --model Qwen/Qwen2.5-Omni-7B \
+    --ip 127.0.0.1 \
+    --port 7861 \
+    --stage-configs-path /path/to/stage_configs.yaml
+```
+
+- `--model`: Local model checkpoint to load (default `Qwen/Qwen2.5-Omni-7B`).
+- `--ip`: Host/IP for the Gradio server (default `127.0.0.1`).
+- `--port`: Port for the Gradio server (default `7861`).
+- `--stage-configs-path`: Optional path to custom stage configs YAML.
+- `--share`: Set to expose a temporary public link via Gradio.

examples/offline_inference/qwen2_5_omni/README.md

@@ -1,4 +1,4 @@
-# Offline Example of vLLM-omni for Qwen2.5-omni
+# Offline Example of vLLM-Omni for Qwen2.5-omni
 
 ## 🛠️ Installation
 

examples/offline_inference/qwen2_5_omni/end2end.py

@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: Apache-2.0
 # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
 """
-This example shows how to use vLLM-omni for running offline inference
+This example shows how to use vLLM-Omni for running offline inference
 with the correct prompt format on Qwen2.5-Omni
 """
 

examples/online_serving/README.md

@@ -1,4 +1,4 @@
-# Online serving Example of vLLM-omni for Qwen2.5-omni
+# Online serving Example of vLLM-Omni for Qwen2.5-omni
 
 ## 🛠️ Installation
 

mkdocs.yml

@@ -56,7 +56,6 @@ theme:
   custom_dir: docs/mkdocs/overrides
 
 hooks:
-  - docs/mkdocs/hooks/generate_examples.py
   - docs/mkdocs/hooks/generate_api_readme.py
   - docs/mkdocs/hooks/url_schemes.py
 

pyproject.toml

@@ -5,12 +5,12 @@ build-backend = "setuptools.build_meta"
 [project]
 name = "vllm-omni"
 version = "0.11.0rc1"
-description = "vLLM-omni: Multi-modality models inference and serving with non-autoregressive structures"
+description = "vLLM-Omni: Multi-modality models inference and serving with non-autoregressive structures"
 readme = "README.md"
 requires-python = ">=3.9,<3.14"  # Align with vLLM v0.11 supported Python versions
 license = {text = "Apache-2.0"}
 authors = [
-    {name = "vLLM-omni Team", email = "vllm-project@gmail.com"}
+    {name = "vLLM-Omni Team", email = "vllm-project@gmail.com"}
 ]
 keywords = ["vllm", "multimodal", "diffusion", "transformer", "inference", "serving"]
 classifiers = [

vllm_omni/__init__.py

@@ -1,5 +1,5 @@
 """
-vLLM-omni: Multi-modality models inference and serving with
+vLLM-Omni: Multi-modality models inference and serving with
 non-autoregressive structures.
 
 This package extends vLLM beyond traditional text-based, autoregressive

vllm_omni/config/__init__.py

@@ -1,5 +1,5 @@
 """
-Configuration module for vLLM-omni.
+Configuration module for vLLM-Omni.
 """
 
 from vllm_omni.config.model import OmniModelConfig

vllm_omni/core/dit_cache_manager.py

@@ -1,5 +1,5 @@
 """
-DiT Cache Manager for vLLM-omni.
+DiT Cache Manager for vLLM-Omni.
 
 This module provides caching functionality for DiT (Diffusion Transformer) models
 to optimize inference performance and memory usage.

vllm_omni/core/sched/__init__.py

@@ -1,5 +1,5 @@
 """
-Scheduling components for vLLM-omni.
+Scheduling components for vLLM-Omni.
 """
 
 from .diffusion_scheduler import DiffusionScheduler

vllm_omni/core/sched/scheduler.py

@@ -7,11 +7,11 @@ from vllm.v1.core.sched.scheduler import Scheduler as VLLMScheduler
 
 class OmniScheduler(VLLMScheduler):
     """
-    OmniScheduler: Scheduler for vLLM-omni multimodal processing.
+    OmniScheduler: Scheduler for vLLM-Omni multimodal processing.
 
     This scheduler extends vLLM's scheduler to support multimodal and
     non-autoregressive processing with additional fields and methods
-    specific to vLLM-omni.
+    specific to vLLM-Omni.
     """
 
     # Ensure scheduled_new_reqs carry omni-specific payloads

vllm_omni/engine/__init__.py

@@ -1,5 +1,5 @@
 """
-Engine components for vLLM-omni.
+Engine components for vLLM-Omni.
 """
 
 from typing import Any, Optional

vllm_omni/entrypoints/async_omni_llm.py

@@ -53,7 +53,7 @@ logger = init_logger(__name__)
 
 
 class AsyncOmniLLM(EngineClient):
-    """Async entry point for vLLM-omni inference.
+    """Async entry point for vLLM-Omni inference.
 
     This class provides an asynchronous interface for running multi-modal
     comprehension and generation models. It orchestrates multiple

vllm_omni/entrypoints/cli/__init__.py

@@ -1,4 +1,4 @@
-"""CLI helpers for vLLM-omni entrypoints."""
+"""CLI helpers for vLLM-Omni entrypoints."""
 
 from .serve import OmniServeCommand
 

vllm_omni/entrypoints/cli/main.py

@@ -1,5 +1,5 @@
 """
-CLI entry point for vLLM-omni that intercepts vLLM commands.
+CLI entry point for vLLM-Omni that intercepts vLLM commands.
 """
 
 import importlib.metadata

vllm_omni/entrypoints/cli/serve.py

@@ -1,5 +1,5 @@
 """
-Omni serve command for vLLM-omni.
+Omni serve command for vLLM-Omni.
 """
 
 import argparse
@@ -52,7 +52,7 @@ class OmniServeCommand(CLISubcommand):
         serve_parser.add_argument(
             "--omni",
             action="store_true",
-            help="Enable vLLM-omni mode",
+            help="Enable vLLM-Omni mode",
         )
         serve_parser.add_argument(
             "--stage-configs-path",

vllm_omni/entrypoints/omni_llm.py

@@ -42,7 +42,7 @@ logger = init_logger(__name__)
 
 
 class OmniLLM:
-    """Main entry point for vLLM-omni inference.
+    """Main entry point for vLLM-Omni inference.
 
     This class provides a high-level interface for running multi-modal
     comprehension and generation models. It orchestrates multiple

vllm_omni/entrypoints/omni_stage.py

@@ -1,5 +1,5 @@
 """
-Stage manager for orchestrating multiple engines in vLLM-omni.
+Stage manager for orchestrating multiple engines in vLLM-Omni.
 
 Enhanced to encapsulate per-stage process lifecycle and worker logic
 (device setup, LLM init, batching, shared-memory IPC), while preserving

vllm_omni/worker/gpu_ar_model_runner.py

@@ -1,4 +1,4 @@
-"""AR GPU Model Runner for vLLM-omni.
+"""AR GPU Model Runner for vLLM-Omni.
 
 Exposes per-request hidden representations via ModelRunnerOutput.pooler_output
 and also outputs sampled tokens.

vllm_omni/worker/gpu_diffusion_model_runner.py

@@ -31,7 +31,7 @@ logger = logging.getLogger(__name__)
 
 
 class GPUDiffusionModelRunner(OmniGPUModelRunner):
-    """Diffusion model runner for vLLM-omni (non-autoregressive).
+    """Diffusion model runner for vLLM-Omni (non-autoregressive).
 
     - Reuses GPUModelRunner preparation, multimodal handling, and TP/PP/DP glue.
     - Does not compute logits or perform token sampling.