diff --git a/docs/contributing/profiling.md b/docs/contributing/profiling.md index 4d18d0fb4..a7df8c322 100644 --- a/docs/contributing/profiling.md +++ b/docs/contributing/profiling.md @@ -2,7 +2,7 @@ > **Warning:** Profiling incurs significant overhead. Use only for development and debugging, never in production. -vLLM-Omni uses the PyTorch Profiler to analyze performance across both **Multi-Stage LLMs** and **Diffusion Models**. +vLLM-Omni uses the PyTorch Profiler to analyze performance across both **multi-stage omni-modality models** and **diffusion models**. ### 1. Set the Output Directory Before running any script, set this environment variable. The system detects this and automatically saves traces here. @@ -11,7 +11,7 @@ Before running any script, set this environment variable. The system detects thi export VLLM_TORCH_PROFILER_DIR=./profiles ``` -### 2. Start Profiling +### 2. Profiling Omni-Modality Models It is best to limit profiling to one iteration to keep trace files manageable. @@ -77,12 +77,12 @@ omni_llm.close() **Examples**: -1. **Qwen-omni 2.5**: [https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/qwen2_5_omni/end2end.py](https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/qwen2_5_omni/end2end.py) +1. **Qwen2.5-Omni**: [https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/qwen2_5_omni/end2end.py](https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/qwen2_5_omni/end2end.py) -2. **Qwen-omni 3.0**: [https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/qwen3_omni/end2end.py](https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/qwen3_omni/end2end.py) +2. **Qwen3-Omni**: [https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/qwen3_omni/end2end.py](https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/qwen3_omni/end2end.py) -**For Diffusion Models as a single stage** +### 3. Profiling diffusion models Diffusion profiling is End-to-End, capturing encoding, denoising loops, and decoding. @@ -131,27 +131,10 @@ python image_to_video.py \ 2. **Wan-AI/Wan2.2-I2V-A14B-Diffusers**: [https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/image_to_video](https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/image_to_video) +> **Note:** +As of now, asynchronous (online) profiling is not fully supported in vLLM-Omni. While start_profile() and stop_profile() methods exist, they are only reliable in offline inference scripts (e.g., the provided end2end.py examples). Do not use them in server-mode or streaming scenarios—traces may be incomplete or fail to flush. -**Online Inference(Async)** - -For online serving using AsyncOmni, the methods are asynchronous. This allows you to toggle profiling dynamically without restarting the server. - -```python -from vllm_omni import AsyncOmni - -# Inside an async function: -async_omni = AsyncOmni.from_engine_args(engine_args) - -await async_omni.start_profile() - -async for output in async_omni.generate(prompt, sampling_params, request_id): - # Process outputs... - pass - -await async_omni.stop_profile() -``` - -### 3. Analyzing Omni Traces +### 4. Analyzing Omni Traces Output files are saved to your configured ```VLLM_TORCH_PROFILER_DIR```. @@ -163,4 +146,4 @@ Output files are saved to your configured ```VLLM_TORCH_PROFILER_DIR```. - [Perfetto](https://ui.perfetto.dev/)(recommended) - ```chrome://tracing```(Chrome only) -**Note**: vLLM-Omni reuses the PyTorch Profiler infrastructure from vLLM. See the official vLLM profiler documentation: [vLLM Profiling Guide](https://docs.vllm.ai/en/latest/dev/profiling.html) +**Note**: vLLM-Omni reuses the PyTorch Profiler infrastructure from vLLM. See the official vLLM profiler documentation: [vLLM Profiling Guide](https://docs.vllm.ai/en/stable/contributing/profiling/)