[Doc] format lora related docs for the user's end (#1009)

Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com>

docs/user_guide/diffusion/lora.md

@@ -0,0 +1,63 @@
+# LoRA (Low-Rank Adaptation) Guide
+
+LoRA (Low-Rank Adaptation) enables fine-tuning diffusion models by adding trainable low-rank matrices to existing model weights. vLLM-Omni currently supports PEFT-style LoRA adapters, allowing you to customize model behavior without modifying the base model weights.
+
+## Overview
+
+LoRA adapters are lightweight, model-specific fine-tuning weights that can be dynamically loaded and applied to diffusion models. vLLM-Omni uses a unified LoRA handling mechanism similar to vLLM with LRU cache management.
+
+## LoRA Adapter Format
+
+LoRA adapters must be in **PEFT (Parameter-Efficient Fine-Tuning)** format. A typical LoRA adapter directory structure:
+
+```
+lora_adapter/
+├── adapter_config.json
+└── adapter_model.safetensors
+```
+
+The `adapter_config.json` file contains metadata about the LoRA adapter, including:
+- `r`: LoRA rank
+- `lora_alpha`: LoRA alpha scaling factor
+- `target_modules`: List of module names to apply LoRA to
+
+## Quick Start
+
+### Offline Inference
+
+#### Pre-loaded LoRA
+
+Load a LoRA adapter at initialization. This adapter is pre-loaded into the cache and can be activated by requests:
+
+```python
+from vllm_omni import Omni
+from vllm_omni.lora.request import LoRARequest
+
+lora_path="/path/to/lora_adapter"
+
+omni = Omni(
+    model="stabilityai/stable-diffusion-3.5-medium",
+    lora_path=lora_path
+)
+
+lora_request = LoRARequest(
+    lora_name="preloaded",
+    lora_int_id=1,
+    lora_path=lora_path
+)
+
+outputs = omni.generate(
+    prompt="A piece of cheesecake",
+    lora_request=lora_request,
+    lora_scale=2.0, # optional arg, default 1.0
+)
+```
+
+!!! note "Server-side Path Requirement"
+    The LoRA adapter path (`local_path`) must be readable on the **server** machine. If your client and server are on different machines, ensure the LoRA adapter is accessible via a shared mount or copied to the server.
+
+
+## See Also
+
+- [Text-to-Image Offline Example](../examples/offline_inference/text_to_image.md#lora) - Complete offline LoRA example
+- [Text-to-Image Online Example](../examples/online_serving/text_to_image.md#lora) - Complete online LoRA example