[Doc]Fix - Align with repo. (#1176)
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
This commit is contained in:
1 parent
d6f93b0f37
commit
ce1ae26084
13 files changed
+335
-29
No files matched your search
@@ -1,7 +1,8 @@
|
||||
# LoRA-Inference
|
||||
# Online LoRA Inference (Diffusion)
|
||||
|
||||
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/lora_inference>.
|
||||
|
||||
|
||||
This example shows how to use **per-request LoRA** with vLLM-Omni diffusion models via the OpenAI-compatible Chat Completions API.
|
||||
|
||||
> Note: The LoRA adapter path must be readable on the **server** machine (usually a local path or a mounted directory).
|
||||
@@ -55,15 +56,17 @@ lora_adapter/
|
||||
└── adapter_model.safetensors
|
||||
```
|
||||
|
||||
## Example materials
|
||||
|
||||
??? abstract "openai_chat_client.py"
|
||||
``````py
|
||||
--8<-- "examples/online_serving/lora_inference/openai_chat_client.py"
|
||||
``````
|
||||
??? abstract "run_curl_lora_inference.sh"
|
||||
``````py
|
||||
``````sh
|
||||
--8<-- "examples/online_serving/lora_inference/run_curl_lora_inference.sh"
|
||||
``````
|
||||
??? abstract "run_server.sh"
|
||||
``````py
|
||||
``````sh
|
||||
--8<-- "examples/online_serving/lora_inference/run_server.sh"
|
||||
``````
|
||||
Reference in new issue
Block a user