anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring (#1908)

Browse Source 
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Signed-off-by: Peiqi Yin <yinpeiqi809@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: Peiqi Yin <yinpeiqi809@gmail.com>
Co-authored-by: wuhang <whlbx@hotmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
This commit is contained in:
Chenguang ZhengyinpePeiqi YinwuhangPeiqi Yinlinyueqianprincepridelishunyang
 authored and GitHub committed 2026-03-18 17:33:53 +08:00
1 parent 6a66a1fe36
commit 3158e9d28f
125 files changed
+5210 -9322
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/configuration/stage_configs.md
+2 -2
View File
@@ -18,7 +18,7 @@ If users want to modify some part of it. The custom stage_configs file can be in
For offline (Assume necessary dependencies have ben imported):
```python
model_name = "Qwen/Qwen2.5-Omni-7B"
omni_llm = Omni(model=model_name, stage_configs_path="/path/to/custom_stage_configs.yaml")
omni = Omni(model=model_name, stage_configs_path="/path/to/custom_stage_configs.yaml")
```
For online serving:
@@ -30,7 +30,7 @@ vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091 --stage-configs-path /path/to
Below is a specific example of stage_configs.yaml in Qwen2.5-omni.
```python
# stage config for running qwen2.5-omni with architecture of OmniLLM.
# stage config for running qwen2.5-omni with AsyncOmniEngine + Orchestrator runtime.
stage_args:
- stage_id: 0 # mark the unique id for each stage
runtime: # The disaggregated configuration
docs/configuration/stage_configs/qwen2_5_omni.yaml
+1 -1
View File
@@ -1,4 +1,4 @@
# stage config for running qwen2.5-omni with architecture of OmniLLM.
# stage config for running qwen2.5-omni with AsyncOmniEngine + Orchestrator runtime.
stage_args:
- stage_id: 0
runtime: