[Bugfix] Fix Compatibility of Ming-flash-omni-2.0 on transformers 5.X and vllm 0.22 (#4080)
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com> Signed-off-by: akshatvishu <akshatnayak197@gmail.com> Co-authored-by: akshatvishu <akshatnayak197@gmail.com>
This commit is contained in:
1 parent
6e8ef213d8
commit
95d56cf062
6 files changed
+97
-26
No files matched your search
@@ -57,12 +57,6 @@ Adjust `devices` in the YAML to match your hardware.
|
||||
|
||||
#### Command
|
||||
|
||||
Thinker only (text output):
|
||||
|
||||
```bash
|
||||
vllm serve Jonathan1909/Ming-flash-omni-2.0 --omni --port 8091
|
||||
```
|
||||
|
||||
Thinker + talker (text and/or audio output):
|
||||
|
||||
```bash
|
||||
@@ -72,6 +66,15 @@ vllm serve Jonathan1909/Ming-flash-omni-2.0 \
|
||||
--log-stats
|
||||
```
|
||||
|
||||
Thinker only (text-only output):
|
||||
|
||||
```bash
|
||||
vllm serve Jonathan1909/Ming-flash-omni-2.0 \
|
||||
--omni \
|
||||
--deploy-config vllm_omni/deploy/ming_flash_omni_thinker_only.yaml \
|
||||
--port 8091
|
||||
```
|
||||
|
||||
`--log-stats` is optional but recommended while validating the deployment.
|
||||
|
||||
#### Verification
|
||||
|
||||
Reference in new issue
Block a user