[Refactor]: Mammothmoda2-dev use x_to_text.py in examples (#5454)

Signed-off-by: princepride <wangzhipeng628@gmail.com>
This commit is contained in:
汪志鹏 authored and GitHub committed 2026-07-27 14:11:49 +00:00
1 parent 8001bb155d
commit e9bc7fc06a
2 files changed
+26 -40

No files matched your search

@@ -11,6 +11,7 @@ Generate text from text or image inputs with vLLM-Omni's shared offline entrypoi
| `ByteDance-Seed/BAGEL-7B-MoT` | Yes | Yes | `vllm_omni/deploy/bagel.yaml` |
| `tencent/HunyuanImage-3.0-Instruct` | Yes | Yes | `vllm_omni/deploy/hunyuan_image3_ar.yaml` |
| `bytedance-research/MammothModa2-Preview` | Yes | Yes | `vllm_omni/deploy/mammoth_moda2_ar.yaml` |
| `bytedance-research/MammothModa2-Dev` | Yes | Yes | `vllm_omni/deploy/mammoth_moda2_ar.yaml` |
The script recognizes these model families from `config.json`, applies the
model-specific prompt format, and selects an AR-only deploy for HunyuanImage-3
@@ -42,6 +43,14 @@ python examples/offline_inference/x_to_text/x_to_text.py \
--prompt "Explain multimodal inference in three concise sentences."
```
The same command supports the Dev checkpoint:
```bash
python examples/offline_inference/x_to_text/x_to_text.py \
--model bytedance-research/MammothModa2-Dev \
--prompt "Explain multimodal inference in three concise sentences."
```
## Image-To-Text
Pass one image with `--image`. The default question can be replaced with any
@@ -70,6 +79,9 @@ python examples/offline_inference/x_to_text/x_to_text.py \
--prompt "Describe this image in detail."
```
Use `bytedance-research/MammothModa2-Dev` in the same command to run I2T with
the Dev checkpoint.
## Key Arguments
| Argument | Default | Description |
+14 -40
View File
@@ -149,49 +149,23 @@ Download the checkpoint:
hf download bytedance-research/MammothModa2-Dev --local-dir ./MammothModa2-Dev
```
Run image-to-text inference from the repository root:
Run text-to-text through the shared understanding example. It recognizes the
Dev checkpoint as MammothModa2 and automatically selects
`mammoth_moda2_ar.yaml`:
```python
from PIL import Image
from vllm import SamplingParams
from vllm.multimodal.image import convert_image_mode
from vllm_omni import Omni
```bash
python examples/offline_inference/x_to_text/x_to_text.py \
--model ./MammothModa2-Dev \
--prompt "Explain multimodal generation in three sentences."
```
prompt = (
"<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n"
"<|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|>"
"Summarize this image.<|im_end|>\n"
"<|im_start|>assistant\n"
)
Add an image for image-to-text or image summarization:
omni = Omni(
model="./MammothModa2-Dev",
deploy_config="vllm_omni/deploy/mammoth_moda2_ar.yaml",
)
try:
outputs = list(
omni.generate(
[{
"prompt": prompt,
"multi_modal_data": {
"image": convert_image_mode(Image.open("./image.png"), "RGB"),
},
"additional_information": {"omni_task": ["chat"]},
}],
[SamplingParams(
temperature=0.2,
top_p=0.9,
top_k=-1,
max_tokens=512,
seed=42,
)],
)
)
finally:
omni.close()
request_output = getattr(outputs[-1], "request_output", outputs[-1])
print(request_output.outputs[0].text.strip())
```bash
python examples/offline_inference/x_to_text/x_to_text.py \
--model ./MammothModa2-Dev \
--image ./image.png \
--prompt "Please summarize the content of this image."
```
The Dev checkpoint is approximately 47.55 GiB on disk. In the verified AR-only