From e9bc7fc06a39eb7abc57e2b984497ab493df0b4a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E6=B1=AA=E5=BF=97=E9=B9=8F?= Date: Mon, 27 Jul 2026 22:11:49 +0800 Subject: [PATCH] [Refactor]: Mammothmoda2-dev use x_to_text.py in examples (#5454) Signed-off-by: princepride --- .../offline_inference/x_to_text/README.md | 12 +++++ recipes/MammothModa2/MammothModa2.md | 54 +++++-------------- 2 files changed, 26 insertions(+), 40 deletions(-) diff --git a/examples/offline_inference/x_to_text/README.md b/examples/offline_inference/x_to_text/README.md index fb3f7ae14..de668b6ee 100644 --- a/examples/offline_inference/x_to_text/README.md +++ b/examples/offline_inference/x_to_text/README.md @@ -11,6 +11,7 @@ Generate text from text or image inputs with vLLM-Omni's shared offline entrypoi | `ByteDance-Seed/BAGEL-7B-MoT` | Yes | Yes | `vllm_omni/deploy/bagel.yaml` | | `tencent/HunyuanImage-3.0-Instruct` | Yes | Yes | `vllm_omni/deploy/hunyuan_image3_ar.yaml` | | `bytedance-research/MammothModa2-Preview` | Yes | Yes | `vllm_omni/deploy/mammoth_moda2_ar.yaml` | +| `bytedance-research/MammothModa2-Dev` | Yes | Yes | `vllm_omni/deploy/mammoth_moda2_ar.yaml` | The script recognizes these model families from `config.json`, applies the model-specific prompt format, and selects an AR-only deploy for HunyuanImage-3 @@ -42,6 +43,14 @@ python examples/offline_inference/x_to_text/x_to_text.py \ --prompt "Explain multimodal inference in three concise sentences." ``` +The same command supports the Dev checkpoint: + +```bash +python examples/offline_inference/x_to_text/x_to_text.py \ + --model bytedance-research/MammothModa2-Dev \ + --prompt "Explain multimodal inference in three concise sentences." +``` + ## Image-To-Text Pass one image with `--image`. The default question can be replaced with any @@ -70,6 +79,9 @@ python examples/offline_inference/x_to_text/x_to_text.py \ --prompt "Describe this image in detail." ``` +Use `bytedance-research/MammothModa2-Dev` in the same command to run I2T with +the Dev checkpoint. + ## Key Arguments | Argument | Default | Description | diff --git a/recipes/MammothModa2/MammothModa2.md b/recipes/MammothModa2/MammothModa2.md index 60c391dec..7056a6f8f 100644 --- a/recipes/MammothModa2/MammothModa2.md +++ b/recipes/MammothModa2/MammothModa2.md @@ -149,49 +149,23 @@ Download the checkpoint: hf download bytedance-research/MammothModa2-Dev --local-dir ./MammothModa2-Dev ``` -Run image-to-text inference from the repository root: +Run text-to-text through the shared understanding example. It recognizes the +Dev checkpoint as MammothModa2 and automatically selects +`mammoth_moda2_ar.yaml`: -```python -from PIL import Image -from vllm import SamplingParams -from vllm.multimodal.image import convert_image_mode -from vllm_omni import Omni +```bash +python examples/offline_inference/x_to_text/x_to_text.py \ + --model ./MammothModa2-Dev \ + --prompt "Explain multimodal generation in three sentences." +``` -prompt = ( - "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n" - "<|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|>" - "Summarize this image.<|im_end|>\n" - "<|im_start|>assistant\n" -) +Add an image for image-to-text or image summarization: -omni = Omni( - model="./MammothModa2-Dev", - deploy_config="vllm_omni/deploy/mammoth_moda2_ar.yaml", -) -try: - outputs = list( - omni.generate( - [{ - "prompt": prompt, - "multi_modal_data": { - "image": convert_image_mode(Image.open("./image.png"), "RGB"), - }, - "additional_information": {"omni_task": ["chat"]}, - }], - [SamplingParams( - temperature=0.2, - top_p=0.9, - top_k=-1, - max_tokens=512, - seed=42, - )], - ) - ) -finally: - omni.close() - -request_output = getattr(outputs[-1], "request_output", outputs[-1]) -print(request_output.outputs[0].text.strip()) +```bash +python examples/offline_inference/x_to_text/x_to_text.py \ + --model ./MammothModa2-Dev \ + --image ./image.png \ + --prompt "Please summarize the content of this image." ``` The Dev checkpoint is approximately 47.55 GiB on disk. In the verified AR-only