# HunyuanImage-3.0-Instruct

## Set up

Please refer to the [stage configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation appropriately for your hardware setup.

## Run examples

**Note**: These examples work with the default configuration on **8x NVIDIA L40S (48GB)**. For different GPU setups, modify the stage configuration to adjust device allocation and memory utilization.

Get into the hunyuan_image3 folder:

```bash
cd examples/offline_inference/hunyuan_image3
```

### Modality Control

HunyuanImage-3.0-Instruct supports multiple modality modes. You can control the mode using the `--modality` argument:

#### Text to Image (text2img)

- **Pipeline**: Text → AR (CoT + latent tokens) → DiT (denoise) → VAE Decode → Image
- **Stages Used**: Stage 0 (AR) + Stage 1 (DiT)
- **KV Transfer**: AR sends KV cache to DiT for conditioned generation
- **Default Config**: `hunyuan_image3_t2i.yaml`

```bash
python end2end.py --model tencent/HunyuanImage-3.0-Instruct \
                  --modality text2img \
                  --prompts "A cute cat sitting on a windowsill watching the sunset"
```

#### Image to Image (img2img)

- **Pipeline**: Image + Text → AR (CoT + recaption + latent) → DiT → Edited Image
- **Stages Used**: Stage 0 (AR) + Stage 1 (DiT)
- **KV Transfer**: AR sends KV cache to DiT
- **Default Config**: `hunyuan_image3_it2i.yaml`

```bash
python end2end.py --model tencent/HunyuanImage-3.0-Instruct \
                  --modality img2img \
                  --image-path /path/to/image.png \
                  --prompts "Make the petals neon pink"
```

#### Image to Text (img2text)

- **Pipeline**: Image + Question → AR → Text description
- **Stages Used**: Stage 0 (AR) only
- **Default Config**: `hunyuan_image3_i2t.yaml`

```bash
python end2end.py --model tencent/HunyuanImage-3.0-Instruct \
                  --modality img2text \
                  --image-path /path/to/image.jpg \
                  --prompts "Describe the content of the picture."
```

#### Text to Text (text2text)

- **Pipeline**: Text → AR → Text
- **Stages Used**: Stage 0 (AR) only
- **Default Config**: `hunyuan_image3_t2t.yaml`

```bash
python end2end.py --model tencent/HunyuanImage-3.0-Instruct \
                  --modality text2text \
                  --prompts "What is the capital of France?"
```

### Inference Steps & Guidance

Control generation quality for image modalities:

```bash
python end2end.py --modality text2img \
                  --steps 50 \
                  --guidance-scale 5.0 \
                  --height 1024 --width 1024 \
                  --prompts "A photo-realistic sunset over the ocean"
```

### Key Arguments

#### 📌 Command Line Arguments (end2end.py)

| Argument               | Type   | Default                              | Description                                                  |
| :--------------------- | :----- | :----------------------------------- | :----------------------------------------------------------- |
| `--model`              | string | `tencent/HunyuanImage-3.0-Instruct` | Model path or name                                           |
| `--modality`           | choice | `text2img`                           | Modality: `text2img`, `img2img`, `img2text`, `text2text`     |
| `--prompts`            | list   | `None`                               | Input text prompts                                           |
| `--image-path`         | string | `None`                               | Input image path (for `img2img`/`img2text`)                  |
| `--output`             | string | `.`                                  | Output directory for saved images                            |
| `--steps`              | int    | `50`                                 | Number of inference steps                                    |
| `--guidance-scale`     | float  | `5.0`                                | Classifier-free guidance scale                               |
| `--seed`               | int    | `42`                                 | Random seed                                                  |
| `--height`             | int    | `1024`                               | Output image height                                          |
| `--width`              | int    | `1024`                               | Output image width                                           |
| `--bot-task`           | string | auto                                 | Override prompt task (e.g. `it2i_think`, `t2i_recaption`)    |
| `--sys-type`           | string | auto                                 | Override system prompt type (e.g. `en_unified`, `en_vanilla`) |
| `--stage-configs-path` | string | auto                                 | Custom stage config YAML path                                |
| `--enforce-eager`      | flag   | `False`                              | Disable torch.compile                                        |
| `--init-timeout`       | int    | `300`                                | Initialization timeout (seconds)                             |

------

#### ⚙️ Stage Configurations

| Config YAML                         | Modality  | Stages | GPUs   | Description                           |
| :---------------------------------- | :-------- | :----- | :----- | :------------------------------------ |
| `hunyuan_image3_t2i.yaml`           | text2img  | 2      | 8      | T2I with AR→DiT, 4 GPU each          |
| `hunyuan_image3_it2i.yaml`          | img2img   | 2      | 8      | IT2I with AR→DiT, 4 GPU each         |
| `hunyuan_image3_i2t.yaml`           | img2text  | 1      | 4      | I2T (AR only)                         |
| `hunyuan_image3_t2t.yaml`           | text2text | 1      | 4      | T2T (AR only)                         |
| `hunyuan_image3_t2i_2gpu.yaml`      | text2img  | 2      | 2      | T2I for 2-GPU setups                  |
| `hunyuan_image3_moe.yaml`           | text2img  | 2      | 8      | T2I with MoE AR→DiT KV reuse          |
| `hunyuan_image3_moe_dit_2gpu_fp8.yaml` | text2img | 2   | 2      | T2I with FP8 quantization             |

------

## Using MoE Config

The `hunyuan_image3_moe.yaml` config enables AR→DiT KV cache reuse with 8 GPUs (4 for AR + 4 for DiT).

```bash
python end2end.py --model tencent/HunyuanImage-3.0-Instruct \
                  --modality text2img \
                  --stage-configs-path hunyuan_image3_moe.yaml \
                  --prompts "A cute cat"
```

------

## Prompt Format

HunyuanImage-3.0 uses a pretrain template format:

```
<|startoftext|>{system_prompt}{<img>}{trigger_tag}{user_prompt}
```

- `<img>`: Placeholder for each input image (auto-inserted by `prompt_utils.py`)
- Trigger tags: `<think>` (CoT), `<recaption>` (recaptioning)
- System prompt: Auto-selected based on task

The `prompt_utils.build_prompt()` handles this formatting automatically.

------

## FAQ

- **OOM errors**: Decrease `gpu_memory_utilization` in the YAML stage config, or use a smaller `max_num_batched_tokens`.
- **Custom image sizes**: Use `--height` and `--width` flags (multiples of 16 recommended).

| Stage             | VRAM (approx)        |
| :---------------- | :------------------- |
| Stage 0 (AR)      | ~15 GiB + KV Cache   |
| Stage 1 (DiT)     | ~30 GiB              |
| Total (8-GPU)     | ~45 GiB + KV Cache   |