[Doc][Misc] Update DreamID-Omni Example; Add DreamID-Omni post process function (#2809)

Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
This commit is contained in:
Yuanheng Zhao authored and GitHub committed 2026-04-16 03:26:48 +00:00
1 parent c83f664fe1
commit de5f8a23b2
5 files changed
+96 -14

No files matched your search

@@ -31,9 +31,9 @@ dreamid_omni/
```
### Run the Inference
```
```python
python x_to_video_audio.py \
--model /xx/dreamid_omni \
--model /path/to/dreamid_omni \
--prompt "Two people walking together and singing happily" \
--image-path ./example0.png ./example1.png \
--audio-path ./example0.wav ./example1.wav \
@@ -43,11 +43,33 @@ python x_to_video_audio.py \
--num-inference-steps 45 \
--height 704 \
--width 1280 \
--output dreamid_omni.mp4
--output out_dreamid_omni_twoip.mp4
```
In the current test scenario (2 images + 2 audio inputs), the VRAM requirement is 72GB, regardless of whether cfg-parallel is enabled or disabled.
The VRAM usage can be reduced by enabling CPU offload via --enable-cpu-offload.
You could take reference images/audios from the test cases in the official repo: https://github.com/Guoxu1233/DreamID-Omni
For example, single IP ref resources can be found under https://github.com/Guoxu1233/DreamID-Omni/tree/main/test_case/oneip, you could download them correspondingly to your local and use them for testing.
```python
# Example usage for oneip, ref media from the official repo DreamID-Omni
python x_to_video_audio.py \
--model /path/to/dreamid_omni \
--prompt "<img1>: In the frame, a woman with black long hair is identified as <sub1>.\n**Overall Environment/Scene**: A lively open-kitchen café at night; stove flames flare, steam rises, and warm pendant lights swing slightly as staff move behind her. The shot is an upper-body close-up.\n**Main Characters/Subjects Appearance**: <sub1> is a young woman with thick dark wavy hair and a side part. She wears a fitted black top under a light apron, a thin gold chain necklace, and small stud earrings.\n**Main Characters/Subjects Actions**: <sub1> tastes the sauce with a spoon, then turns her face toward the camera while still holding the spoon, her expression shifting from focused to conflicted.\n<sub1> maintains eye contact, swallows as if choosing her words, and says, <S>I keep telling myself I’m fine,but some nights it feels like I’m just performing calm.<E>" \
--image-path 9.png \
--audio-path 9.wav \
--video-negative-prompt "jitter, bad hands, blur, distortion" \
--audio-negative-prompt "robotic, muffled, echo, distorted" \
--cfg-parallel-size 2 \
--num-inference-steps 45 \
--height 704 \
--width 1280 \
--output out_dreamid_omni_oneip.mp4
```
Key arguments:
- `--prompt`: text description (string).
- `--model`: path to the model local directory.