[Docs] Add recipe for Helios (#3114)
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn>
This commit is contained in:
1 parent
900b4187a1
commit
5dcf18a984
2 files changed
+111
-3
No files matched your search
@@ -0,0 +1,109 @@
|
||||
# Helios for video generation
|
||||
|
||||
## Summary
|
||||
|
||||
- Vendor: Helios
|
||||
- Model: `BestWishYsh/Helios-Base`, `BestWishYsh/Helios-Mid`, `BestWishYsh/Helios-Distilled`
|
||||
- Task: Text-to-video, image-to-video, and video-to-video generation
|
||||
- Mode: Offline inference with the bundled Helios example
|
||||
- Maintainer: Community
|
||||
|
||||
## When to use this recipe
|
||||
|
||||
Use this recipe when you want a known-good starting point for running Helios
|
||||
video generation with vLLM-Omni. The concrete command below focuses on
|
||||
`BestWishYsh/Helios-Base` text-to-video generation on one NVIDIA H20 GPU. The
|
||||
same example script also supports Helios-Mid, Helios-Distilled, image-to-video,
|
||||
and video-to-video; see the linked example docs for those model-specific flags.
|
||||
|
||||
## References
|
||||
|
||||
- Upstream repository: <https://github.com/PKU-YuanGroup/Helios>
|
||||
- Model weights:
|
||||
<https://huggingface.co/BestWishYsh/Helios-Base>
|
||||
- Upstream or canonical docs:
|
||||
[`docs/user_guide/examples/offline_inference/helios.md`](../../docs/user_guide/examples/offline_inference/helios.md)
|
||||
- Related example under `examples/`:
|
||||
[`examples/offline_inference/helios/README.md`](../../examples/offline_inference/helios/README.md)
|
||||
|
||||
## Hardware Support
|
||||
|
||||
## GPU
|
||||
|
||||
### 1x NVIDIA H20
|
||||
|
||||
#### Environment
|
||||
|
||||
- OS: Ubuntu Linux x86_64
|
||||
- Python: 3.12.12
|
||||
- Driver / runtime: NVIDIA driver `580.126.20`, CUDA `13.0`
|
||||
- Hardware: 1x NVIDIA H20 GPU from an 8x H20 host
|
||||
- vLLM version: `0.19.0`
|
||||
- vLLM-Omni version or commit: `a3903810`
|
||||
|
||||
#### Command
|
||||
|
||||
Run the baseline Helios-Base text-to-video example from the repository root:
|
||||
|
||||
```bash
|
||||
cd examples/offline_inference/helios
|
||||
|
||||
python end2end.py \
|
||||
--model BestWishYsh/Helios-Base \
|
||||
--sample-type t2v \
|
||||
--prompt "A dynamic time-lapse video showing the rapidly moving scenery from the window of a speeding train." \
|
||||
--guidance-scale 5.0 \
|
||||
--output helios_t2v_base.mp4
|
||||
```
|
||||
|
||||
To use cache-dit acceleration, run on a vLLM-Omni checkout that includes Helios cache-dit support and enable the cache backend:
|
||||
|
||||
```bash
|
||||
cd examples/offline_inference/helios
|
||||
|
||||
python end2end.py \
|
||||
--cache-backend cache_dit \
|
||||
--enable-cache-dit-summary \
|
||||
--model BestWishYsh/Helios-Base \
|
||||
--sample-type t2v \
|
||||
--prompt "A dynamic time-lapse video showing the rapidly moving scenery from the window of a speeding train." \
|
||||
--guidance-scale 5.0 \
|
||||
--output helios_t2v_base.mp4
|
||||
```
|
||||
|
||||
#### Verification
|
||||
|
||||
The script should print the resolved generation configuration, total generation
|
||||
time, and output path. A successful run ends with output similar to:
|
||||
|
||||
```text
|
||||
Total generation time: <seconds> seconds (<milliseconds> ms)
|
||||
Saved generated video to helios_t2v_base.mp4
|
||||
```
|
||||
|
||||
#### Important flags and stage configs
|
||||
|
||||
- `--model BestWishYsh/Helios-Base` selects the base Helios checkpoint used by
|
||||
this recipe.
|
||||
- `--sample-type t2v` selects text-to-video generation. Use `i2v` with
|
||||
`--image-path` for image-to-video, or `v2v` with `--video-path` for
|
||||
video-to-video.
|
||||
- `--guidance-scale 5.0` matches the Helios-Base recommendation. Use
|
||||
`--guidance-scale 1.0` for Helios-Distilled.
|
||||
- `--cache-backend cache_dit` enables the cache-dit acceleration path.
|
||||
- `--enable-cache-dit-summary` prints cache-dit summary information after
|
||||
diffusion forward passes.
|
||||
- No separate stage config is required for this offline recipe; the bundled
|
||||
Helios example configures the pipeline through `end2end.py` arguments.
|
||||
- For Helios-Mid, add `--is-enable-stage2`,
|
||||
`--pyramid-num-inference-steps-list 20 20 20`, `--use-cfg-zero-star`,
|
||||
`--use-zero-init`, and `--zero-steps 1`.
|
||||
- For Helios-Distilled, add `--is-enable-stage2`,
|
||||
`--pyramid-num-inference-steps-list 2 2 2`, and
|
||||
`--is-amplify-first-chunk`.
|
||||
|
||||
#### Known limitations
|
||||
|
||||
- Helios generates video in 33-frame chunks. For best performance, set
|
||||
`--num-frames` to a multiple of `33`; non-multiple values are rounded up to
|
||||
the nearest multiple of `33`.
|
||||
+2
-3
@@ -30,6 +30,7 @@ recipes/
|
||||
| [`audiox/AudioX.md`](./audiox/AudioX.md) | Offline + online unified text/video→audio diffusion | 1x L4 24GB |
|
||||
| [`Baidu/ERNIE-Image.md`](./Baidu/ERNIE-Image.md) | Text-to-image online serving (ERNIE-Image 8B) | 1x or 2x RTX 4090 24GB |
|
||||
| [`fishaudio/Fish-Speech-S2-Pro.md`](./fishaudio/Fish-Speech-S2-Pro.md) | Online serving for TTS | 1x A800 80GB |
|
||||
| [`Helios/Helios.md`](./Helios/Helios.md) | Text-to-video, image-to-video, and video-to-video generation | 1x NVIDIA H20 |
|
||||
| [`inclusionAI/Ming-flash-omni-2.0.md`](./inclusionAI/Ming-flash-omni-2.0.md) | Online serving for multimodal chat + standalone TTS | 4x H100 / 1x H100 80GB |
|
||||
| [`LTX/LTX-2.md`](./LTX/LTX-2.md) | Text-to-video and image-to-video serving | 1x H200 141GB |
|
||||
| [`LTX/LTX-2.3.md`](./LTX/LTX-2.3.md) | Text-to-video with audio generation (22B) | 1x GPU (96GB VRAM) |
|
||||
@@ -44,9 +45,7 @@ recipes/
|
||||
| [`Wan-AI/Wan2.2-I2V.md`](./Wan-AI/Wan2.2-I2V.md) | Image-to-video serving (Wan2.2 14B) | 8x Ascend NPU (A2/A3) |
|
||||
| [`Wan-AI/Wan2.2-S2V.md`](./Wan-AI/Wan2.2-S2V.md) | Speech-to-video serving (Wan2.2 14B) | 2x A100/H100 80GB |
|
||||
| [`zai-org/GLM-TTS.md`](./zai-org/GLM-TTS.md) | Online serving for Chinese/English zero-shot voice-cloned TTS | 1x A40 48GB |
|
||||
|
||||
- [`GLM/GLM-Image.md`](./GLM/GLM-Image.md):online serving recipe for
|
||||
image generation on `1x A800 80GB` and `2x A800 80GB`
|
||||
| [`GLM/GLM-Image.md`](./GLM/GLM-Image.md) | Online serving for image generation | 1x A800 80GB / 2x A800 80GB |
|
||||
|
||||
Within a single recipe file, include different hardware support sections such
|
||||
as `GPU`, `ROCm`, and `NPU`, and add concrete tested configurations like
|
||||
|
||||
Reference in new issue
Block a user