From 5dcf18a984b0534cdd7c8db70b19e904774a31eb Mon Sep 17 00:00:00 2001 From: Jason <72191212+JasonJ2021@users.noreply.github.com> Date: Fri, 29 May 2026 09:34:37 +0800 Subject: [PATCH] [Docs] Add recipe for Helios (#3114) Signed-off-by: JasonJ2021 --- recipes/Helios/Helios.md | 109 +++++++++++++++++++++++++++++++++++++++ recipes/README.md | 5 +- 2 files changed, 111 insertions(+), 3 deletions(-) create mode 100644 recipes/Helios/Helios.md diff --git a/recipes/Helios/Helios.md b/recipes/Helios/Helios.md new file mode 100644 index 000000000..ebc446b74 --- /dev/null +++ b/recipes/Helios/Helios.md @@ -0,0 +1,109 @@ +# Helios for video generation + +## Summary + +- Vendor: Helios +- Model: `BestWishYsh/Helios-Base`, `BestWishYsh/Helios-Mid`, `BestWishYsh/Helios-Distilled` +- Task: Text-to-video, image-to-video, and video-to-video generation +- Mode: Offline inference with the bundled Helios example +- Maintainer: Community + +## When to use this recipe + +Use this recipe when you want a known-good starting point for running Helios +video generation with vLLM-Omni. The concrete command below focuses on +`BestWishYsh/Helios-Base` text-to-video generation on one NVIDIA H20 GPU. The +same example script also supports Helios-Mid, Helios-Distilled, image-to-video, +and video-to-video; see the linked example docs for those model-specific flags. + +## References + +- Upstream repository: +- Model weights: + +- Upstream or canonical docs: + [`docs/user_guide/examples/offline_inference/helios.md`](../../docs/user_guide/examples/offline_inference/helios.md) +- Related example under `examples/`: + [`examples/offline_inference/helios/README.md`](../../examples/offline_inference/helios/README.md) + +## Hardware Support + +## GPU + +### 1x NVIDIA H20 + +#### Environment + +- OS: Ubuntu Linux x86_64 +- Python: 3.12.12 +- Driver / runtime: NVIDIA driver `580.126.20`, CUDA `13.0` +- Hardware: 1x NVIDIA H20 GPU from an 8x H20 host +- vLLM version: `0.19.0` +- vLLM-Omni version or commit: `a3903810` + +#### Command + +Run the baseline Helios-Base text-to-video example from the repository root: + +```bash +cd examples/offline_inference/helios + +python end2end.py \ + --model BestWishYsh/Helios-Base \ + --sample-type t2v \ + --prompt "A dynamic time-lapse video showing the rapidly moving scenery from the window of a speeding train." \ + --guidance-scale 5.0 \ + --output helios_t2v_base.mp4 +``` + +To use cache-dit acceleration, run on a vLLM-Omni checkout that includes Helios cache-dit support and enable the cache backend: + +```bash +cd examples/offline_inference/helios + +python end2end.py \ + --cache-backend cache_dit \ + --enable-cache-dit-summary \ + --model BestWishYsh/Helios-Base \ + --sample-type t2v \ + --prompt "A dynamic time-lapse video showing the rapidly moving scenery from the window of a speeding train." \ + --guidance-scale 5.0 \ + --output helios_t2v_base.mp4 +``` + +#### Verification + +The script should print the resolved generation configuration, total generation +time, and output path. A successful run ends with output similar to: + +```text +Total generation time: seconds ( ms) +Saved generated video to helios_t2v_base.mp4 +``` + +#### Important flags and stage configs + +- `--model BestWishYsh/Helios-Base` selects the base Helios checkpoint used by + this recipe. +- `--sample-type t2v` selects text-to-video generation. Use `i2v` with + `--image-path` for image-to-video, or `v2v` with `--video-path` for + video-to-video. +- `--guidance-scale 5.0` matches the Helios-Base recommendation. Use + `--guidance-scale 1.0` for Helios-Distilled. +- `--cache-backend cache_dit` enables the cache-dit acceleration path. +- `--enable-cache-dit-summary` prints cache-dit summary information after + diffusion forward passes. +- No separate stage config is required for this offline recipe; the bundled + Helios example configures the pipeline through `end2end.py` arguments. +- For Helios-Mid, add `--is-enable-stage2`, + `--pyramid-num-inference-steps-list 20 20 20`, `--use-cfg-zero-star`, + `--use-zero-init`, and `--zero-steps 1`. +- For Helios-Distilled, add `--is-enable-stage2`, + `--pyramid-num-inference-steps-list 2 2 2`, and + `--is-amplify-first-chunk`. + +#### Known limitations + +- Helios generates video in 33-frame chunks. For best performance, set + `--num-frames` to a multiple of `33`; non-multiple values are rounded up to + the nearest multiple of `33`. diff --git a/recipes/README.md b/recipes/README.md index 13204453e..ef11edeb3 100644 --- a/recipes/README.md +++ b/recipes/README.md @@ -30,6 +30,7 @@ recipes/ | [`audiox/AudioX.md`](./audiox/AudioX.md) | Offline + online unified text/video→audio diffusion | 1x L4 24GB | | [`Baidu/ERNIE-Image.md`](./Baidu/ERNIE-Image.md) | Text-to-image online serving (ERNIE-Image 8B) | 1x or 2x RTX 4090 24GB | | [`fishaudio/Fish-Speech-S2-Pro.md`](./fishaudio/Fish-Speech-S2-Pro.md) | Online serving for TTS | 1x A800 80GB | +| [`Helios/Helios.md`](./Helios/Helios.md) | Text-to-video, image-to-video, and video-to-video generation | 1x NVIDIA H20 | | [`inclusionAI/Ming-flash-omni-2.0.md`](./inclusionAI/Ming-flash-omni-2.0.md) | Online serving for multimodal chat + standalone TTS | 4x H100 / 1x H100 80GB | | [`LTX/LTX-2.md`](./LTX/LTX-2.md) | Text-to-video and image-to-video serving | 1x H200 141GB | | [`LTX/LTX-2.3.md`](./LTX/LTX-2.3.md) | Text-to-video with audio generation (22B) | 1x GPU (96GB VRAM) | @@ -44,9 +45,7 @@ recipes/ | [`Wan-AI/Wan2.2-I2V.md`](./Wan-AI/Wan2.2-I2V.md) | Image-to-video serving (Wan2.2 14B) | 8x Ascend NPU (A2/A3) | | [`Wan-AI/Wan2.2-S2V.md`](./Wan-AI/Wan2.2-S2V.md) | Speech-to-video serving (Wan2.2 14B) | 2x A100/H100 80GB | | [`zai-org/GLM-TTS.md`](./zai-org/GLM-TTS.md) | Online serving for Chinese/English zero-shot voice-cloned TTS | 1x A40 48GB | - -- [`GLM/GLM-Image.md`](./GLM/GLM-Image.md):online serving recipe for - image generation on `1x A800 80GB` and `2x A800 80GB` +| [`GLM/GLM-Image.md`](./GLM/GLM-Image.md) | Online serving for image generation | 1x A800 80GB / 2x A800 80GB | Within a single recipe file, include different hardware support sections such as `GPU`, `ROCm`, and `NPU`, and add concrete tested configurations like