[Feature] feat: add video frame interpolation postprocess (#2555)
Signed-off-by: David Chen <530634352@qq.com>
This commit is contained in:
1 parent
61a3cbdff5
commit
6c6551dff8
17 files changed
+961
-9
No files matched your search
@@ -14,7 +14,7 @@ vLLM-Omni supports various advanced features for diffusion models:
|
||||
|
||||
- Acceleration: **cache methods**, **parallelism methods**, **startup optimizations**
|
||||
- Memory optimization: **cpu offloading**, **quantization**
|
||||
- Extensions: **LoRA inference**
|
||||
- Extensions: **LoRA inference**, **frame interpolation**
|
||||
- Execution modes: **step execution**
|
||||
|
||||
## Supported Features
|
||||
@@ -69,6 +69,7 @@ Extension methods add specialized capabilities to diffusion models beyond standa
|
||||
| Method | Description | Best For |
|
||||
|--------|-------------|----------|
|
||||
| **[LoRA Inference](diffusion/lora.md)** | Enables inference with Low-Rank Adaptation (LoRA) adapters weights | Reinforcement learning extensions |
|
||||
| **[Frame Interpolation](diffusion/frame_interpolation.md)** | Inserts intermediate video frames after generation for smoother motion | Video generation pipelines that need higher temporal smoothness |
|
||||
|
||||
|
||||
### Execution Modes
|
||||
@@ -143,6 +144,11 @@ The following tables show which models support each feature:
|
||||
| **HunyuanVideo-1.5 T2V I2V** | ❌ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ (decode) | ✅ | ❌ |
|
||||
| **DreamID-Omni** | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
|
||||
|
||||
**Frame Interpolation Support**
|
||||
|
||||
- **Supported**: Wan2.2 text-to-video, image-to-video, and TI2V pipelines
|
||||
- **Not supported**: Wan2.1-VACE, LTX-2, Helios, HunyuanVideo-1.5, DreamID-Omni
|
||||
|
||||
### AudioGen
|
||||
|
||||
| Model | ⚡TeaCache | ⚡Cache-DiT | 🔀SP (Ulysses & Ring) | 🔀CFG-Parallel | 🔀Tensor-Parallel | 🔀HSDP | 💾CPU Offload (Layerwise) | 💾VAE-Patch-Parallel | 💾Quantization | 🔄Step Execution |
|
||||
@@ -258,6 +264,7 @@ Measured on NVIDIA H800:
|
||||
**Extensions:**
|
||||
|
||||
- **[LoRA Inference Guide](diffusion/lora.md)** - Low-Rank Adaptation for style customization and fine-tuning
|
||||
- **[Frame Interpolation Guide](diffusion/frame_interpolation.md)** - Worker-side post-generation video frame interpolation for smoother motion
|
||||
|
||||
**Execution Modes:**
|
||||
|
||||
|
||||
Reference in new issue
Block a user