[Feature] feat: add video frame interpolation postprocess (#2555)

Signed-off-by: David Chen <530634352@qq.com>
This commit is contained in:
WeiQing Chen authored and GitHub committed 2026-04-15 11:52:20 +08:00
1 parent 61a3cbdff5
commit 6c6551dff8
17 files changed
+961 -9

No files matched your search

@@ -72,6 +72,9 @@ curl -X POST http://localhost:8091/v1/videos/sync \
-F "guidance_scale_2=1.0" \
-F "boundary_ratio=0.875" \
-F "flow_shift=12.0" \
-F "enable_frame_interpolation=true" \
-F "frame_interpolation_exp=1" \
-F "frame_interpolation_scale=1.0" \
-F "seed=42" \
-o sync_i2v_output.mp4
```
@@ -114,6 +117,9 @@ create_response=$(curl -s http://localhost:8091/v1/videos \
-F "guidance_scale_2=1.0" \
-F "boundary_ratio=0.875" \
-F "flow_shift=12.0" \
-F "enable_frame_interpolation=true" \
-F "frame_interpolation_exp=1" \
-F "frame_interpolation_scale=1.0" \
-F "seed=42")
video_id=$(echo "$create_response" | jq -r '.id')
@@ -172,9 +178,35 @@ curl -X POST http://localhost:8091/v1/videos \
-F "guidance_scale_2=1.0" \
-F "boundary_ratio=0.875" \
-F "flow_shift=12.0" \
-F "enable_frame_interpolation=true" \
-F "frame_interpolation_exp=1" \
-F "frame_interpolation_scale=1.0" \
-F "seed=42"
```
Frame interpolation is also available for supported Wan2.2 I2V requests. See
[Frame Interpolation](../../diffusion/frame_interpolation.md) for worker-side
execution details and feature constraints.
### Frame Interpolation Example
```bash
curl -X POST http://localhost:8091/v1/videos/sync \
-F "prompt=A bear playing with yarn, smooth motion" \
-F "input_reference=@/path/to/qwen-bear.png" \
-F "width=832" \
-F "height=480" \
-F "num_frames=33" \
-F "fps=16" \
-F "num_inference_steps=40" \
-F "guidance_scale=1.0" \
-F "guidance_scale_2=1.0" \
-F "enable_frame_interpolation=true" \
-F "frame_interpolation_exp=1" \
-F "frame_interpolation_scale=1.0" \
-o sync_i2v_interpolated.mp4
```
## Create Response Format
`POST /v1/videos` returns a job record, not inline base64 video data.
@@ -165,6 +165,9 @@ curl -X POST http://localhost:8091/v1/videos \
-F "guidance_scale_2=4.0" \
-F "boundary_ratio=0.875" \
-F "flow_shift=5.0" \
-F "enable_frame_interpolation=true" \
-F "frame_interpolation_exp=1" \
-F "frame_interpolation_scale=1.0" \
-F "seed=42"
```
@@ -187,6 +190,35 @@ curl -X POST http://localhost:8091/v1/videos \
| `flow_shift` | float | None | Scheduler flow shift (Wan2.2) |
| `seed` | int | None | Random seed (reproducible) |
| `lora` | object | None | LoRA configuration |
| `enable_frame_interpolation` | bool | false | Enable RIFE frame interpolation before MP4 encoding |
| `frame_interpolation_exp` | int | 1 | Interpolation exponent; 1=2x temporal resolution, 2=4x |
| `frame_interpolation_scale` | float | 1.0 | RIFE inference scale; use 0.5 for high-resolution inputs |
| `frame_interpolation_model_path` | str | None | Local directory or Hugging Face repo ID with `flownet.pkl`; defaults to `elfgum/RIFE-4.22.lite` |
## Frame Interpolation
Frame interpolation is an optional post-processing step for `/v1/videos` and
`/v1/videos/sync`. It synthesizes intermediate frames between generated frames
without rerunning the diffusion model. If the generated video has `N` frames,
the interpolated output frame count is `(N - 1) * 2**exp + 1`. The encoder FPS
is multiplied by `2**exp` so the output duration remains close to the original.
Frame interpolation runs in the diffusion worker post-processing path instead of
the API server encoding path, so it can reuse the worker's current accelerator
device without blocking the FastAPI event loop.
Example: generate 5 frames and interpolate to 9 frames:
```bash
curl -X POST http://localhost:8091/v1/videos/sync \
-F "prompt=A dog running through a park" \
-F "num_frames=5" \
-F "fps=8" \
-F "enable_frame_interpolation=true" \
-F "frame_interpolation_exp=1" \
-F "frame_interpolation_scale=1.0" \
-o sync_t2v_interpolated.mp4
```
## Create Response Format