[Feature] feat: add video frame interpolation postprocess (#2555)
Signed-off-by: David Chen <530634352@qq.com>
This commit is contained in:
1 parent
61a3cbdff5
commit
6c6551dff8
17 files changed
+961
-9
No files matched your search
@@ -72,6 +72,9 @@ curl -X POST http://localhost:8091/v1/videos/sync \
|
||||
-F "guidance_scale_2=1.0" \
|
||||
-F "boundary_ratio=0.875" \
|
||||
-F "flow_shift=12.0" \
|
||||
-F "enable_frame_interpolation=true" \
|
||||
-F "frame_interpolation_exp=1" \
|
||||
-F "frame_interpolation_scale=1.0" \
|
||||
-F "seed=42" \
|
||||
-o sync_i2v_output.mp4
|
||||
```
|
||||
@@ -114,6 +117,9 @@ create_response=$(curl -s http://localhost:8091/v1/videos \
|
||||
-F "guidance_scale_2=1.0" \
|
||||
-F "boundary_ratio=0.875" \
|
||||
-F "flow_shift=12.0" \
|
||||
-F "enable_frame_interpolation=true" \
|
||||
-F "frame_interpolation_exp=1" \
|
||||
-F "frame_interpolation_scale=1.0" \
|
||||
-F "seed=42")
|
||||
|
||||
video_id=$(echo "$create_response" | jq -r '.id')
|
||||
@@ -172,9 +178,35 @@ curl -X POST http://localhost:8091/v1/videos \
|
||||
-F "guidance_scale_2=1.0" \
|
||||
-F "boundary_ratio=0.875" \
|
||||
-F "flow_shift=12.0" \
|
||||
-F "enable_frame_interpolation=true" \
|
||||
-F "frame_interpolation_exp=1" \
|
||||
-F "frame_interpolation_scale=1.0" \
|
||||
-F "seed=42"
|
||||
```
|
||||
|
||||
Frame interpolation is also available for supported Wan2.2 I2V requests. See
|
||||
[Frame Interpolation](../../diffusion/frame_interpolation.md) for worker-side
|
||||
execution details and feature constraints.
|
||||
|
||||
### Frame Interpolation Example
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8091/v1/videos/sync \
|
||||
-F "prompt=A bear playing with yarn, smooth motion" \
|
||||
-F "input_reference=@/path/to/qwen-bear.png" \
|
||||
-F "width=832" \
|
||||
-F "height=480" \
|
||||
-F "num_frames=33" \
|
||||
-F "fps=16" \
|
||||
-F "num_inference_steps=40" \
|
||||
-F "guidance_scale=1.0" \
|
||||
-F "guidance_scale_2=1.0" \
|
||||
-F "enable_frame_interpolation=true" \
|
||||
-F "frame_interpolation_exp=1" \
|
||||
-F "frame_interpolation_scale=1.0" \
|
||||
-o sync_i2v_interpolated.mp4
|
||||
```
|
||||
|
||||
## Create Response Format
|
||||
|
||||
`POST /v1/videos` returns a job record, not inline base64 video data.
|
||||
|
||||
@@ -165,6 +165,9 @@ curl -X POST http://localhost:8091/v1/videos \
|
||||
-F "guidance_scale_2=4.0" \
|
||||
-F "boundary_ratio=0.875" \
|
||||
-F "flow_shift=5.0" \
|
||||
-F "enable_frame_interpolation=true" \
|
||||
-F "frame_interpolation_exp=1" \
|
||||
-F "frame_interpolation_scale=1.0" \
|
||||
-F "seed=42"
|
||||
```
|
||||
|
||||
@@ -187,6 +190,35 @@ curl -X POST http://localhost:8091/v1/videos \
|
||||
| `flow_shift` | float | None | Scheduler flow shift (Wan2.2) |
|
||||
| `seed` | int | None | Random seed (reproducible) |
|
||||
| `lora` | object | None | LoRA configuration |
|
||||
| `enable_frame_interpolation` | bool | false | Enable RIFE frame interpolation before MP4 encoding |
|
||||
| `frame_interpolation_exp` | int | 1 | Interpolation exponent; 1=2x temporal resolution, 2=4x |
|
||||
| `frame_interpolation_scale` | float | 1.0 | RIFE inference scale; use 0.5 for high-resolution inputs |
|
||||
| `frame_interpolation_model_path` | str | None | Local directory or Hugging Face repo ID with `flownet.pkl`; defaults to `elfgum/RIFE-4.22.lite` |
|
||||
|
||||
## Frame Interpolation
|
||||
|
||||
Frame interpolation is an optional post-processing step for `/v1/videos` and
|
||||
`/v1/videos/sync`. It synthesizes intermediate frames between generated frames
|
||||
without rerunning the diffusion model. If the generated video has `N` frames,
|
||||
the interpolated output frame count is `(N - 1) * 2**exp + 1`. The encoder FPS
|
||||
is multiplied by `2**exp` so the output duration remains close to the original.
|
||||
|
||||
Frame interpolation runs in the diffusion worker post-processing path instead of
|
||||
the API server encoding path, so it can reuse the worker's current accelerator
|
||||
device without blocking the FastAPI event loop.
|
||||
|
||||
Example: generate 5 frames and interpolate to 9 frames:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8091/v1/videos/sync \
|
||||
-F "prompt=A dog running through a park" \
|
||||
-F "num_frames=5" \
|
||||
-F "fps=8" \
|
||||
-F "enable_frame_interpolation=true" \
|
||||
-F "frame_interpolation_exp=1" \
|
||||
-F "frame_interpolation_scale=1.0" \
|
||||
-o sync_t2v_interpolated.mp4
|
||||
```
|
||||
|
||||
## Create Response Format
|
||||
|
||||
|
||||
Reference in new issue
Block a user