[Model][Frontend] MiniMax-H3: Add opt-in planar video response encoding (#6288)
Signed-off-by: MosCloud <bwzhang1991@163.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
3d34b8b977
commit
4882a36c41
9 files changed
+1077
-23
No files matched your search
@@ -118,6 +118,21 @@ On Atlas 800I A3 (64 GB HBM per device) the combined service does not fit at
|
||||
above) or HSDP — see
|
||||
[§ Memory and attention optimizations](#memory-and-attention-optimizations-a3).
|
||||
|
||||
### CPU MP4 response encoding (Atlas A2)
|
||||
|
||||
Non-streaming MP4 responses use one public automatic encoder. It checks the
|
||||
runtime frame shape, common dtype, and RGB channel-plane contiguity for every
|
||||
request; compatible inputs use direct planar PyAV frames, while unsupported
|
||||
inputs fall back to the legacy muxer before opening the PyAV container. No CLI
|
||||
flag, model declaration, or user configuration is required. Streaming fMP4
|
||||
output remains on its existing incremental path.
|
||||
|
||||
Current performance and correctness validation is limited to one Atlas A2 host
|
||||
with 8x Ascend 910B4-1 NPUs, the `FL2VA`/`t2va` partition, one request at a
|
||||
time, 1344x768 at 24 fps, and 5, 8.7, and 15 second requests. This optimization
|
||||
only changes CPU MP4 response encoding; it does not change DiT execution or
|
||||
stage 0.
|
||||
|
||||
### Optional optimizations
|
||||
|
||||
Two independent optimizations may be enabled on top of the configuration
|
||||
|
||||
@@ -65,6 +65,31 @@ On AMD ROCm, install without the `[fa4]` extra (FA4 is CUDA-only) and use
|
||||
`ffmpeg` and `ffprobe` must be available on `PATH`. They are used for
|
||||
reference-video preparation and MP4 output.
|
||||
|
||||
## CPU MP4 response encoding
|
||||
|
||||
For CUDA and ROCm deployments, non-streaming MP4 responses are encoded on the
|
||||
host CPU through PyAV/libx264 after generation. The response encoder selects the
|
||||
path automatically at runtime: inputs with a supported frame shape, common dtype,
|
||||
and RGB per-channel contiguous layout use direct planar frames; other inputs
|
||||
fall back to the legacy path before the PyAV container is opened. No CLI flag,
|
||||
model declaration, or user configuration is required. Streaming fMP4 output is
|
||||
unchanged.
|
||||
|
||||
A community benchmark on 2x Xeon 8480C reported the following comparison
|
||||
between the legacy and direct planar paths
|
||||
([full result](https://github.com/vllm-project/vllm-omni/pull/6288#issuecomment-5337546499)):
|
||||
|
||||
| Metric | Legacy | Direct planar | Change |
|
||||
|---|---:|---:|---:|
|
||||
| Median wall time | 1.805 s | 1.394 s | -22.8% |
|
||||
| Median process CPU time | 3.613 s | 3.207 s | -11.2% |
|
||||
| Peak RSS | 3182 MiB | 2794 MiB | -387 MiB (-12.2%) |
|
||||
|
||||
Across the 1.0-8.7 s sweep, wall-time improvement was approximately 21.8-22.6%;
|
||||
outputs were byte-identical and full decode passed. This is evidence for host
|
||||
CPU response encoding. Actual gains depend on the CPU and runtime, and should
|
||||
not be interpreted as GPU, DiT, or stage 0 speedups.
|
||||
|
||||
## Start a server
|
||||
|
||||
Pass the repository ID directly. The pipeline uses `FL2VA` for model discovery
|
||||
|
||||
Reference in new issue
Block a user