[feat] Streaming diffusion video generation output (#3737)

Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

examples/online_serving/streaming_video_generation/README.md

@@ -0,0 +1,94 @@
+# Streaming Video Generation
+
+This example uses the custom WebSocket endpoint `WS /v1/realtime/video` to receive a video byte stream as chunks are produced.
+It covers text-only video generation. Image/reference input is intentionally not included for now.
+
+## Start The Server
+
+Start a diffusion video model with streaming output enabled:
+
+```bash
+vllm serve BestWishYsh/Helios-Distilled \
+  --omni \
+  --diffusion-streaming-output \
+  --port 8000
+```
+
+The `--diffusion-streaming-output` CLI flag is forwarded as `streaming_output=True` in the default diffusion stage `engine_args`, then loaded by `OmniDiffusionConfig.from_kwargs()`.
+
+## WebSocket Protocol
+
+| Direction | Message | Format | Description |
+| --- | --- | --- | --- |
+| Client to server | `session.start` | JSON text: `{"type":"session.start","model":"...","prompt":"...","format":"m4s"}` | Starts generation. `format` is optional and accepts `m4s` (default). Sampling fields such as `width`, `height`, `fps`, `num_frames`, and `extra_params` may be included. |
+| Server to client | `video.start` | JSON text: `{"type":"video.start","request_id":"...","format":"m4s","config":{...}}` | Confirms the session and mirrors the accepted `format`. |
+| Server to client | Video chunk | Binary WebSocket frame | Fragmented MP4 (`m4s`) video bytes. |
+| Client to server | `session.stop` | JSON text: `{"type":"session.stop"}` | Requests cancellation of the active session. |
+| Client to server | `session.ping` | JSON text: `{"type":"session.ping"}` | Optional keepalive; refreshes the server stall clock. |
+| Server to client | `session.done` | JSON text: `{"type":"session.done","request_id":"...","chunks":3,"stopped":false}` | Ends a completed or stopped session. |
+| Server to client | `session.pong` | JSON text: `{"type":"session.pong"}` | Reply to `session.ping`. |
+| Server to client | `error` | JSON text: `{"type":"error","message":"..."}` | Reports invalid input, unsupported formats, generation failures, control-message errors, or stall timeout. |
+
+During generation the client normally sends only `session.start` and then receives binary chunks; silence on the client socket is expected. The server closes the session with a stall error only when there is no engine progress and no `session.ping` for about 60 seconds.
+
+## Install Client Dependency
+
+```bash
+pip install websockets
+# For the Gradio demo:
+pip install 'vllm-omni[demo]' websockets
+```
+
+## Run The Client
+
+```bash
+python streaming_video_client.py \
+  --host 127.0.0.1 \
+  --port 8000 \
+  --model BestWishYsh/Helios-Distilled \
+  --prompt "A serene lakeside sunrise with mist over the water." \
+  --width 640 \
+  --height 384 \
+  --fps 16 \
+  --num-frames 99 \
+  --guidance-scale 1.0 \
+  --seed 42 \
+  --output helios_stream.mp4
+```
+
+The client sends one `session.start` message, prints each received binary video chunk with its byte size and elapsed time, and saves the received bytes to `--output` after `session.done`.
+The client remuxes the gathered stream to a regular progressive MP4 file so that local playback knows the video duration.
+
+## Run The Gradio Demo
+
+```bash
+python gradio_demo.py \
+  --host 127.0.0.1 \
+  --port 7860
+```
+
+The Gradio demo requests fMP4 (`m4s`) chunks and appends them directly in the browser with a Media Source Extensions player.
+
+## Model Choice
+
+### Helios
+
+The example uses `BestWishYsh/Helios-Distilled` model by default.
+
+To ensure streaming-level generation speed, `pyramid_num_inference_steps_list` is suggested to be as low as `[1, 1, 1]`. Both example clients uses the following Helios-Distilled preset by default:
+
+```json
+{
+  "is_enable_stage2": true,
+  "pyramid_num_stages": 3,
+  "pyramid_num_inference_steps_list": [1, 1, 1],
+  "is_amplify_first_chunk": true
+}
+```
+
+Disable it in the CLI example with `--no-helios-distilled-preset`, or override/extend it with `--extra-params`:
+
+```bash
+python streaming_video_client.py \
+  --extra-params '{"pyramid_num_inference_steps_list":[2, 2, 2]}'
+```

examples/online_serving/streaming_video_generation/gradio_demo.py

@@ -0,0 +1,301 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""Gradio demo for `/v1/realtime/video`.
+
+Usage:
+    python gradio_demo.py --host 127.0.0.1 --port 7860
+
+Start a vLLM-Omni server with streaming output enabled first, for example:
+    vllm serve BestWishYsh/Helios-Distilled --omni --diffusion-streaming-output --port 8000
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+from typing import Any
+
+try:
+    import gradio as gr
+except ImportError:
+    raise ImportError("gradio is required to run this demo. Install it with: pip install 'vllm-omni[demo]'") from None
+
+DEFAULT_PROMPT = "A vibrant tropical fish swimming gracefully among colorful coral reefs in a clear, turquoise ocean. The fish has bright blue and yellow scales with a small, distinctive orange spot on its side, its fins moving fluidly. The coral reefs are alive with a variety of marine life, including small schools of colorful fish and sea turtles gliding by. The water is crystal clear, allowing for a view of the sandy ocean floor below. The reef itself is adorned with a mix of hard and soft corals in shades of red, orange, and green. The photo captures the fish from a slightly elevated angle, emphasizing its lively movements and the vivid colors of its surroundings. A close-up shot with dynamic movement."
+VIDEO_STREAM_VIEW_HTML = Path(__file__).with_name("video-stream-view.html")
+VIDEO_STREAM_VIEW_JS = Path(__file__).with_name("video-stream-view.js")
+
+
+def _maybe_set(payload: dict[str, Any], key: str, value: Any) -> None:
+    if value is None:
+        return
+    if isinstance(value, str) and not value.strip():
+        return
+    payload[key] = value
+
+
+def _optional_int(value: int | float | None) -> int | None:
+    return None if value is None else int(value)
+
+
+def _optional_float(value: int | float | None) -> float | None:
+    return None if value is None else float(value)
+
+
+def _build_session_start(
+    *,
+    model: str,
+    prompt: str,
+    negative_prompt: str,
+    width: int | float | None,
+    height: int | float | None,
+    fps: int | float | None,
+    num_frames: int | float | None,
+    num_inference_steps: int | float | None,
+    guidance_scale: int | float | None,
+    guidance_scale_2: int | float | None,
+    boundary_ratio: int | float | None,
+    flow_shift: int | float | None,
+    true_cfg_scale: int | float | None,
+    seed: int | float | None,
+) -> dict[str, Any]:
+    payload: dict[str, Any] = {
+        "type": "session.start",
+        "model": model,
+        "prompt": prompt,
+        "format": "m4s",
+    }
+
+    _maybe_set(payload, "negative_prompt", negative_prompt)
+    _maybe_set(payload, "width", _optional_int(width))
+    _maybe_set(payload, "height", _optional_int(height))
+    _maybe_set(payload, "fps", _optional_int(fps))
+    _maybe_set(payload, "num_frames", _optional_int(num_frames))
+    _maybe_set(payload, "num_inference_steps", _optional_int(num_inference_steps))
+    _maybe_set(payload, "guidance_scale", _optional_float(guidance_scale))
+    _maybe_set(payload, "guidance_scale_2", _optional_float(guidance_scale_2))
+    _maybe_set(payload, "boundary_ratio", _optional_float(boundary_ratio))
+    _maybe_set(payload, "flow_shift", _optional_float(flow_shift))
+    _maybe_set(payload, "true_cfg_scale", _optional_float(true_cfg_scale))
+    _maybe_set(payload, "seed", _optional_int(seed))
+
+    if "Helios" in model:
+        payload["extra_params"] = {
+            "is_enable_stage2": True,
+            "pyramid_num_stages": 3,
+            "pyramid_num_inference_steps_list": [1, 1, 1],
+            "is_amplify_first_chunk": True,
+        }
+
+    return payload
+
+
+def _set_optional_field_enabled(enabled: bool) -> Any:
+    return gr.update(interactive=enabled)
+
+
+def build_browser_stream_config(
+    host: str,
+    port: int | float,
+    model: str,
+    prompt: str,
+    negative_prompt: str,
+    width: int | float | None,
+    height: int | float | None,
+    fps: int | float,
+    num_frames: int | float | None,
+    num_inference_steps: int | float | None,
+    include_guidance_scale: bool,
+    guidance_scale: int | float | None,
+    include_guidance_scale_2: bool,
+    guidance_scale_2: int | float | None,
+    include_seed: bool,
+    seed: int | float | None,
+    include_boundary_ratio: bool,
+    boundary_ratio: int | float | None,
+    include_flow_shift: bool,
+    flow_shift: int | float | None,
+    include_true_cfg_scale: bool,
+    true_cfg_scale: int | float | None,
+) -> tuple[str, Any]:
+    """Build the browser-side WebSocket request config for the HTML/MSE player."""
+    if not model.strip():
+        raise gr.Error("Model is required.")
+    if not prompt.strip():
+        raise gr.Error("Prompt is required.")
+
+    payload = _build_session_start(
+        model=model.strip(),
+        prompt=prompt.strip(),
+        negative_prompt=negative_prompt,
+        width=width,
+        height=height,
+        fps=fps,
+        num_frames=num_frames,
+        num_inference_steps=num_inference_steps,
+        guidance_scale=guidance_scale if include_guidance_scale else None,
+        guidance_scale_2=guidance_scale_2 if include_guidance_scale_2 else None,
+        boundary_ratio=boundary_ratio if include_boundary_ratio else None,
+        flow_shift=flow_shift if include_flow_shift else None,
+        true_cfg_scale=true_cfg_scale if include_true_cfg_scale else None,
+        seed=seed if include_seed else None,
+    )
+    config = {
+        "url": f"ws://{host}:{int(port)}/v1/realtime/video",
+        "payload": payload,
+    }
+    return json.dumps(config, ensure_ascii=False), gr.update(value="Streaming...", interactive=False)
+
+
+def create_demo() -> gr.Blocks:
+    with gr.Blocks(title="Streaming Video Generation") as demo:
+        gr.Markdown("# Streaming Video Generation")
+        gr.Markdown(
+            "Connects to `WS /v1/realtime/video` in the browser and appends fMP4 chunks "
+            "directly to a Media Source Extensions video player."
+        )
+
+        with gr.Row():
+            with gr.Column(scale=1):
+                host = gr.Textbox(label="Server Host", value="127.0.0.1")
+                port = gr.Number(label="Server Port", value=8000, precision=0)
+                model = gr.Textbox(label="Model", value="BestWishYsh/Helios-Distilled")
+                prompt = gr.Textbox(label="Prompt", value=DEFAULT_PROMPT, lines=3)
+                negative_prompt = gr.Textbox(label="Negative Prompt", value="", lines=2)
+
+                with gr.Row():
+                    width = gr.Number(label="Width", value=640, precision=0)
+                    height = gr.Number(label="Height", value=384, precision=0)
+
+                with gr.Row():
+                    fps = gr.Number(label="FPS", value=16, precision=0)
+                    num_frames = gr.Number(label="Num Frames", value=132, precision=0)
+                    num_inference_steps = gr.Number(label="Num Inference Steps", value=50, precision=0)
+
+                with gr.Row():
+                    with gr.Column():
+                        guidance_scale = gr.Number(
+                            label="Guidance Scale",
+                            value=1.0,
+                            interactive=True,
+                            scale=4,
+                        )
+                        include_guidance_scale = gr.Checkbox(label="Send", value=True, scale=1, min_width=70)
+                    with gr.Column():
+                        guidance_scale_2 = gr.Number(
+                            label="Guidance Scale 2",
+                            value=None,
+                            interactive=False,
+                            scale=4,
+                        )
+                        include_guidance_scale_2 = gr.Checkbox(label="Send", value=False, scale=1, min_width=70)
+                    with gr.Column():
+                        true_cfg_scale = gr.Number(
+                            label="True CFG Scale",
+                            value=None,
+                            interactive=False,
+                            scale=4,
+                        )
+                        include_true_cfg_scale = gr.Checkbox(label="Send", value=False, scale=1, min_width=70)
+                    # I want it to wrap after 3, but it auto wrap after 2. I'll rather fully delegate to auto wrapping.
+                    with gr.Column():
+                        boundary_ratio = gr.Number(
+                            label="Boundary Ratio",
+                            value=None,
+                            interactive=False,
+                            scale=4,
+                        )
+                        include_boundary_ratio = gr.Checkbox(label="Send", value=False, scale=1, min_width=70)
+                    with gr.Column():
+                        flow_shift = gr.Number(label="Flow Shift", value=None, interactive=False, scale=4)
+                        include_flow_shift = gr.Checkbox(label="Send", value=False, scale=1, min_width=70)
+                    with gr.Column():
+                        seed = gr.Number(label="Seed", value=42, precision=0, interactive=True, scale=4)
+                        include_seed = gr.Checkbox(label="Send", value=True, scale=1, min_width=70)
+
+                with gr.Row():
+                    start_button = gr.Button("Start", variant="primary", elem_id="streaming-video-start")
+                    hidden_stream_config = gr.Textbox(value="", visible=False)
+
+            with gr.Column(scale=1):
+                gr.HTML(
+                    VIDEO_STREAM_VIEW_HTML.read_text(encoding="utf-8"),
+                    label="Streaming Preview",
+                    js_on_load=VIDEO_STREAM_VIEW_JS.read_text(encoding="utf-8"),
+                )
+
+        inputs = [
+            host,
+            port,
+            model,
+            prompt,
+            negative_prompt,
+            width,
+            height,
+            fps,
+            num_frames,
+            num_inference_steps,
+            include_guidance_scale,
+            guidance_scale,
+            include_guidance_scale_2,
+            guidance_scale_2,
+            include_seed,
+            seed,
+            include_boundary_ratio,
+            boundary_ratio,
+            include_flow_shift,
+            flow_shift,
+            include_true_cfg_scale,
+            true_cfg_scale,
+        ]
+        for toggle, field in [
+            (include_guidance_scale, guidance_scale),
+            (include_guidance_scale_2, guidance_scale_2),
+            (include_seed, seed),
+            (include_boundary_ratio, boundary_ratio),
+            (include_flow_shift, flow_shift),
+            (include_true_cfg_scale, true_cfg_scale),
+        ]:
+            toggle.change(
+                fn=_set_optional_field_enabled,
+                inputs=toggle,
+                outputs=field,
+            )
+        start_button.click(
+            fn=build_browser_stream_config,
+            inputs=inputs,
+            outputs=[hidden_stream_config, start_button],
+        ).then(
+            fn=lambda config: config,
+            inputs=hidden_stream_config,
+            outputs=hidden_stream_config,
+            js="""
+            (config) => {
+              if (window.vllmStreamingVideoStart) {
+                window.vllmStreamingVideoStart(config);
+              }
+              return config;
+            }
+            """,
+        )
+
+    return demo
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description="Streaming video generation Gradio demo")
+    parser.add_argument("--host", default="127.0.0.1", help="Host/IP for gradio launch")
+    parser.add_argument("--port", type=int, default=7860, help="Port for gradio launch")
+    parser.add_argument("--share", action="store_true", help="Create a public Gradio link")
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+    demo = create_demo()
+    demo.queue().launch(server_name=args.host, server_port=args.port, share=args.share)
+
+
+if __name__ == "__main__":
+    main()

examples/online_serving/streaming_video_generation/streaming_video_client.py

@@ -0,0 +1,279 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""Text-only WebSocket client for `/v1/realtime/video`.
+
+The client prints a line for every binary video chunk it receives and writes
+the received bytes to disk when the session finishes or is interrupted.
+Image/reference input is intentionally not supported in this example yet.
+
+Requirements:
+    pip install websockets
+"""
+
+import argparse
+import asyncio
+import io
+import json
+import os
+import time
+from pathlib import Path
+from typing import Any
+
+import av
+
+try:
+    from websockets.asyncio.client import connect  # pyright: ignore[reportMissingImports]
+except ImportError:
+    try:
+        from websockets import connect  # pyright: ignore[reportMissingImports]
+    except ImportError:
+        print("Please install websockets: pip install websockets")
+        raise SystemExit(1)
+
+
+HELIOS_DISTILLED_EXTRA_PARAMS = {
+    "is_enable_stage2": True,
+    "pyramid_num_stages": 3,
+    "pyramid_num_inference_steps_list": [1, 1, 1],
+    "is_amplify_first_chunk": True,
+}
+
+
+def _json_object(value: str, *, argument_name: str) -> dict[str, Any]:
+    try:
+        parsed = json.loads(value)
+    except json.JSONDecodeError as exc:
+        raise argparse.ArgumentTypeError(f"{argument_name} must be valid JSON: {exc}") from exc
+    if not isinstance(parsed, dict):
+        raise argparse.ArgumentTypeError(f"{argument_name} must be a JSON object")
+    return parsed
+
+
+def _maybe_set(payload: dict[str, Any], key: str, value: Any) -> None:
+    if value is not None:
+        payload[key] = value
+
+
+def build_session_start(args: argparse.Namespace) -> dict[str, Any]:
+    payload: dict[str, Any] = {
+        "type": "session.start",
+        "model": args.model,
+        "prompt": args.prompt,
+    }
+
+    _maybe_set(payload, "negative_prompt", args.negative_prompt)
+    _maybe_set(payload, "width", args.width)
+    _maybe_set(payload, "height", args.height)
+    _maybe_set(payload, "size", args.size)
+    _maybe_set(payload, "seconds", args.seconds)
+    _maybe_set(payload, "fps", args.fps)
+    _maybe_set(payload, "num_frames", args.num_frames)
+    _maybe_set(payload, "num_inference_steps", args.num_inference_steps)
+    _maybe_set(payload, "guidance_scale", args.guidance_scale)
+    _maybe_set(payload, "guidance_scale_2", args.guidance_scale_2)
+    _maybe_set(payload, "boundary_ratio", args.boundary_ratio)
+    _maybe_set(payload, "flow_shift", args.flow_shift)
+    _maybe_set(payload, "true_cfg_scale", args.true_cfg_scale)
+    _maybe_set(payload, "seed", args.seed)
+
+    extra_params: dict[str, Any] = {}
+    if "Helios-Distilled" in args.model and args.helios_distilled_preset:
+        extra_params.update(HELIOS_DISTILLED_EXTRA_PARAMS)
+    if args.extra_params:
+        extra_params.update(args.extra_params)
+    if extra_params:
+        payload["extra_params"] = extra_params
+
+    return payload
+
+
+def _count_decoded_frames(video_bytes: bytes, *, stream_format: str) -> int | None:
+    """Return the number of decodable video frames in a streamed byte sequence, if known."""
+    if not video_bytes:
+        return 0
+
+    av_format = "mp4" if stream_format == "m4s" else None
+    try:
+        with av.open(io.BytesIO(video_bytes), format=av_format, mode="r") as container:
+            stream = container.streams.video[0]
+            return sum(1 for _ in container.decode(stream))
+    except Exception:
+        return None
+
+
+def save_video(
+    chunks: list[bytes],
+    output_path: Path,
+    *,
+    fps: float,
+    stream_format: str,
+) -> None:
+    if not chunks:
+        print("No video chunks received; nothing to save.")
+        return
+
+    from vllm_omni.diffusion.utils.media_utils import finalize_streaming_video_bytes
+
+    streamed_bytes = b"".join(chunks)
+    playback_bytes = finalize_streaming_video_bytes(
+        streamed_bytes,
+        input_format="m4s",
+        fps=fps,
+    )
+
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    tmp_path = output_path.with_suffix(output_path.suffix + ".tmp")
+    with tmp_path.open("wb") as f:
+        f.write(playback_bytes)
+    os.replace(tmp_path, output_path)
+    print(
+        f"Saved {len(chunks)} streamed chunk(s), {len(streamed_bytes)} streamed bytes "
+        f"-> {len(playback_bytes)} playback bytes at {output_path}"
+    )
+
+
+async def stream_video(args: argparse.Namespace) -> None:
+    payload = build_session_start(args)
+    output_path = Path(args.output)
+    url = f"ws://{args.host}:{args.port}/v1/realtime/video"
+    chunks: list[bytes] = []
+    total_bytes = 0
+    total_frames = 0
+    accumulated_stream = bytearray()
+    stream_format = "m4s"
+    done = False
+    ws = None
+
+    try:
+        async with connect(url, max_size=None) as websocket:
+            ws = websocket
+            await websocket.send(json.dumps(payload))
+            print(f"Connected: {url}")
+            print(f"Sent session.start: {json.dumps(payload, ensure_ascii=False)}")
+            started_at = time.perf_counter()
+            last_chunk_at = started_at
+
+            while True:
+                message = await websocket.recv()
+                now = time.perf_counter()
+
+                if isinstance(message, bytes):
+                    chunks.append(message)
+                    chunk_bytes = len(message)
+                    chunk_elapsed = now - last_chunk_at
+                    last_chunk_at = now
+
+                    total_bytes += chunk_bytes
+                    accumulated_stream.extend(message)
+                    decoded_total = _count_decoded_frames(bytes(accumulated_stream), stream_format=stream_format)
+                    if decoded_total is not None:
+                        chunk_frames = max(decoded_total - total_frames, 0)
+                        total_frames = decoded_total
+                    else:
+                        chunk_frames = 0
+
+                    total_elapsed = now - started_at
+                    print(
+                        f"[chunk {len(chunks):03d}] "
+                        f"bytes={chunk_bytes} "
+                        f"frames={chunk_frames} "
+                        f"elapsed={chunk_elapsed:.2f}s "
+                        f"total_bytes={total_bytes} "
+                        f"total_frames={total_frames} "
+                        f"total_elapsed={total_elapsed:.2f}s"
+                    )
+                    continue
+
+                msg = json.loads(message)
+                msg_type = msg.get("type")
+                if msg_type == "video.start":
+                    stream_format = msg.get("format") or stream_format
+                    print(f"Video session started: request_id={msg.get('request_id')} format={msg.get('format')}")
+                elif msg_type == "session.done":
+                    print(f"Session complete: {json.dumps(msg, ensure_ascii=False)}")
+                    done = True
+                    break
+                elif msg_type == "error":
+                    print(f"ERROR: {msg.get('message', msg)}")
+                else:
+                    print(f"Received control message: {msg}")
+    except (KeyboardInterrupt, asyncio.CancelledError):
+        print("\nInterrupted. Asking server to stop the active session...")
+        if ws is not None:
+            try:
+                await ws.send(json.dumps({"type": "session.stop"}))
+                await ws.close(code=1000, reason="client interrupted")
+            except Exception:
+                pass
+        raise
+    finally:
+        print(
+            "Saving video... (May take a while. Remuxing concatenated chunks into one progressive MP4 file with total duration metadata in file header)"
+        )
+        save_video(
+            chunks,
+            output_path,
+            fps=float(payload.get("fps") or args.fps),
+            stream_format=stream_format,
+        )
+        if not done:
+            print("Saved partial output because the stream did not finish normally.")
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description="Streaming video generation WebSocket client")
+    parser.add_argument("--host", default="127.0.0.1", help="API server host")
+    parser.add_argument("--port", type=int, default=8000, help="API server port")
+    parser.add_argument(
+        "--model",
+        default="BestWishYsh/Helios-Distilled",
+        help="Model name served by the API server",
+    )
+    parser.add_argument(
+        "--prompt",
+        default="A vibrant tropical fish swimming gracefully among colorful coral reefs in a clear, turquoise ocean. The fish has bright blue and yellow scales with a small, distinctive orange spot on its side, its fins moving fluidly. The coral reefs are alive with a variety of marine life, including small schools of colorful fish and sea turtles gliding by. The water is crystal clear, allowing for a view of the sandy ocean floor below. The reef itself is adorned with a mix of hard and soft corals in shades of red, orange, and green. The photo captures the fish from a slightly elevated angle, emphasizing its lively movements and the vivid colors of its surroundings. A close-up shot with dynamic movement.",
+        help="Text prompt for video generation",
+    )
+    parser.add_argument("--output", default="streaming_video_output.mp4", help="Path for the received video bytes")
+    parser.add_argument("--width", type=int, default=640, help="Video width")
+    parser.add_argument("--height", type=int, default=384, help="Video height")
+    parser.add_argument("--size", default=None, help="Video size as WIDTHxHEIGHT; alternative to width/height")
+    parser.add_argument("--seconds", default=None, help="Clip duration in seconds")
+    parser.add_argument("--fps", type=int, default=16, help="Frames per second")
+    parser.add_argument("--num-frames", type=int, default=99, help="Number of generated frames")
+
+    parser.add_argument("--negative-prompt", default=None, help="Negative prompt")
+    parser.add_argument("--num-inference-steps", type=int, default=None, help="Number of diffusion steps")
+    parser.add_argument("--guidance-scale", type=float, default=1.0, help="Classifier-free guidance scale")
+    parser.add_argument("--guidance-scale-2", type=float, default=None, help="High-noise CFG scale for video models")
+    parser.add_argument("--boundary-ratio", type=float, default=None, help="Boundary split ratio for video models")
+    parser.add_argument("--flow-shift", type=float, default=None, help="Scheduler flow_shift for video models")
+    parser.add_argument("--true-cfg-scale", type=float, default=None, help="True CFG scale for models that support it")
+    parser.add_argument("--seed", type=int, default=42, help="Random seed")
+
+    parser.add_argument(
+        "--helios-distilled-preset",
+        action=argparse.BooleanOptionalAction,
+        default=True,
+        help="Send the Helios-Distilled extra_params used by the bundled example.",
+    )
+    parser.add_argument(
+        "--extra-params",
+        type=lambda value: _json_object(value, argument_name="--extra-params"),
+        default=None,
+        help="JSON object merged into extra_params; overrides preset keys on conflict.",
+    )
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+    try:
+        asyncio.run(stream_video(args))
+    except KeyboardInterrupt:
+        pass
+
+
+if __name__ == "__main__":
+    main()

examples/online_serving/streaming_video_generation/video-stream-view.html

@@ -0,0 +1,11 @@
+<!-- This file and the corresponding JS file are needed to display a streamable video view in the Gradio demo.
+ Gradio's built-in video player does not support modern M4S streaming format. -->
+<div id="vllm-streaming-video-view" style="display:flex; flex-direction:column; gap:10px;">
+  <video id="vllm-streaming-video-player" autoplay muted controls playsinline
+    style="width:100%; min-height:320px; background:#111; border-radius:8px;"></video>
+  <div style="display:flex; align-items:center; gap:10px;">
+    <span id="vllm-streaming-video-status" style="font-weight:600;">Ready</span>
+  </div>
+  <pre id="vllm-streaming-video-log" style="height:220px; overflow:auto; margin:0; padding:10px; border-radius:8px;
+              background:#0f172a; color:#dbeafe; font-size:12px; white-space:pre-wrap;">Ready.</pre>
+</div>

examples/online_serving/streaming_video_generation/video-stream-view.js

@@ -0,0 +1,198 @@
+/**
+ * This file and the corresponding HTML file are needed to display a streamable video view in the Gradio demo.
+ * Gradio's built-in video player does not support modern M4S streaming format.
+ */
+(function () {
+  const playerId = "vllm-streaming-video-player";
+  const statusId = "vllm-streaming-video-status";
+  const logId = "vllm-streaming-video-log";
+  let ws = null;
+  let mediaSource = null;
+  let sourceBuffer = null;
+  let queue = [];
+  let chunkCount = 0;
+  let totalBytes = 0;
+  let done = false;
+  let objectUrl = null;
+
+  function el(id) {
+    return document.getElementById(id);
+  }
+
+  function setStatus(text) {
+    const node = el(statusId);
+    if (node) node.textContent = text;
+  }
+
+  function log(line) {
+    const node = el(logId);
+    if (!node) return;
+    node.textContent += "\n" + line;
+    node.scrollTop = node.scrollHeight;
+  }
+
+  function setStartButton(disabled, label) {
+    const btn = document.querySelector("#streaming-video-start button") || document.getElementById("streaming-video-start");
+    if (!btn) return;
+    btn.disabled = disabled;
+    btn.textContent = label;
+  }
+
+  function chooseMime() {
+    const candidates = [
+      'video/mp4; codecs="avc1.42E01E"',
+      'video/mp4; codecs="avc1.4D401F"',
+      'video/mp4; codecs="avc1.64001F"',
+      "video/mp4"
+    ];
+    for (const mime of candidates) {
+      if (window.MediaSource && MediaSource.isTypeSupported(mime)) return mime;
+    }
+    return null;
+  }
+
+  function pump() {
+    if (!sourceBuffer || sourceBuffer.updating || queue.length === 0) return;
+    try {
+      sourceBuffer.appendBuffer(queue.shift());
+    } catch (err) {
+      done = true;
+      setStatus("SourceBuffer error");
+      setStartButton(false, "Restart");
+      log("SourceBuffer append error: " + err.message);
+      finishStream();
+    }
+  }
+
+  function finishStream() {
+    if (!mediaSource || mediaSource.readyState !== "open") return;
+    if (sourceBuffer && sourceBuffer.updating) return;
+    if (queue.length > 0) return;
+    try {
+      mediaSource.endOfStream();
+    } catch (_) { }
+  }
+
+  function stopCurrent() {
+    if (ws && ws.readyState === WebSocket.OPEN) {
+      try { ws.send(JSON.stringify({ type: "session.stop" })); } catch (_) { }
+      try { ws.close(); } catch (_) { }
+    }
+    ws = null;
+    sourceBuffer = null;
+    mediaSource = null;
+    queue = [];
+    if (objectUrl) URL.revokeObjectURL(objectUrl);
+    objectUrl = null;
+  }
+
+  window.vllmStreamingVideoStart = function (configJson) {
+    stopCurrent();
+    chunkCount = 0;
+    totalBytes = 0;
+    done = false;
+    const logNode = el(logId);
+    if (logNode) logNode.textContent = "Starting...";
+    setStatus("Starting...");
+
+    let config;
+    try {
+      config = JSON.parse(configJson);
+    } catch (err) {
+      setStatus("Invalid request");
+      setStartButton(false, "Restart");
+      log("Invalid request config: " + err.message);
+      return configJson;
+    }
+
+    const mime = chooseMime();
+    if (!mime) {
+      setStatus("MSE unsupported");
+      setStartButton(false, "Restart");
+      log("This browser does not support MP4 Media Source Extensions playback.");
+      return configJson;
+    }
+
+    const video = el(playerId);
+    if (!video) return configJson;
+    mediaSource = new MediaSource();
+    objectUrl = URL.createObjectURL(mediaSource);
+    video.src = objectUrl;
+
+    mediaSource.addEventListener("sourceopen", () => {
+      try {
+        sourceBuffer = mediaSource.addSourceBuffer(mime);
+        sourceBuffer.mode = "segments";
+        sourceBuffer.addEventListener("updateend", () => {
+          pump();
+          if (done) finishStream();
+        });
+      } catch (err) {
+        setStatus("SourceBuffer error");
+        setStartButton(false, "Restart");
+        log("SourceBuffer error: " + err.message);
+        return;
+      }
+
+      ws = new WebSocket(config.url);
+      ws.binaryType = "arraybuffer";
+      ws.onopen = () => {
+        setStatus("Streaming...");
+        setStartButton(true, "Streaming...");
+        log("Connected: " + config.url);
+        ws.send(JSON.stringify(config.payload));
+        log("Sent session.start: " + JSON.stringify(config.payload));
+      };
+      ws.onmessage = (event) => {
+        if (typeof event.data === "string") {
+          const msg = JSON.parse(event.data);
+          if (msg.type === "video.start") {
+            log("Video session started: request_id=" + (msg.request_id || "") + " format=" + (msg.format || ""));
+          } else if (msg.type === "session.done") {
+            done = true;
+            setStatus("Done");
+            setStartButton(false, "Restart");
+            log("Session complete: " + JSON.stringify(msg));
+            finishStream();
+          } else if (msg.type === "error") {
+            done = true;
+            setStatus("Error");
+            setStartButton(false, "Restart");
+            log("ERROR: " + (msg.message || JSON.stringify(msg)));
+            finishStream();
+          } else {
+            log("Control message: " + JSON.stringify(msg));
+          }
+          return;
+        }
+
+        const data = event.data;
+        queue.push(data);
+        chunkCount += 1;
+        totalBytes += data.byteLength;
+        log(`[chunk ${String(chunkCount).padStart(3, "0")}] bytes=${data.byteLength} total_bytes=${totalBytes}`);
+        pump();
+      };
+      ws.onerror = () => {
+        done = true;
+        setStatus("Error");
+        setStartButton(false, "Restart");
+        log("ERROR: WebSocket error");
+        finishStream();
+      };
+      ws.onclose = () => {
+        if (!done) {
+          done = true;
+          setStatus("Closed");
+          setStartButton(false, "Restart");
+          log("WebSocket closed before session.done");
+          finishStream();
+        }
+      };
+    }, { once: true });
+
+    return configJson;
+  };
+
+  console.log("video-stream-view.html loaded");
+})();