Files
vllm-omni/examples/offline_inference
lishunyang12 7c934b6188 [Lance] Add Gradio demo and align inference with upstream
- Add ``examples/offline_inference/lance/gradio_demo.py`` — a unified 7-task UI
  (t2i / t2v / i2v / image_edit / video_edit / x2t_image / x2t_video) that
  loads ``Lance_3B`` and ``Lance_3B_Video`` on two GPUs so image and video
  tasks run concurrently.  Surfaces official Lance examples per task; aspect
  ratio is auto-snapped to one of upstream's 6 buckets from the input
  image/video.
- Fix x2t_image / x2t_video output format: route x2t_image through a new
  ``_forward_x2t_image`` handler (was falling through to the img2img path
  and emitting garbage), build the chat prefill in three segments
  (prefix → ViT → suffix) so the ``<|im_end|>`` markers line up with
  upstream's template, and start generation from a newline token instead of
  ``<|im_start|>`` so the assistant turn doesn't close itself with an
  immediate EOS.  Add a ``system_prompt`` override to ``render_lance_prompt``
  so per-example QA instructions surface ("Look at the image carefully and
  answer the question.") instead of the caption-style default.
- ``_forward_image_edit``: also stash the PIL image into ``custom_output``
  so callers reading ``outputs[0].output`` *or* ``custom_output["image"]``
  both work — the orchestrator IPC strips the bare ``output`` field across
  process boundaries.
- ``pipeline_registry.py``, ``model_executor/models/lance/*``, registry
  cleanup: remove the dead Lance-specific autoregressive shim
  (``lance.py``) and its stage-input processor; the diffusion pipeline owns
  the full flow now.
- Aspect-ratio buckets aligned with upstream's ``BucketResize`` (6 ratios:
  21:9 / 16:9 / 4:3 / 1:1 / 3:4 / 9:16, sized for ``resolution_vae`` ∈
  {480, 640, 768}).  3:2 / 2:3 dropped — upstream snaps those to 4:3 / 3:4.
- Gradio defaults match upstream's official scripts (cfg_text_scale=4.0,
  validation_num_timesteps=30, validation_timestep_shift=3.5, seed=42); only
  the demo *duration* is intentionally shorter than upstream's 121-frame
  default to keep the UI snappy (user can extend to 10s via slider for
  byte-identical reproduction).

Signed-off-by: lishunyang12 <lishunyang12@163.com>

Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com>
2026-05-31 11:40:40 +00:00
..