lishunyang12
7c934b6188
- Add ``examples/offline_inference/lance/gradio_demo.py`` — a unified 7-task UI
(t2i / t2v / i2v / image_edit / video_edit / x2t_image / x2t_video) that
loads ``Lance_3B`` and ``Lance_3B_Video`` on two GPUs so image and video
tasks run concurrently. Surfaces official Lance examples per task; aspect
ratio is auto-snapped to one of upstream's 6 buckets from the input
image/video.
- Fix x2t_image / x2t_video output format: route x2t_image through a new
``_forward_x2t_image`` handler (was falling through to the img2img path
and emitting garbage), build the chat prefill in three segments
(prefix → ViT → suffix) so the ``<|im_end|>`` markers line up with
upstream's template, and start generation from a newline token instead of
``<|im_start|>`` so the assistant turn doesn't close itself with an
immediate EOS. Add a ``system_prompt`` override to ``render_lance_prompt``
so per-example QA instructions surface ("Look at the image carefully and
answer the question.") instead of the caption-style default.
- ``_forward_image_edit``: also stash the PIL image into ``custom_output``
so callers reading ``outputs[0].output`` *or* ``custom_output["image"]``
both work — the orchestrator IPC strips the bare ``output`` field across
process boundaries.
- ``pipeline_registry.py``, ``model_executor/models/lance/*``, registry
cleanup: remove the dead Lance-specific autoregressive shim
(``lance.py``) and its stage-input processor; the diffusion pipeline owns
the full flow now.
- Aspect-ratio buckets aligned with upstream's ``BucketResize`` (6 ratios:
21:9 / 16:9 / 4:3 / 1:1 / 3:4 / 9:16, sized for ``resolution_vae`` ∈
{480, 640, 768}). 3:2 / 2:3 dropped — upstream snaps those to 4:3 / 3:4.
- Gradio defaults match upstream's official scripts (cfg_text_scale=4.0,
validation_num_timesteps=30, validation_timestep_shift=3.5, seed=42); only
the demo *duration* is intentionally shorter than upstream's 121-frame
default to keep the UI snappy (user can extend to 10s via slider for
byte-identical reproduction).
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com>
Lance (ByteDance) — offline inference
Lance is a 3B unified
autoregressive + diffusion multimodal model on a Qwen2.5-VL backbone. It is
BAGEL-lineage (ByteDance Mixture-of-Transformers): the released Lance_3B
checkpoint uses the same *_moe_gen MoT weight layout as BAGEL, so vLLM-Omni
implements it by reusing the BAGEL transformer core and specializing only the
ViT (Qwen2.5-VL vision), the VAE (Wan2.2) and the checkpoint layout.
This example covers all six Lance modalities from the upstream HF model card:
t2i, t2v, image_edit, video_edit, x2t_image (image understanding) and
x2t_video (video understanding).
Hardware
Single NVIDIA GPU with 16 GB+ VRAM in BF16 (we test on B300 / A100). CUDA ≥ 12.4.
Run
# Text-to-image
python examples/offline_inference/lance/end2end.py \
--model bytedance-research/Lance \
--prompts "a corgi astronaut on the moon, cinematic" \
--steps 30 --cfg-text-scale 4.0 --timestep-shift 3.5 \
--height 1024 --width 1024 \
--output ./out
# Text-to-video (uses the Lance_3B_Video subfolder; see ``--modality``
# choices for all six task variants)
python examples/offline_inference/lance/end2end.py \
--model bytedance-research/Lance/Lance_3B_Video --modality text2video \
--num-frames 25 --video-height 480 --video-width 768 \
--prompts "a cat playing piano, cinematic" \
--steps 30 --fps 8 --output ./out
video_edit requires --model bytedance-research/Lance/Lance_3B_Video so
the 3-D latent_pos_embed table is loaded; the other paths can point at
the top-level bytedance-research/Lance repo and resolve the right
sub-checkpoint automatically.
The HF repo bundles everything (Lance_3B/, Lance_3B_Video/,
Qwen2.5-VL-ViT/, Wan2.2_VAE.pth); no separate downloads are required.
Defaults
Matches upstream inference_lance.sh: 30 denoising steps, timestep-shift 3.5,
text CFG 4.0, seed 42, 1024×1024 (override with --height / --width). For
the understanding paths (img2text / video2text), sampling is enabled by
default at --text-temperature 0.8 because Lance's greedy decoder emits an
immediate EOS for many prompts.