lishunyang
69bcc6416d
Adds Lance (bytedance-research/Lance), a 3B unified AR+diffusion multimodal model on a Qwen2.5-VL backbone. Lance is BAGEL-lineage: it reuses vLLM-Omni's BAGEL Mixture-of-Transformers core verbatim and specializes only the ViT (Qwen2.5-VL vision), the VAE (Wan2.2) and the checkpoint layout. All six single-stage Lance modalities verified end-to-end on 1x B300: t2i, t2v, image_edit, video_edit, x2t_image (image understanding) and x2t_video (video understanding). 3-D mRoPE is wired through BagelRotaryEmbedding for the x2t / video / edit paths. Includes the upstream gap-analysis quality fixes (task-specific system prompts, full Qwen chat template, upstream inference defaults and aspect-ratio buckets). Ships an offline example (examples/offline_inference/lance/), an OpenAI-compatible online-serving example (examples/online_serving/lance/) and an e2e test (tests/e2e/online_serving/test_lance.py). See LANCE_PR.md for the architecture rationale, per-modality status and the two remaining follow-ups (two-stage AR+DiT, video_edit polish). Closes #3697. Signed-off-by: lishunyang <lishunyang03@gmail.com>
30 lines
976 B
YAML
30 lines
976 B
YAML
# Lance-3B (ByteDance) deploy — single-stage, CUDA. Target: 1x NVIDIA A100 80GB.
|
|
#
|
|
# Lance is BAGEL-lineage (Qwen2-MoT unified AR+diffusion). The single-stage
|
|
# DiT stage is self-contained: it owns the Qwen2-MoT LLM, the Qwen2.5-VL ViT,
|
|
# the Wan2.2 VAE and the tokenizer (text2img / image_edit / x2t_image).
|
|
#
|
|
# Lance-3B BF16 footprint: ~7GB LLM + Qwen2.5-VL ViT + Wan2.2 VAE — comfortably
|
|
# within one A100 80GB. The video checkpoint (Lance_3B_Video) is a follow-up.
|
|
#
|
|
# Select via:
|
|
# vllm serve bytedance-research/Lance --omni \
|
|
# --deploy-config vllm_omni/deploy/lance.yaml
|
|
# or programmatically:
|
|
# Omni(model="bytedance-research/Lance",
|
|
# deploy_config_path="vllm_omni/deploy/lance.yaml")
|
|
|
|
pipeline: lance
|
|
async_chunk: false
|
|
|
|
stages:
|
|
- stage_id: 0
|
|
max_num_batched_tokens: 32768
|
|
max_num_seqs: 1
|
|
enforce_eager: true
|
|
trust_remote_code: true
|
|
enable_prefix_caching: false
|
|
devices: "0"
|
|
default_sampling_params:
|
|
seed: 42
|