lishunyang
69bcc6416d
Adds Lance (bytedance-research/Lance), a 3B unified AR+diffusion multimodal model on a Qwen2.5-VL backbone. Lance is BAGEL-lineage: it reuses vLLM-Omni's BAGEL Mixture-of-Transformers core verbatim and specializes only the ViT (Qwen2.5-VL vision), the VAE (Wan2.2) and the checkpoint layout. All six single-stage Lance modalities verified end-to-end on 1x B300: t2i, t2v, image_edit, video_edit, x2t_image (image understanding) and x2t_video (video understanding). 3-D mRoPE is wired through BagelRotaryEmbedding for the x2t / video / edit paths. Includes the upstream gap-analysis quality fixes (task-specific system prompts, full Qwen chat template, upstream inference defaults and aspect-ratio buckets). Ships an offline example (examples/offline_inference/lance/), an OpenAI-compatible online-serving example (examples/online_serving/lance/) and an e2e test (tests/e2e/online_serving/test_lance.py). See LANCE_PR.md for the architecture rationale, per-modality status and the two remaining follow-ups (two-stage AR+DiT, video_edit polish). Closes #3697. Signed-off-by: lishunyang <lishunyang03@gmail.com>
16 lines
392 B
Bash
Executable File
16 lines
392 B
Bash
Executable File
#!/bin/bash
|
|
# Lance online serving startup script
|
|
|
|
MODEL="${MODEL:-bytedance-research/Lance}"
|
|
PORT="${PORT:-8091}"
|
|
DEPLOY_CONFIG="${DEPLOY_CONFIG:-vllm_omni/deploy/lance.yaml}"
|
|
|
|
echo "Starting Lance server..."
|
|
echo "Model: $MODEL"
|
|
echo "Deploy config: $DEPLOY_CONFIG"
|
|
echo "Port: $PORT"
|
|
|
|
vllm serve "$MODEL" --omni \
|
|
--deploy-config "$DEPLOY_CONFIG" \
|
|
--port "$PORT"
|