anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
888 Commits 235 Branches 31 Tags
4914466f0bcacf7cfe25b5846e2edd561d97f019
Commit Graph
888 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam 4914466f0b Update Dockerfile.ci to install vLLM wheel version 3dd142adb7f7fa3b0b8345ee046ba36a12d9e530 and remove ExtractHiddenStatesProposer references from GPU model runners for cleaner code. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 15:09:26 +00:00
tzhouam 2398a7c1d7 Merge origin/main into dev/rebase-v0170 
Resolve conflict in omni_generation_scheduler.py: keep both
num_cached_tokens tracking and additional_information caching.

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 14:31:05 +00:00
Junhong Liu 742d845b8e Make chunk_size and left_context_size configurable via YAML for async chunking (#1423) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
 2026-03-03 22:06:52 +08:00
tzhouam 16824aa295 Add max_query_len parameter to _LocalPredictorKVCache for improved query handling 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 14:04:07 +00:00
Yuanheng Zhao b13761ed26 [Bugfix] Fix filepath resolution for model with subdir and GLM-Image generation (#1609) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-03-03 23:40:53 +13:00
tzhouam 2da9d28f1b Enhance reproducibility in Ulysses attention tests by replacing current_omni_platform.seed_everything with torch's manual_seed and adding support for CUDA devices. Introduce get_supported_tasks method in AsyncOmni class to dynamically determine available tasks based on stage capabilities. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 09:45:51 +00:00
tzhouam 4d8a76e721 Reintroduce installation of flashinfer-jit-cache in Dockerfile.ci to enhance dependency management for the CI environment, ensuring compatibility with the latest requirements. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 09:23:09 +00:00
dengyunyang d275c0c993 [FEATURE] Tile/Patch parallelism refactor for easily support other models (#1366) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-03-03 15:48:51 +08:00
Alicia e7f2c88701 [CI] Add scripts for bechmark collection and email distribution. (#1307) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-02 23:00:33 -08:00
tzhouam a17078974f Remove flashinfer-jit-cache installation from Dockerfile.ci to streamline dependency management and avoid potential conflicts. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 07:00:18 +00:00
tzhouam 8dcf1061f1 Update Dockerfile.ci to install additional dependencies including flashinfer-jit-cache, nvidia-cublas-cu12, and numpy, ensuring a more comprehensive environment setup for development. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 06:47:10 +00:00
Yuanheng Zhao 3b2b5012eb [Doc] Fix links in the configuration doc (#1615) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-03-03 04:52:51 +00:00
Zhou Taichang 90a48f4444 [Debug] Enable curl retry aligned with openai (#1539) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 11:56:42 +08:00
tzhouam 98926707f5 rebase 2026/3/3 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 03:50:34 +00:00
zhanqiuhu 80c89e00cb [Bugfix] Fix Qwen3-TTS code predictor crash due to missing vLLM config context (#1619) 
Signed-off-by: Zhanqiu <zh338@cornell.edu>
 2026-03-03 11:02:13 +08:00
tzhouam 3e539fcb6a Merge remote-tracking branch 'origin/main' into dev/vllm-align 
Made-with: Cursor

# Conflicts:
#	vllm_omni/entrypoints/openai/serving_chat.py
#	vllm_omni/worker/gpu_model_runner.py

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:57:27 +00:00
tzhouam e0497550c4 Update Dockerfile.ci to install a specific version of flashinfer-cubin (0.6.4) directly, simplifying the installation process and ensuring compatibility with the current setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:51:35 +00:00
tzhouam 029bb56e3a Update Dockerfile.ci to use importlib.metadata for retrieving the current version of flashinfer, enhancing compatibility with the latest Python standards. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:28:24 +00:00
tzhouam 65169baa9d Update Dockerfile.ci to create a symbolic link for Python 3, ensuring compatibility with Python scripts during the build process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 01:58:35 +00:00
Alex BrooksandSYLAR a09b6499f8 [Bugfix][Model] Fix LongCat Image Config Handling / Layer Creation (#1485) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-03-03 00:50:08 +08:00
Samit 22a51a7cdb [Feature][Wan2.2] Speed up diffusion model startup by multi-thread weight loading (#1504) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-03-02 14:53:25 +00:00
rongfu.leng a4b2e48718 [Bugfix] Import InputPreprocessor into Renderer (#1566) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-02 21:54:15 +08:00
Baoyuan Qi f371f7bf05 [bugfix] Fix unexpected argument 'is_finished' in function llm2code2wav_async_chunk of mimo-audio (#1570) 
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2026-03-02 21:49:06 +08:00
rein yang f082f6eedb [Bugfix] fix kernel error for qwen3-omni (#1602) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-03-02 18:12:30 +08:00
Ting FU 1ca198eaff [BugFix] Fix load_weights error when loading HunyuanImage3.0 (#1598) 
Signed-off-by: Semmer2 <semmer@live.cn>
 2026-03-02 08:40:48 +00:00
tzhouam 908e3fe761 Update Dockerfile.ci to dynamically install the current version of flashinfer-cubin, ensuring compatibility with the latest changes in the flashinfer package. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 07:00:59 +00:00
tzhouam 8fb247a481 Update Dockerfile.ci to install specific version of flashinfer-cubin (0.6.4) and ensure vllm installation remains intact. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 06:48:58 +00:00
tzhouam be4226bae0 Remove unnecessary uninstallation of nvidia-cublas-cu12 and streamline vllm installation in Dockerfile.ci 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 06:38:24 +00:00
汪志鹏andCursor e37a89ffcb [Feature]: Support cfg kv-cache transfer in multi-stage (#1422) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-03-02 05:06:28 +00:00
tzhouam 473d3ecd29 Update vLLM wheel version and adjust GPU memory utilization in CI configuration 
- Updated the vLLM wheel commit hash in Dockerfile.ci to a60985b07eaf0fefa4c353ad2309b745707591ed.
- Modified the mock shared memory handle name in test_basic_connectors.py for clarity.
- Increased GPU memory utilization to 0.9 and adjusted max_num_batched_tokens to 4096 in qwen2_5_omni_ci.yaml to enable testing on H800.
- Refactored GPU model runner classes to remove deprecated activate_lora parameter and ensure compatibility with num_active_loras.
- Enhanced request handling in the OmniGPUModelRunner class to improve streaming request updates.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 03:08:45 +00:00
dsinghviandZhou Taichang 68764cc4f4 [Refactor]: Phase1 for rebasing_additional_info (#1394) 
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-02 10:19:41 +08:00
tzhouam 26744bcb35 Merge origin/main into dev/vllm-align 
Resolved conflicts:
- vllm_omni/inputs/preprocess.py: took origin/main's elif mm_processor_kwargs branch, fixed undefined mm_uuids reference
- vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: took origin/main's NPU support

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 02:10:32 +00:00
Roger Wang fec0182ac2 0.16.0 release (#1576) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-28 21:28:27 -08:00
汪志鹏 3d9fa8d53f [BugFix]: fix a lot of bug (#1565) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
v0.16.0 		2026-02-28 16:26:56 +08:00
wangyu cd2234a1bf [CI] Modify some CI test cases to run on L4 environment to reduce H100 resource usage. (#1543) 
Signed-off-by: yenuo26 <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
 2026-02-28 14:41:51 +08:00
Ting FU b0156d8e1c [BugFix] Fix unexpected crash when init OmniDiffusion (#1562) 
Signed-off-by: Semmer2 <semmer@live.cn>
 2026-02-28 14:21:49 +08:00
Canlin Guo c81266703a [NPU][Bugfix] Align GPU side and recover qwen3-tts (#1564) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-28 12:46:13 +08:00
Yueqian Lin 37775ffe60 fix(qwen3-tts): fix Base ICL voice clone producing corrupted audio (#1554) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-02-28 11:49:47 +08:00
rein yang bfcfee7867 [Docs] update async chunk docs diagram [skip ci] (#1530) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-02-28 10:42:50 +08:00
Zhou Taichang fe0c02ee78 [Debug] Merge vllm pull 35368 (#1534) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-28 10:39:25 +08:00
Junhong Liu 42d86af1f1 Fix no embed text spk tokens (#1540) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-02-28 09:33:58 +08:00
Alicia 11f59f600d [CI][skip ci]Update H100 image link based on #1518 (#1538) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-02-27 18:25:18 +08:00
AliciaandYueqian Lin 64f5ef615c [Doc] [skip ci]Sync. (#1363) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-02-27 17:28:35 +08:00
Zhou Taichang bc13f7cf38 [Doc] Update installation instructions for vllm 0.16.0 (#1505) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-27 16:06:30 +08:00
Isotr0py 1f862ce04f [Chore] Cleanup dead code in GGUF DiT code path (#1533) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-27 07:17:27 +00:00
Chendi.Xueandgcanlin 482f9b89a1 [XPU][NPU][ROCM] enable cpu_offloading flag for non_cuda (#1488) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-27 14:52:15 +08:00
SHIJIN ZHANGand齐保元 ba9594724e [Bugfix] Fix redundant finished req status updating on OmniGenerationScheduler (#1510) 
Signed-off-by: shijin zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: 齐保元 <qibaoyuan@xiaomi.com>
 2026-02-27 14:42:38 +08:00
Canlin GuoHongsheng LiuNicolò Lucchesi
 		eaef3a31a4 [Profiler] Support online profiling (#1136) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Canlin Guo <961750412@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2026-02-27 14:41:20 +08:00
Sy03 0cc3481acf [Feat] : Support Async chunk cleanup (#1087) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-02-27 14:39:33 +08:00
TJian 7ac8d613f6 [Version] Auto generate version using setuptool_scm (#1224) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-02-27 14:34:27 +08:00