anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
134 Commits 235 Branches 31 Tags
c10cbbd2ea63bdfb8c451d769a69a711c6ed0b33
Commit Graph
134 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam c10cbbd2ea add data download instructions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:29 +08:00
tzhouam 7b9fa7708a modify code based on review suggestions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:29 +08:00
tzhouam a42dba230d Revert "Revert "Update run script to include initialization sleep time and support for loading prompts from a .pt file."" 
This reverts commit 5f3f53a39d.

Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:29 +08:00
tzhouam dfe92f7d86 Revert "Update run script to include initialization sleep time and support for loading prompts from a .pt file." 
This reverts commit c18fc40f10.

Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:29 +08:00
tzhouam 86b8669494 Squash 7 commits: 36ada0478a 0ef1fefd45 519c140d20 c18fc40f10 5f3f53a39d da385277a2 edefcc244b 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
tzhouam b5136c2e0d remove text output for audio output 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
tzhouam 13dfb56a68 Refactor end-to-end inference script and enhance argument parsing. Added support for loading prompts from a .pt file, introduced new command-line arguments for initialization and output handling, and improved error handling for prompt loading. Removed deprecated files related to previous implementations. 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
tzhouam 6c6ff9f27c Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class. 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
tzhouam b8b82347d3 debug ar model runner 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
tzhouam 62b0e81795 Port changes from feat/multi-request-stream (vs main) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
齐保元 8f7b102108 [fix] support loading model from local path 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
齐保元 5c038ae553 [feature] cpu worker support 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
齐保元 8b30818f2e [feature] cpu worker support 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
Hongsheng Liu 86841b69e9 Merge pull request #48 from Gaohan123/refractor_v2 
[Refractor] Dependency refractored to vLLM v0.11.0
 2025-11-07 15:47:53 +13:00
Gaohan123 e5adae0a07 fix other reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-06 19:25:55 +08:00
Hongsheng Liu 5cc8ab06db Merge pull request #47 from Gaohan123/refractor 
[Bugfix] fix huggingface download problem for spk_dict.pt
 2025-11-06 16:03:31 +13:00
Gaohan123 1aa5ae61cb fix reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 23:26:58 +08:00
Gaohan123 e2b803fbe6 fix a bug 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 13:05:12 +08:00
Gaohan123 c80348ce71 supplement some new infomation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 10:47:30 +08:00
Gaohan123 5b683c75de supplement some infomation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 10:46:05 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 d4a364f052 fix huggingface download problem for spk_dict.pt 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-01 16:45:34 +08:00
Zhou Taichang 6d34a3ed11 Merge pull request #46 from Gaohan123/fix_install_v2 
[Bugfix] Further supplement installation guide
 2025-10-30 17:25:14 +08:00
Gaohan123 03ae52a064 add installation of uv 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 17:18:19 +08:00
Gaohan123 1702033c3e fix some typos 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 17:16:25 +08:00
Gaohan123 27f06e27fc Further supplement installation guide 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 14:45:54 +08:00
Hongsheng Liu aa2aa52a82 Merge pull request #44 from Gaohan123/fix_install 
[bugfix] fix problem of installation
 2025-10-27 22:11:33 +08:00
Gaohan123 78830c8ad9 fix problem of installation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-27 22:06:35 +08:00
Hongsheng Liu 0b56cd3d5b Merge pull request #36 from Gaohan123/remove_unused 
Remove unused files and fix some bugs
 2025-10-25 23:21:52 +08:00
Gaohan123 c19e4d26d1 fix requirements 2025-10-25 18:02:16 +08:00
Gaohan123 9b6ce0d245 fix typo 2025-10-25 18:02:16 +08:00
Gaohan123 fed8785eb9 fix some docs 2025-10-25 18:02:16 +08:00
Gaohan123 6cf73c4e80 Remove unused files and fix some bugs 2025-10-25 18:02:11 +08:00
Hongsheng Liu 6248bbf646 Merge pull request #35 from tzhouam/end2end_fixup 
End2end fixup
 2025-10-25 17:04:42 +08:00
tzhouam f6158e7576 debug 2025-10-24 18:01:02 +08:00
tzhouam 605d1de013 Update entry points in vllm_omni module 
- Reintroduced AsyncOmniLLM to the main entry points.
- Added import for patch module.
- Cleaned up comments regarding AsyncOmniLLM support.
 2025-10-24 17:54:19 +08:00
tzhouam d45f8a6be7 debug typo 2025-10-24 17:47:49 +08:00
tzhouam 61b344dc39 Add import for OmniModelsRegistry in model executor initialization 2025-10-24 17:44:31 +08:00
tzhouam 3b1324ea39 Merge branch 'main' into end2end_fixup 2025-10-24 17:41:03 +08:00
tzhouam 3265e857eb Refactor OmniLLM and MultimodalOutputProcessor initialization 
- Removed AsyncOmniLLM from the main entry points as it is not supported yet.
- Updated the constructor of MultimodalOutputProcessor to set a default value for engine_core_output_type.
- Adjusted output type handling in various methods to utilize the new default engine_core_output_type for consistency.
 2025-10-24 17:15:34 +08:00
Hongsheng Liu a884dddc89 Merge pull request #32 from Gaohan123/precommit_set 
add precommit and solve conflict
 2025-10-24 17:04:49 +08:00
Gaohan123 171f3c1ea7 fix conflicts 2025-10-24 16:49:19 +08:00
Gaohan123 22f36de34e fix error 2025-10-24 16:21:52 +08:00
Gaohan123 eafc7c44c3 fix error for pre-commit 2025-10-24 16:21:50 +08:00
Gaohan123 a04e945166 add precommit 2025-10-24 16:20:58 +08:00
Hongsheng Liu 1bf3424e96 Merge pull request #19 from tzhouam/feat/omni-schedulers 
[Core]Add scheduling components for vLLM-omni
 2025-10-24 14:08:57 +08:00
Hongsheng Liu 9e79a9a9b1 Merge pull request #18 from tzhouam/feat/hidden-state-input-request-input-processor-patch 
[Inputs, Engine]Add Omni model components and input processing for hidden states support
 2025-10-24 14:05:36 +08:00
tzhouam eb006f66e5 Enhance OmniScheduler error handling with logging 
- Added logging for exceptions when wrapping scheduled requests in OmniScheduler.
- Removed unnecessary blank lines for cleaner code structure.
 2025-10-24 12:00:36 +08:00
tzhouam 87b3ae7e70 Merge branch 'feat/omni-schedulers' of https://github.com/tzhouam/vllm-omni into feat/omni-schedulers 2025-10-24 11:57:45 +08:00
tzhouam c23d24ca19 Refactor comments in diffusion_scheduler.py for clarity and consistency 
- Translated comments from Chinese to English for better accessibility.
- Improved clarity of comments related to request handling and scheduling logic.
- Ensured comments align with the code functionality and maintain a uniform style.
 2025-10-24 11:57:39 +08:00