anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
153 Commits 235 Branches 31 Tags
c0a264452ca74d5c678bfc32e8e87de2ef8d3b60
Commit Graph
153 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Roger Wang c0a264452c update 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-23 19:55:56 -08:00
Roger Wang 35e063e102 update 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-23 19:24:02 -08:00
Roger Wang 89b1bdbb0e add 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-23 19:12:58 -08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Alicia 09fab2ecc9 CI Workflows. (#50) 
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
 2025-11-17 18:13:05 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-17 14:44:10 +08:00
Baoyuan Qi bf82e18374 [fix] adapt hidden state device for multi-hardware support (#61) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-16 17:31:04 +08:00
Samit 8cafb4294d [Docs] Setup Documentation System and Re-organize Dependencies (#49) 
Signed-off-by: SamitHuang <285365963@qq.com>
 2025-11-15 21:45:43 +08:00
Gao Han a089dd4c4e Merge pull request #51 from tzhouam/feat/multi-request-stream-new 
[Feature] Multi Request Stream for Sync Mode
 2025-11-13 10:14:27 +08:00
tzhouam 8f6280f4f6 adjust the doc 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-12 15:51:56 +08:00
tzhouam 26fcae5def Debug and Update doc 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 14:49:45 +08:00
tzhouam c12bd006bc Make Example Easier To Use 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 11:56:08 +08:00
tzhouam bc69d8e931 rename util further rename imports 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:57:15 +08:00
tzhouam 12cc2cd245 rename util 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:55:57 +08:00
tzhouam b943abb22f add data download instructions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:49:53 +08:00
tzhouam edae58b137 modify code based on review suggestions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:03:26 +08:00
tzhouam 5052bfa0b9 Merge branch 'feat/multi-request-stream-new' of https://github.com/tzhouam/vllm-omni into feat/multi-request-stream-new 2025-11-10 16:43:11 +08:00
tzhouam c83d223e91 Squash 7 commits: 36ada0478a 0ef1fefd45 519c140d20 c18fc40f10 5f3f53a39d da385277a2 edefcc244b 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 16:42:58 +08:00
tzhouam edefcc244b Merge branch 'feat/multi-request-stream-new' of https://github.com/tzhouam/vllm-omni into feat/multi-request-stream-new 2025-11-10 16:37:49 +08:00
tzhouam da385277a2 Revert "Revert "Update run script to include initialization sleep time and support for loading prompts from a .pt file."" 
This reverts commit 5f3f53a39d.
 2025-11-10 16:37:37 +08:00
tzhouam 5f3f53a39d Revert "Update run script to include initialization sleep time and support for loading prompts from a .pt file." 
This reverts commit c18fc40f10.
 2025-11-10 16:36:55 +08:00
tzhouam c18fc40f10 Update run script to include initialization sleep time and support for loading prompts from a .pt file. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 16:36:00 +08:00
Isotr0py f1d8a00772 Merge pull request #52 from vllm-project/dev_qby 
[fix] Add support for loading model from a local path
 2025-11-10 16:03:57 +08:00
tzhouam 519c140d20 Update run script to include initialization sleep time and support for loading prompts from a .pt file. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 15:47:28 +08:00
tzhouam 0ef1fefd45 add prompt file 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 15:44:44 +08:00
tzhouam 36ada0478a add back the sleep time for the stage init to avoid single card conflict 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 15:07:01 +08:00
tzhouam 08aef1f964 remove text output for audio output 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 14:41:44 +08:00
tzhouam 8e8783d78a Refactor end-to-end inference script and enhance argument parsing. Added support for loading prompts from a .pt file, introduced new command-line arguments for initialization and output handling, and improved error handling for prompt loading. Removed deprecated files related to previous implementations. 2025-11-10 11:58:47 +08:00
齐保元 d7618b1be4 [fix] support loading model from local path 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-09 04:37:19 +08:00
tzhouam c5631ca753 Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class. 2025-11-07 18:05:00 +08:00
tzhouam 3c2ad1bf12 debug ar model runner 2025-11-07 16:25:48 +08:00
tzhouam 6a7a27801b Port changes from feat/multi-request-stream (vs main) 2025-11-07 15:48:47 +08:00
Hongsheng Liu 86841b69e9 Merge pull request #48 from Gaohan123/refractor_v2 
[Refractor] Dependency refractored to vLLM v0.11.0
 2025-11-07 15:47:53 +13:00
Gaohan123 e5adae0a07 fix other reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-06 19:25:55 +08:00
Hongsheng Liu 5cc8ab06db Merge pull request #47 from Gaohan123/refractor 
[Bugfix] fix huggingface download problem for spk_dict.pt
 2025-11-06 16:03:31 +13:00
Gaohan123 1aa5ae61cb fix reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 23:26:58 +08:00
Gaohan123 e2b803fbe6 fix a bug 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 13:05:12 +08:00
Gaohan123 c80348ce71 supplement some new infomation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 10:47:30 +08:00
Gaohan123 5b683c75de supplement some infomation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 10:46:05 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 d4a364f052 fix huggingface download problem for spk_dict.pt 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-01 16:45:34 +08:00
Zhou Taichang 6d34a3ed11 Merge pull request #46 from Gaohan123/fix_install_v2 
[Bugfix] Further supplement installation guide
 2025-10-30 17:25:14 +08:00
Gaohan123 03ae52a064 add installation of uv 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 17:18:19 +08:00
Gaohan123 1702033c3e fix some typos 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 17:16:25 +08:00
Gaohan123 27f06e27fc Further supplement installation guide 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 14:45:54 +08:00
Hongsheng Liu aa2aa52a82 Merge pull request #44 from Gaohan123/fix_install 
[bugfix] fix problem of installation
 2025-10-27 22:11:33 +08:00
Gaohan123 78830c8ad9 fix problem of installation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-27 22:06:35 +08:00
Hongsheng Liu 0b56cd3d5b Merge pull request #36 from Gaohan123/remove_unused 
Remove unused files and fix some bugs
 2025-10-25 23:21:52 +08:00
Gaohan123 c19e4d26d1 fix requirements 2025-10-25 18:02:16 +08:00
Gaohan123 9b6ce0d245 fix typo 2025-10-25 18:02:16 +08:00