anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,955 Commits 235 Branches 31 Tags
b36a3431bda13b53dfe74d9d2e5617ec9f86e5b7
Commit Graph
56 Commits
Author SHA1 Message Date
tzhouamandClaude Opus 5 b36a3431bd Package this fork as vllm-embedding: README, docs, roadmap, harness 
Turns a branch of accumulated edge/CPU work into something someone else can
read, reproduce and continue.

README.md states what the fork is for (multimodal models on edge CPUs rather
than servers), the headline memory and speed results with their costs, and why
the numbers are worth reading -- five retracted conclusions, all measurement
errors rather than coding errors. Upstream's README is preserved at
docs/README.vllm-omni.md and NOTICE records the Apache-2.0 derivation and the
files this fork changes.

docs/edge/:
  methodology.md  the measurement discipline, which is the most transferable
                  thing here: what each of the five retractions was, and the
                  rule it left behind. Includes the two traps specific to this
                  stack -- vLLM's AOT compile cache keying on config so two
                  arms sharing an artifact load the wrong graph, and allocated
                  not being resident until mincore(2) says so.
  results.md      every measured figure with its spread and baseline, and an
                  explicit note that the four memory levers overlap and must
                  not be added.
  cpu-4bit.md     the two 4-bit paths, the packed layout as measured rather
                  than assumed, why the path choice is a config field, and the
                  GPTQ output-head trap that invalidated a whole round of W4A8
                  numbers.
  harness.md      how to run a comparison, and the two arguments that matter
                  more than they look (--context-len, --prompt-len), each of
                  which has produced a wrong conclusion by being left at its
                  default.

ROADMAP.md orders by whether the verification method is correct rather than by
expected size, lists what is done with results, what is next with expected
gains and how each would be verified, and -- at equal length -- what was
considered and rejected, including the CPU megakernel with the measurements
that rule it out for now.

benchmarks/edge_harness/ brings the tooling into the repo so the numbers are
reproducible: the grid runner, the single-run benchmarks, the memory ledger,
the fidelity metric, the Android agent suite, the quantizers, and the tests
that pin the harness's judgement. 82 tests pass from the new location.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L1ax5wYLA6XbqEL1i8L9gZ
 2026-09-14 13:25:32 +08:00
chickeytonClaude Fable 5Gao Han
 		5d7fdf9350 [Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-05 08:20:21 +00:00
Hongsheng Liu d1c8beeb18 [Doc] Update README and docs for v0.28.0 (#6858) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-01 19:50:26 +08:00
Alicia ea0a24465b [Docs] Restore the Star History chart (#6446) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-08-21 01:35:03 -07:00
Samit 8a5bf2fb3f [doc]: Update latest news section for verl-omni release (#6187) 
Signed-off-by: Samit <285365963@qq.com>
 2026-08-14 12:04:15 +08:00
Hongsheng Liu 9159cedbd7 docs: update vLLM-Omni architecture overview (#5914) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-09 09:16:18 +08:00
Hongsheng Liu 30cba84222 [Doc] Update README and installation docs for v0.26.0 (#5715) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-04 14:18:51 +08:00
Hongsheng Liu a6da469134 docs: update README for v0.24.0 (#5060) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-07-13 08:26:37 +08:00
Yueqian Lin 208932210f [Chore][TTS] Reference recipes.vllm.ai recipes, remove dead code, and fix stale references (#4567) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-06-19 15:05:26 +08:00
Hongsheng Liuhsliu_ustcClaude Opus 4.7
 		39ae6e12f8 [Docs] Update README supported models section with TTS and Diffusion categories (#4300) 
Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-10 19:40:50 +08:00
Samit 6ddaf6188d Fix README.md typo (#4292) 2026-06-09 18:08:09 +08:00
Hongsheng LiuYueqian LinClaude Opus 4.7
 		5de3b11822 docs: update README and supported models for v0.22.0 release (#4233) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-09 14:39:19 +08:00
Hongsheng Liuanddeepseek-v4-pro c0e132d973 [Doc] Reorganize available recipes into a table (#3671) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Co-authored-by: deepseek-v4-pro <noreply@anthropic.com>
 2026-05-18 00:02:34 +08:00
Hongsheng Liuandhsliu_ustc c0b7509f07 update v0.20.0 readme (#3594) 
Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
 2026-05-14 17:23:55 +08:00
ChenWenjing fe6cec6ba3 [doc][skip ci] remove redundant content in readme (#2901) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-04-18 18:18:12 +08:00
ChenWenjing 4124a1f858 [Feature]Support audio streaming input and output-phase2 (#2581) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-04-18 15:42:41 +08:00
Hongsheng Liu a5bf33f25a [skip ci] update release 0.18.0 (#2380) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-03-31 20:45:07 +08:00
Roger Wang 6a5fa588c6 Update latest news section in README.md (#1909) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-03-15 22:50:42 -07:00
Hongsheng Liuandhsliu_ustc ff3365f6a2 [skip ci] update readme slides link (#1850) 
Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
 2026-03-12 20:27:42 +08:00
Alicia 09b66bf6db [Doc][skip ci] Update readme with Video link for vLLM HK First Meetup (#1833) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-12 16:06:36 +13:00
Hongsheng LiuClaude Opus 4.6hsliu_ustc
 		bb051ab1df [skip ci] Update slides link (#1730) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
 2026-03-08 20:45:54 +08:00
Hongsheng Liu b84d73be4b docs: Announce vllm-omni-skills community project (#1651) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
 2026-03-04 20:56:50 +08:00
Roger Wang fec0182ac2 0.16.0 release (#1576) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-28 21:28:27 -08:00
Alicia efce272cb4 [Doc] Highlight paper & slides. (#1186) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-02-04 14:08:49 +08:00
Hongsheng Liuandhsliu_ustc 3fb3c8aba4 Docs update paper link (#1169) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
 2026-02-03 13:19:36 +08:00
Roger Wang de2cac95e8 [Doc] First stable release of vLLM-Omni (#1129) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-01 00:21:57 -08:00
Zhou Taichang d2a72f380f [Doc] Update rebase doc (#878) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-21 03:59:02 +00:00
+13
Zhou TaichangJaredforRealrootJustQJXU MingshiSihyeon JangJiangyun ZhucatcatZiming HuangYuhan LiuAliciaSamitYueqian LinWeiQing ChenPeiqi YinDidan DengHongsheng Liuwangyuwangyu31577汪志鹏John Liu BUAAD!NE$Hroot
 		776c3a794e Dev/rebase 0.14.0 (#847) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-21 00:25:25 +08:00
WeiQing Chen 4203bfa350 [Misc] add wechat group and star history on README (#801) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-01-16 03:22:16 +13:00
Hongsheng Liu 65b55d1c83 update design doc (#711) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
 2026-01-09 13:30:17 +08:00
Roger Wang aa5b628b6a [Doc] Update version 0.12.0 (#662) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-01-06 14:34:15 +08:00
Roger Wang 9fe730a44b [Misc] update image reference for PyPI (#150) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-12-01 09:37:29 -08:00
Roger Wang 510aeb7da4 [Misc] Misc description updates (#146) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-12-01 12:09:24 +00:00
Roger Wang a6c5a6aca4 [Doc] Update developer & user channel (#138) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-12-01 00:25:44 -08:00
Hongsheng LiuandRoger Wang 0c245dd1f9 [Doc] Fix readme typos (#108) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-11-30 21:32:04 +08:00
Roger WangandGao Han a5800cd2c2 [Doc] Misc documentation polishing (#98) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-11-29 20:42:39 +08:00
Gao Han db81ee9f69 [Docs] optimize and supplement docs system (#86) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-28 11:03:52 +08:00
Hongsheng LiuandRoger Wang 1fb46ddd62 update readme for v0.11.0rc1 release (#69) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-11-24 14:28:25 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-17 14:44:10 +08:00
Samit 8cafb4294d [Docs] Setup Documentation System and Re-organize Dependencies (#49) 
Signed-off-by: SamitHuang <285365963@qq.com>
 2025-11-15 21:45:43 +08:00
Gaohan123 1aa5ae61cb fix reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 23:26:58 +08:00
Gaohan123 5b683c75de supplement some infomation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 10:46:05 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 03ae52a064 add installation of uv 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 17:18:19 +08:00
Gaohan123 1702033c3e fix some typos 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 17:16:25 +08:00
Gaohan123 27f06e27fc Further supplement installation guide 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 14:45:54 +08:00
Gaohan123 78830c8ad9 fix problem of installation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-27 22:06:35 +08:00
Gaohan123 9b6ce0d245 fix typo 2025-10-25 18:02:16 +08:00
Gaohan123 6cf73c4e80 Remove unused files and fix some bugs 2025-10-25 18:02:11 +08:00