Jiangyun Zhu
5707fc78d5
[CI] fix ci and format existing code ( #71 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2025-11-18 20:54:26 +08:00
Gao Han
c5a89302f1
[Feature] Support online inference ( #64 )
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-17 14:44:10 +08:00
tzhouam
c5631ca753
Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class.
2025-11-07 18:05:00 +08:00
tzhouam
3c2ad1bf12
debug ar model runner
2025-11-07 16:25:48 +08:00
tzhouam
6a7a27801b
Port changes from feat/multi-request-stream (vs main)
2025-11-07 15:48:47 +08:00
Gaohan123
6e1478222e
Dependency refractored to vLLM v0.11.0
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-04 22:06:41 +08:00
Gaohan123
171f3c1ea7
fix conflicts
2025-10-24 16:49:19 +08:00
tzhouam
28ae542eed
match runner, worker name with file name
2025-10-23 14:35:23 +08:00
tzhouam
53b86439fc
Refactor ARModelRunner to streamline hidden state handling
...
- Removed unnecessary imports of numpy and torch within the request scheduling logic.
- Simplified the handling of auxiliary hidden states by initializing it to None when not present.
- Optimized CPU tensor conversion for text hidden states to improve performance.
2025-10-22 17:16:15 +08:00
tzhouam
6d04ea8450
rename to gpu-xx-model-runner style
2025-10-22 15:28:23 +08:00
Jiangyun Zhu