tzhouam
6c6ff9f27c
Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class.
...
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com >
2025-11-14 18:44:28 +08:00
tzhouam
b8b82347d3
debug ar model runner
...
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com >
2025-11-14 18:44:28 +08:00
tzhouam
62b0e81795
Port changes from feat/multi-request-stream (vs main)
...
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com >
2025-11-14 18:44:28 +08:00
Gaohan123
6e1478222e
Dependency refractored to vLLM v0.11.0
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-04 22:06:41 +08:00
Gaohan123
171f3c1ea7
fix conflicts
2025-10-24 16:49:19 +08:00
tzhouam
28ae542eed
match runner, worker name with file name
2025-10-23 14:35:23 +08:00
tzhouam
53b86439fc
Refactor ARModelRunner to streamline hidden state handling
...
- Removed unnecessary imports of numpy and torch within the request scheduling logic.
- Simplified the handling of auxiliary hidden states by initializing it to None when not present.
- Optimized CPU tensor conversion for text hidden states to improve performance.
2025-10-22 17:16:15 +08:00
tzhouam
6d04ea8450
rename to gpu-xx-model-runner style
2025-10-22 15:28:23 +08:00
tzhouam