Commit Graph
1 Commits
Author SHA1 Message Date
tzhouam 91c11c3ff4 Add AR GPU Model Runner and Worker
- Introduced ARModelRunner for autoregressive GPU model execution, enabling per-request hidden state outputs and token sampling.
- Added ARGPUWorker to initialize GPU settings and manage distributed environment for the AR model runner.
- Enhanced memory management and error handling for GPU resources during initialization.
2025-10-22 15:25:17 +08:00