Files
vllm-omni/vllm_omni/entrypoints
tzhouamandClaude Opus 4.8 f18bb69b7d fix(entrypoints): pass ChatTemplateConfig to pooling serving classes for vLLM 0.25
vLLM 0.25 reworked the pooling serving base: PoolingServing.__init__ now
requires a `chat_template_config: ChatTemplateConfig` keyword and no longer
accepts the old `chat_template` / `chat_template_content_format` /
`trust_request_chat_template` kwargs. Build a ChatTemplateConfig once and pass
it to the embedding / pooling / classification / score / responses serving
objects. ServingTokenization keeps the flat kwargs (unchanged upstream).

Without this, omni_init_app_state crashes at startup (missing required arg /
unexpected kwargs) whenever an embed/classify/score task is served.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:21:18 +08:00
..