Merge branch 'main' into codex-hwr-domain-timeout

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

.buildkite/cuda/test-merge.yml

@@ -155,6 +155,7 @@ steps:
         source_file_dependencies:
           - tests/e2e/offline_inference/test_minicpmo_4_5.py
           - tests/e2e/online_serving/test_minicpmo_4_5.py
+          - vllm_omni/transformers_utils/chat_templates/minicpmo45_native.jinja
           - vllm_omni/model_executor/models/minicpmo_4_5/
           - vllm_omni/model_executor/models/step_audio2/
           - vllm_omni/model_executor/models/cosyvoice3/code2wav_core/

.buildkite/cuda/test-ready.yml

@@ -188,6 +188,7 @@ steps:
       - label: "Omni · MiniCPM-o 4.5 Online Test"
         source_file_dependencies:
           - tests/e2e/online_serving/test_minicpmo_4_5.py
+          - vllm_omni/transformers_utils/chat_templates/minicpmo45_native.jinja
           - vllm_omni/model_executor/models/minicpmo_4_5/
           - vllm_omni/model_executor/models/step_audio2/
           - vllm_omni/model_executor/models/cosyvoice3/code2wav_core/

docs/design/feature/host_weight_runtime.md

@@ -313,8 +313,15 @@ One process per exact identity owns a build; other workers wait and then acquire
 leases for the published artifact. Publication is invisible until all payloads
 and metadata are validated, hashed, fsynced, and atomically renamed.
 
-`coordination_timeout_seconds` bounds filesystem lock acquisition. It does not
-cancel synchronous validation, a producer that has already started, or atomic
+`coordination_timeout_seconds` bounds domain-initialization and lookup/build
+lock acquisition. Store construction and each later resolution or publication
+operation have separate budgets from the same wait policy, rather than one
+end-to-end startup deadline. A domain-init timeout follows the retryable domain
+failure policy below. After contention ends, a fresh runtime construction can
+retry initialization; the timed-out runtime retains its original failure.
+
+The coordination budget does not cancel filesystem I/O, synchronous validation,
+a producer that has already started, or atomic
 publication. A hung in-process producer therefore blocks its owning process and
 must be handled by external process supervision. Enforceable producer
 cancellation requires a future process-isolated producer contract.

docs/design/module/host_weight_runtime.md

@@ -136,8 +136,16 @@ so only when it is marked retryable; unsupported capabilities, identity
 collisions, producer failures, and publication failures remain visible as
 startup failures.
 
-`coordination_timeout_seconds` bounds acquisition of lookup and build locks; it
-is not a hard wall-clock deadline for in-process validation, production, or
+`coordination_timeout_seconds` bounds acquisition of the domain-initialization
+lock and lookup/build locks. Store construction and each later resolution or
+publication operation receive separate coordination budgets from the same
+`WaitPolicy`; they do not share an end-to-end startup deadline. A domain-init
+lock timeout is a retryable domain failure: preferred mode may fall back and
+required mode fails. Retrying initialization requires constructing a fresh
+runtime after contention ends; a waiter never cancels or unlocks the owner.
+
+The coordination budget is not a hard wall-clock deadline for filesystem I/O,
+in-process validation, production, or
 atomic publication. Once this V1 implementation becomes the producer, the
 synchronous producer and publication run to completion. A hung producer blocks
 its owning process and requires external process supervision; enforceable

pyproject.toml

@@ -161,7 +161,7 @@ where = ["."]
 include = ["vllm_omni*"]
 
 [tool.setuptools.package-data]
-"vllm_omni" = ["_version.py", "py.typed"]
+"vllm_omni" = ["_version.py", "py.typed", "transformers_utils/chat_templates/*.jinja"]
 
 [tool.setuptools_scm]
 # no extra settings needed, presence enables setuptools-scm

recipes/OpenBMB/MiniCPM-o-4_5.md

@@ -28,13 +28,13 @@ also provided.
 
 - Default deploy configs (auto-loaded by HF `model_type=minicpmo` +
   `hf_config.version="4.5"`):
-  - Default single-GPU compatibility layout (auto-loaded):
+    - Default single-GPU compatibility layout (auto-loaded):
     [`vllm_omni/deploy/minicpmo_4_5.yaml`](../../vllm_omni/deploy/minicpmo_4_5.yaml)
-  - Recommended 2-GPU continuous-batching layout:
+    - Recommended 2-GPU continuous-batching layout:
     [`vllm_omni/deploy/minicpmo_4_5_2gpu.yaml`](../../vllm_omni/deploy/minicpmo_4_5_2gpu.yaml),
-  - 3-GPU layout:
+    - 3-GPU layout:
     [`vllm_omni/deploy/minicpmo_4_5_3gpu.yaml`](../../vllm_omni/deploy/minicpmo_4_5_3gpu.yaml)
-  - 8x RTX 4090 layout:
+    - 8x RTX 4090 layout:
     [`vllm_omni/deploy/minicpmo_4_5_8x4090.yaml`](../../vllm_omni/deploy/minicpmo_4_5_8x4090.yaml)
 - Online example + Gradio demo:
   [`examples/online_serving/minicpmo/`](../../examples/online_serving/minicpmo/)
@@ -126,9 +126,20 @@ video. Use the two-GPU profile for production throughput.
 ```bash
 vllm serve openbmb/MiniCPM-o-4_5 --omni \
     --trust-remote-code \
+    --chat-template vllm_omni/transformers_utils/chat_templates/minicpmo45_native.jinja \
+    --chat-template-content-format openai \
     --host 0.0.0.0 --port 8099
 ```
 
+Run these commands from the repository root. The bundled chat template
+matches the Hugging Face `chat(omni_mode=True)` content layout: media and text
+parts are concatenated without adding separators, while whitespace explicitly
+included in a text part is preserved. This matters for speech generation because
+an inserted newline changes the Thinker condition passed to Talker. For image
+questions, place the image before the question, as in the native image-chat
+layout. Continue to request `use_tts_template: true` and
+`enable_thinking: false` for spoken answers.
+
 The deploy config is auto-loaded by the model registry — no
 `--deploy-config` flag needed for this default single-GPU layout.
 
@@ -271,6 +282,8 @@ uses GPU 5. GPUs 6–7 are left free.
 vllm serve openbmb/MiniCPM-o-4_5 --omni \
     --deploy-config vllm_omni/deploy/minicpmo_4_5_8x4090.yaml \
     --trust-remote-code \
+    --chat-template vllm_omni/transformers_utils/chat_templates/minicpmo45_native.jinja \
+    --chat-template-content-format openai \
     --host 0.0.0.0 --port 8099
 ```
 

tests/e2e/online_serving/test_minicpmo_4_5.py

@@ -24,6 +24,9 @@ from tests.helpers.stage_config import get_deploy_config_path, modify_stage_conf
 os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
 
 _MODEL = "openbmb/MiniCPM-o-4_5"
+_NATIVE_CHAT_TEMPLATE = (
+    Path(__file__).resolve().parents[3] / "vllm_omni/transformers_utils/chat_templates/minicpmo45_native.jinja"
+)
 _CI_DEPLOY = modify_stage_config(
     get_deploy_config_path("minicpmo_4_5.yaml"),
     updates={
@@ -63,7 +66,14 @@ test_params = [
             model=_MODEL,
             stage_config_path=_CI_DEPLOY,
             use_stage_cli=False,
-            server_args=["--trust-remote-code", "--async-chunk"],
+            server_args=[
+                "--trust-remote-code",
+                "--async-chunk",
+                "--chat-template",
+                str(_NATIVE_CHAT_TEMPLATE),
+                "--chat-template-content-format",
+                "openai",
+            ],
             env_dict={
                 "VLLM_CONFIGURE_LOGGING": "1",
                 "VLLM_LOGGING_CONFIG_PATH": str(_PROMPT_LOGGING_CONFIG_PATH),
@@ -265,11 +275,18 @@ def test_image_to_text_audio_001(omni_server, openai_client) -> None:
     Input Setting: stream=True
     """
     image_data_url = f"data:image/jpeg;base64,{generate_synthetic_image(24, 24)['base64']}"
-    messages = dummy_messages_from_mix_data(
-        system_prompt=get_system_prompt(),
-        image_data_url=image_data_url,
-        content_text=get_prompt("text_image"),
-    )
+    # Match the native image-chat layout explicitly; do not rely on the
+    # renderer moving media placeholders ahead of text.
+    messages = [
+        get_system_prompt(),
+        {
+            "role": "user",
+            "content": [
+                {"type": "image_url", "image_url": {"url": image_data_url}},
+                {"type": "text", "text": get_prompt("text_image")},
+            ],
+        },
+    ]
 
     request_config = {
         "model": omni_server.model,

tests/host_weight_runtime/test_runtime_resolution.py

@@ -4,6 +4,10 @@
 
 from __future__ import annotations
 
+import multiprocessing as mp
+import threading
+import time
+from multiprocessing.connection import Connection
 from pathlib import Path
 from typing import cast
 
@@ -15,6 +19,7 @@ from vllm_omni.host_weight_runtime import (
     CanonicalJson,
     ComponentIdentity,
     FailureCode,
+    HostWeightError,
     HostWeightFailure,
     HostWeightLease,
     HostWeightLeaseCarrier,
@@ -40,12 +45,14 @@ from vllm_omni.host_weight_runtime import (
     StoreStatus,
     TensorWriteSpec,
     ValidationLevel,
+    WaitPolicy,
     WeightArtifactIdentity,
     WeightProductionSpec,
     WeightRepresentation,
     WeightSourceIdentity,
 )
 from vllm_omni.host_weight_runtime.filesystem import FilesystemHostWeightStore, detect_storage_class
+from vllm_omni.host_weight_runtime.filesystem.locks import FileLock, lock_is_active
 
 pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
 
@@ -518,6 +525,107 @@ def test_retryable_store_initialization_failure_obeys_runtime_mode(
     assert resolution.report.attempts[0].failure.retryable
 
 
+def _initialize_contended_runtime(config: HostWeightRuntimeConfig, output: Connection) -> None:
+    try:
+        output.send("starting")
+        started = time.monotonic()
+        runtime = HostWeightRuntime.from_config(config)
+        resolution = runtime.resolve(_identity())
+        try:
+            output.send((time.monotonic() - started, resolution.report))
+        finally:
+            if resolution.lease is not None:
+                resolution.lease.close()
+    finally:
+        output.close()
+
+
+@pytest.mark.parametrize(
+    ("mode", "expected"),
+    [
+        (RuntimeMode.PREFERRED, ResolutionOutcome.CANONICAL_FALLBACK),
+        (RuntimeMode.REQUIRED, ResolutionOutcome.FAILED),
+    ],
+)
+def test_domain_lock_timeout_and_fresh_runtime_recovery(
+    tmp_path: Path, mode: RuntimeMode, expected: ResolutionOutcome
+) -> None:
+    config = HostWeightRuntimeConfig(mode=mode, domain=_domain(tmp_path), wait=WaitPolicy(0.1))
+    initial = HostWeightRuntime.from_config(config)
+    built = initial.resolve(_identity(), producer=CountingProducer(_identity()))
+    assert built.lease is not None
+    built.lease.close()
+    metadata = {name: (tmp_path / name).read_bytes() for name in ("domain.json", "domain-policy.json")}
+    lock_path = tmp_path / "locks" / "domain-init.lock"
+    ctx = mp.get_context("spawn")
+    reader, writer = ctx.Pipe(duplex=False)
+    process = ctx.Process(target=_initialize_contended_runtime, args=(config, writer))
+
+    with FileLock(lock_path, exclusive=True, deadline=None):
+        process.start()
+        writer.close()
+        try:
+            # Import/bootstrap is outside the short coordination budget.
+            assert reader.poll(60), "waiter did not start"
+            assert reader.recv() == "starting"
+            assert reader.poll(5), "domain initialization did not respect the coordination timeout"
+            elapsed, report = reader.recv()
+            assert 0.1 <= elapsed < 5
+            assert report.outcome is expected
+            failure = report.attempts[0].failure
+            assert failure is not None
+            assert failure.stage is ResolutionStage.DOMAIN
+            assert failure.code is FailureCode.DOMAIN_UNAVAILABLE
+            assert failure.retryable
+            assert "timed out waiting for" in failure.message
+            assert "domain-init.lock" in failure.message
+            assert lock_is_active(lock_path), "waiter must not unlock the owner"
+            process.join(5)
+            assert process.exitcode == 0
+        finally:
+            if process.is_alive():
+                process.kill()
+            process.join(5)
+            process.close()
+            reader.close()
+
+    assert {name: (tmp_path / name).read_bytes() for name in metadata} == metadata
+    recovered = HostWeightRuntime.from_config(config).resolve(_identity())
+    assert recovered.report.outcome is ResolutionOutcome.LOCAL_HIT
+    assert recovered.lease is not None
+    recovered.lease.close()
+
+
+def test_direct_store_waits_for_domain_lock_released_before_deadline(tmp_path: Path) -> None:
+    config = HostWeightRuntimeConfig(mode=RuntimeMode.PREFERRED, domain=_domain(tmp_path))
+    assert config.domain is not None
+    initial = HostWeightRuntime.from_config(config)
+    assert initial.store is not None
+    lock_path = tmp_path / "locks" / "domain-init.lock"
+    with FileLock(lock_path, exclusive=True, deadline=None) as owner:
+        release = threading.Timer(0.1, owner.close)
+        release.start()
+        try:
+            store = FilesystemHostWeightStore(config.domain, config.capacity, config.integrity, wait=WaitPolicy(5))
+        finally:
+            release.join(5)
+    assert store.root == tmp_path
+
+
+def test_direct_store_domain_timeout_is_typed(tmp_path: Path) -> None:
+    config = HostWeightRuntimeConfig(mode=RuntimeMode.PREFERRED, domain=_domain(tmp_path))
+    assert config.domain is not None
+    HostWeightRuntime.from_config(config)
+    lock_path = tmp_path / "locks" / "domain-init.lock"
+    with FileLock(lock_path, exclusive=True, deadline=None):
+        with pytest.raises(HostWeightError) as caught:
+            FilesystemHostWeightStore(config.domain, config.capacity, config.integrity, wait=WaitPolicy(0.01))
+        assert caught.value.failure.code is FailureCode.DOMAIN_UNAVAILABLE
+        assert caught.value.failure.retryable
+        assert lock_is_active(lock_path)
+    assert not lock_is_active(lock_path)
+
+
 @pytest.mark.parametrize(
     ("stage", "code"),
     [

tests/model_executor/models/minicpmo_4_5/test_chat_template.py

@@ -0,0 +1,45 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+"""MiniCPM structured-content rendering must preserve native part boundaries."""
+
+from pathlib import Path
+
+import pytest
+from jinja2 import TemplateError
+from transformers.utils.chat_template_utils import _compile_jinja_template
+
+pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
+
+TEMPLATE = Path(__file__).resolve().parents[4] / "vllm_omni/transformers_utils/chat_templates/minicpmo45_native.jinja"
+
+
+@pytest.mark.parametrize(
+    "parts,expected",
+    [
+        ([{"type": "image"}, {"type": "text", "text": "Question?"}], "(<image>./</image>)Question?"),
+        ([{"type": "text", "text": "Question?"}, {"type": "image"}], "Question?(<image>./</image>)"),
+        ([{"type": "image"}, {"type": "text", "text": "\n\nQuestion?"}], "(<image>./</image>)\n\nQuestion?"),
+        ([{"type": "text", "text": " A "}, {"type": "text", "text": " B "}], " A  B "),
+        ([{"type": "audio"}, {"type": "text", "text": "Transcribe."}], "(<audio>./</audio>)Transcribe."),
+        ([{"type": "video"}, {"type": "image"}], "(<video>./</video>)(<image>./</image>)"),
+        ("\nKeep this whitespace.\n", "\nKeep this whitespace.\n"),
+    ],
+)
+def test_native_content_concatenation(parts, expected):
+    rendered = _compile_jinja_template(TEMPLATE.read_text()).render(
+        messages=[{"role": "user", "content": parts}],
+        add_generation_prompt=True,
+        enable_thinking=False,
+        use_tts_template=True,
+    )
+    assert rendered == (
+        f"<|im_start|>user\n{expected}<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n<|tts_bos|>"
+    )
+
+
+def test_unsupported_part_is_not_silently_discarded():
+    with pytest.raises(TemplateError, match="Unsupported MiniCPM-o content part"):
+        _compile_jinja_template(TEMPLATE.read_text()).render(
+            messages=[{"role": "user", "content": [{"type": "unknown"}]}],
+            add_generation_prompt=True,
+        )

vllm_omni/host_weight_runtime/filesystem/store.py

@@ -23,7 +23,7 @@ import regex as re
 import torch
 from safetensors import SafetensorError, safe_open
 
-from ..config import CapacityPolicy, IntegrityPolicy, StorageClass, StorageDomainPolicy, ValidationLevel
+from ..config import CapacityPolicy, IntegrityPolicy, StorageClass, StorageDomainPolicy, ValidationLevel, WaitPolicy
 from ..errors import FailureCode, HostWeightError, HostWeightFailure, ResolutionStage
 from ..identity import CanonicalJson, WeightArtifactIdentity, canonical_json
 from ..inspection import (
@@ -429,6 +429,8 @@ class FilesystemHostWeightStore:
         domain: StorageDomainPolicy,
         capacity: CapacityPolicy,
         integrity: IntegrityPolicy,
+        *,
+        wait: WaitPolicy = WaitPolicy(),
     ) -> None:
         self.domain_policy = domain
         self.capacity_policy = capacity
@@ -464,7 +466,7 @@ class FilesystemHostWeightStore:
                 )
             )
         try:
-            self._initialize_domain()
+            self._initialize_domain(deadline=time.monotonic() + wait.coordination_timeout_seconds)
         except HostWeightError:
             raise
         except OSError as exc:
@@ -485,7 +487,7 @@ class FilesystemHostWeightStore:
                 )
             ) from exc
 
-    def _initialize_domain(self) -> None:
+    def _initialize_domain(self, *, deadline: float) -> None:
         try:
             self.filesystem_type = detect_filesystem_type(self.root)
             detected_storage_class = _storage_class_for_filesystem_type(self.filesystem_type)
@@ -545,7 +547,7 @@ class FilesystemHostWeightStore:
                         )
                     )
 
-        with FileLock(self.locks_dir / "domain-init.lock", exclusive=True, deadline=None):
+        with FileLock(self.locks_dir / "domain-init.lock", exclusive=True, deadline=deadline):
             domain_path = self.root / _DOMAIN_FILE
             if domain_path.exists():
                 domain_metadata = _read_json_file(domain_path)

vllm_omni/host_weight_runtime/runtime.py

@@ -70,7 +70,7 @@ class HostWeightRuntime:
         from .filesystem import FilesystemHostWeightStore  # noqa: PLC0415
 
         try:
-            store = FilesystemHostWeightStore(config.domain, config.capacity, config.integrity)
+            store = FilesystemHostWeightStore(config.domain, config.capacity, config.integrity, wait=config.wait)
         except HostWeightError as exc:
             return cls(
                 config,

vllm_omni/transformers_utils/chat_templates/minicpmo45_native.jinja

@@ -0,0 +1,116 @@
+{# SPDX-License-Identifier: Apache-2.0
+   Adapted from OpenBMB/MiniCPM-o-4_5 tokenizer_config.json.
+   Match chat(omni_mode=True): concatenate structured content parts without
+   injecting separators, preserving whitespace explicitly supplied as text.
+#}
+{%- set normalized = namespace(messages=[]) -%}
+{%- for message in messages -%}
+    {%- if message.content is string or message.content is none -%}
+        {%- set normalized.messages = normalized.messages + [message] -%}
+    {%- else -%}
+        {%- set content = namespace(text='') -%}
+        {%- for part in message.content -%}
+            {%- if part.type == 'text' -%}
+                {%- set content.text = content.text + part.text -%}
+            {%- elif part.type == 'image' -%}
+                {%- set content.text = content.text + '(<image>./</image>)' -%}
+            {%- elif part.type == 'audio' -%}
+                {%- set content.text = content.text + '(<audio>./</audio>)' -%}
+            {%- elif part.type == 'video' -%}
+                {%- set content.text = content.text + '(<video>./</video>)' -%}
+            {%- else -%}
+                {{- raise_exception('Unsupported MiniCPM-o content part: ' + part.type) -}}
+            {%- endif -%}
+        {%- endfor -%}
+        {%- set normalized.messages = normalized.messages + [dict(message, content=content.text)] -%}
+    {%- endif -%}
+{%- endfor -%}
+{%- set messages = normalized.messages -%}
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+    {%- if use_tts_template is defined and use_tts_template is true %}
+        {{- '<|tts_bos|>' }}
+    {%- endif %}
+{%- endif -%}