[Edge] Classify and document the edge env vars; fix the client decoder call

- environment_variable_inventory: VLLM_OMNI_CPU_INDUCTOR / HW_PROFILE /
  INIT_TIMELINE / STEP_STATS_DIR (public), STAGE_ID / INIT_TIMELINE_DIR
  (internal); reference page rows added.
- edge_decoder_client.LocalDecoder: batched_chunked_decode takes a [B, Q, F]
  tensor, not a list (the WP5 e2e run failed with a TypeError); the decoder
  now also carries the ramp schedule attributes the Code2Wav stage sets.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
tzhouamandClaude Fable 5.1 committed 2026-09-09 15:22:52 +08:00
1 parent fa8b1a878e
commit 536866b916
4 files changed
+33 -9

No files matched your search

@@ -84,6 +84,10 @@ depends on the installed kernels and model path.
| `SPEAKER_MAX_UPLOADED` | Integer; default `1000` | Speech server; read when speaker storage initializes | Environment-only setting. A non-integer logs a warning and uses `1000`; range is not otherwise validated. | Stable |
| `VLLM_OMNI_ASYNC_OUTPUT_TIMEOUT` | Float seconds; default `600` | Diffusion engine async-output wait in `step_streaming`; resolved per call on the request path, not at import | Environment-only setting. A non-float or `<=0` value warns once and uses the default. | Experimental |
| `VLLM_OMNI_EVENT_DRIVEN_ORCH` | `1`, `true`, `yes` or `on` enables; default `0` (off) | Orchestration loop and the serving-side final-output drain; read once when the `Orchestrator` is constructed | Environment-only setting. Values are stripped and case-normalized; any unrecognized value leaves the legacy poll loop selected. | Experimental |
| `VLLM_OMNI_CPU_INDUCTOR` | `1` enables; default off | CPU OmniPlatform (`VLLM_TARGET_DEVICE=cpu`); read when the platform reports `supports_torch_inductor` | Environment-only setting. Turns on torch.compile/inductor for the CPU AR model runner (measured variant, see `benchmarks/tts/cpu_thread_sweep.py`). | Experimental |
| `VLLM_OMNI_HW_PROFILE` | Filesystem path to a hardware-profile JSON | `deploy_profile="auto"` hardware adaptation (`vllm_omni/edge/hardware_probe.py`); read when the profile is loaded | Environment-only override of the live probe, used for emulation and tests; an unreadable file raises. | Experimental |
| `VLLM_OMNI_INIT_TIMELINE` | Filesystem path (JSON) | Engine init timeline (`vllm_omni/engine/init_timeline.py`); read when `AsyncOmniEngine` starts | Environment-only setting. When set, init phases from the orchestrator and every stage worker are merged and written to this path; a partial timeline is written on init timeout. | Diagnostic |
| `VLLM_OMNI_STEP_STATS_DIR` | Filesystem directory | Per-step overhead attribution (`vllm_omni/edge/step_stats.py`); read once per process on first use | Environment-only setting. Each process dumps `<role>_<pid>.json` (scheduler, engine step, chunk hop and dispatch timings); `stream_latency_bench.py --step-stats` merges them. Zero cost when unset. | Diagnostic |
| `VLLM_OMNI_INPUT_WAIT_TIMEOUT_S` | Float seconds; default `600`; `<=0` disables | Full-payload input coordinator, not async-chunk transfer; read when the scheduler module imports in each worker | Environment-only setting. A non-float logs a warning and uses `600`. | Stable operational control |
| `VLLM_OMNI_ORCH_MONITOR_PATH` | Filesystem path; default `<current-working-directory>/vllm_omni_orch_monitor_<timestamp>.json` | Orchestrator monitor enabled by `--enable-orch-monitor`; read when the monitor is created | Environment-only path override. Parent directories are created; write errors are logged. | Diagnostic |
| `VLLM_OMNI_VIDEO_SYNC_TIMEOUT` | Float seconds; default `600` | Synchronous Videos API; read when the API server module imports | Environment-only setting. A non-float raises `ValueError` during import. | Experimental |
@@ -42,7 +42,13 @@ class LocalDecoder:
"""Stateful chunked Qwen3-TTS decoder mirroring the Code2Wav stage."""
def __init__(
self, model: str, device: str = "cpu", left_context: int = 72, chunk_frames: int = 25, dtype: str = "float32"
self,
model: str,
device: str = "cpu",
left_context: int = 72,
chunk_frames: int = 25,
dtype: str = "float32",
ramp: list[int] | None = None,
):
from safetensors.torch import load_file
@@ -66,8 +72,12 @@ class LocalDecoder:
self.decoder.eval().to(device=device, dtype=getattr(torch, dtype))
if hasattr(self.decoder, "precompute_snake_caches"):
self.decoder.precompute_snake_caches()
if hasattr(self.decoder, "_incremental_chunk_frames"):
self.decoder._incremental_chunk_frames = chunk_frames
# Mirror the Code2Wav stage's chunk-schedule attributes so the decoder's
# prefix/suffix transitions match the server's (qwen3_tts_code2wav.py).
ramp = list(ramp or [])
self.decoder._initial_codec_chunk_frames = int(ramp[0]) if ramp else 1
self.decoder._incremental_chunk_frames = chunk_frames
self.decoder._incremental_chunk_ramp = ramp
self.device = device
self.num_quantizers = int(cfg.decoder_config.num_quantizers)
self.sample_rate = int(cfg.output_sample_rate)
@@ -95,7 +105,7 @@ class LocalDecoder:
return np.zeros(0, dtype=np.float32)
qf = torch.as_tensor(codes.T, dtype=torch.long, device=self.device) # [Q, F]
wavs = self.decoder.batched_chunked_decode(
[qf],
qf.unsqueeze(0), # [B=1, Q, F]; the decoder slices rows itself
[int(qf.shape[-1])],
caches=[self.state],
chunk_size=self.chunk_frames,
@@ -157,13 +167,15 @@ class ExecutorchWindowedDecoder:
return wav[-n * self.hop :] if n < size else wav
def build_decoder(args: argparse.Namespace, steady: int):
def build_decoder(args: argparse.Namespace, steady: int, schedule: list[int] | None = None):
backend = getattr(args, "decoder_backend", "torch")
if backend == "executorch":
if not getattr(args, "pte_manifest", None):
raise SystemExit("--decoder-backend executorch needs --pte-manifest <export dir>/manifest.json")
return ExecutorchWindowedDecoder(args.pte_manifest)
return LocalDecoder(args.model, device=args.device, left_context=args.left_context, chunk_frames=steady)
return LocalDecoder(
args.model, device=args.device, left_context=args.left_context, chunk_frames=steady, ramp=list(schedule)
)
# ------------------------------------------------------------------- client
@@ -173,7 +185,7 @@ async def run(args: argparse.Namespace) -> dict[str, Any]:
ramp = args.ramp or DEFAULT_RAMP
schedule = parse_chunk_ramp({"codec_chunk_ramp": ramp}, steady=ramp[-1]) or ramp
steady = schedule[-1]
decoder = build_decoder(args, steady)
decoder = build_decoder(args, steady, list(schedule))
url = f"ws://{args.host}:{args.port}/v1/audio/speech/stream"
config: dict[str, Any] = {
"type": "session.config",
+2 -2
View File
@@ -164,12 +164,12 @@ def test_inventory_matches_reviewed_snapshot_counts():
"""Make an inventory expansion an explicit review decision."""
category_counts = Counter(item.category for item in ENVIRONMENT_VARIABLE_INVENTORY.values())
assert category_counts == {
EnvironmentVariableCategory.PUBLIC_OMNI: 24,
EnvironmentVariableCategory.PUBLIC_OMNI: 28,
EnvironmentVariableCategory.INHERITED_VLLM: 20,
EnvironmentVariableCategory.PLATFORM_EXTERNAL: 27,
EnvironmentVariableCategory.MODEL_SPECIFIC: 62,
EnvironmentVariableCategory.BENCHMARK_TRANSITIONAL: 20,
EnvironmentVariableCategory.INTERNAL: 2,
EnvironmentVariableCategory.INTERNAL: 4,
}
disposition_counts = Counter(
@@ -64,6 +64,11 @@ _PUBLIC_OMNI = (
"SPEAKER_MAX_UPLOADED",
"SPEAKER_SAMPLES_DIR",
"VLLM_OMNI_ASYNC_OUTPUT_TIMEOUT",
# edge branch (vllm_omni/edge, init timeline, CPU platform)
"VLLM_OMNI_CPU_INDUCTOR",
"VLLM_OMNI_HW_PROFILE",
"VLLM_OMNI_INIT_TIMELINE",
"VLLM_OMNI_STEP_STATS_DIR",
"VLLM_OMNI_EVENT_DRIVEN_ORCH",
"VLLM_OMNI_INPUT_WAIT_TIMEOUT_S",
"VLLM_OMNI_ORCH_MONITOR_PATH",
@@ -238,6 +243,9 @@ _BENCHMARK_TRANSITIONAL = (
_INTERNAL = (
"VLLM_OMNI_DLO_DP_WAVE_TIMEOUT",
"VLLM_OMNI_REPLICA_ID",
# edge branch: set by the orchestrator for its stage subprocesses
"VLLM_OMNI_STAGE_ID",
"VLLM_OMNI_INIT_TIMELINE_DIR",
)
_REDACTED = frozenset(