From c9d5545b520787f3c7f1b22c1000b1b2a5feeb8d Mon Sep 17 00:00:00 2001 From: "Guanzhe (Leo) Huang" <116847811+leohuang257@users.noreply.github.com> Date: Mon, 15 Jun 2026 11:36:19 +0800 Subject: [PATCH] [Doc] validate Cosmos3-Nano online FP8 (#4393) Signed-off-by: leohuang257 --- docs/user_guide/quantization/fp8.md | 1 + recipes/cosmos3/Cosmos3-Nano.md | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/user_guide/quantization/fp8.md b/docs/user_guide/quantization/fp8.md index 1abcd4389..7981c7ca2 100644 --- a/docs/user_guide/quantization/fp8.md +++ b/docs/user_guide/quantization/fp8.md @@ -85,6 +85,7 @@ warmup_quack_fp8([(14040, 2048, 6144), (14040, 2048, 2048)]) | FLUX.2-klein | `black-forest-labs/FLUX.2-klein-4B` | Yes | Yes | All layers | None | | | HunyuanImage-3.0 | `tencent/HunyuanImage-3.0`, `tencent/HunyuanImage-3.0-Instruct` | Yes | Yes | All layers; use the Hunyuan stage config for multi-stage runs | None | | | HunyuanVideo-1.5 | `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v`, `720p_t2v`, `480p_i2v` | Yes | Yes | All layers | None | | +| Cosmos3 | `nvidia/Cosmos3-Nano` | Yes | Not validated | All layers | None | | ### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS) diff --git a/recipes/cosmos3/Cosmos3-Nano.md b/recipes/cosmos3/Cosmos3-Nano.md index 2ee9c2ed7..59e2a3f1f 100644 --- a/recipes/cosmos3/Cosmos3-Nano.md +++ b/recipes/cosmos3/Cosmos3-Nano.md @@ -101,7 +101,10 @@ vllm serve nvidia/Cosmos3-Nano \ To run **without** guardrails (you are responsible for license compliance), add `--no-guardrails` (no token/`cosmos-guardrail` needed). For extra GPUs use `--ulysses-degree N` (context parallel) or `--tensor-parallel-size N`; -`--enable-layerwise-offload` reduces VRAM on smaller GPUs. The pipeline +`--enable-layerwise-offload` reduces VRAM on smaller GPUs; +`--quantization fp8` (online, no calibration) cuts peak VRAM for 720p video +generation from ~50 GB to ~36 GB with BF16-level quality (T2V composition can +shift at the same seed). The pipeline auto-resolves from `model_index.json`; pass `--model-class-name Cosmos3OmniDiffusersPipeline` to force it explicitly.