[Doc] validate Cosmos3-Nano online FP8 (#4393)

Signed-off-by: leohuang257 <masugchds@gmail.com>
This commit is contained in:
Guanzhe (Leo) Huang authored and GitHub committed 2026-06-15 03:36:19 +00:00
1 parent 10404e4c5b
commit c9d5545b52
2 files changed
+5 -1

No files matched your search

+1
View File
@@ -85,6 +85,7 @@ warmup_quack_fp8([(14040, 2048, 6144), (14040, 2048, 2048)])
| FLUX.2-klein | `black-forest-labs/FLUX.2-klein-4B` | Yes | Yes | All layers | None | |
| HunyuanImage-3.0 | `tencent/HunyuanImage-3.0`, `tencent/HunyuanImage-3.0-Instruct` | Yes | Yes | All layers; use the Hunyuan stage config for multi-stage runs | None | |
| HunyuanVideo-1.5 | `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v`, `720p_t2v`, `480p_i2v` | Yes | Yes | All layers | None | |
| Cosmos3 | `nvidia/Cosmos3-Nano` | Yes | Not validated | All layers | None | |
### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)
+4 -1
View File
@@ -101,7 +101,10 @@ vllm serve nvidia/Cosmos3-Nano \
To run **without** guardrails (you are responsible for license compliance),
add `--no-guardrails` (no token/`cosmos-guardrail` needed). For extra GPUs use
`--ulysses-degree N` (context parallel) or `--tensor-parallel-size N`;
`--enable-layerwise-offload` reduces VRAM on smaller GPUs. The pipeline
`--enable-layerwise-offload` reduces VRAM on smaller GPUs;
`--quantization fp8` (online, no calibration) cuts peak VRAM for 720p video
generation from ~50 GB to ~36 GB with BF16-level quality (T2V composition can
shift at the same seed). The pipeline
auto-resolves from `model_index.json`; pass
`--model-class-name Cosmos3OmniDiffusersPipeline` to force it explicitly.