diff --git a/docs/serving/audio_generate_api.md b/docs/serving/audio_generate_api.md index 7fd7e0d74..0187a77d9 100644 --- a/docs/serving/audio_generate_api.md +++ b/docs/serving/audio_generate_api.md @@ -55,7 +55,7 @@ with open("cat.wav", "wb") as f: ### Endpoint -``` +```text POST /v1/audio/generate Content-Type: application/json ``` @@ -63,7 +63,7 @@ Content-Type: application/json ### Request Parameters | Parameter | Type | Default | Description | -|-----------|------|---------|-------------| +| ----------- | ------ | --------- | ------------- | | `input` | string | **required** | Text prompt describing the audio to generate | | `model` | string | server's model | Model to use (optional, should match server if specified) | | `response_format` | string | "wav" | Audio format: wav, mp3, flac, pcm, opus | @@ -72,7 +72,7 @@ Content-Type: application/json #### Diffusion Parameters | Parameter | Type | Default | Description | -|-----------|------|---------|-------------| +| ----------- | ------ | --------- | ------------- | | `audio_length` | float | null | Audio duration in seconds (default value is the max ~47s for `stable-audio-open-1.0`) | | `audio_start` | float | 0.0 | Audio start time in seconds | | `negative_prompt` | string | null | Text describing what to avoid in generation | @@ -85,7 +85,7 @@ Content-Type: application/json Returns binary audio data with the appropriate `Content-Type` header: | `response_format` | Content-Type | -|--------------------|--------------| +| -------------------- | -------------- | | `wav` | `audio/wav` | | `mp3` | `audio/mpeg` | | `flac` | `audio/flac` | @@ -210,7 +210,7 @@ with open("thunder.wav", "wb") as f: Controls how closely the generated audio follows the text prompt. | Range | Behaviour | -|-------|-----------| +| ------- | ----------- | | 3 - 5 | More creative / varied output | | 7 (default) | Balanced adherence | | 10+ | Strict adherence to the prompt | @@ -220,7 +220,7 @@ Controls how closely the generated audio follows the text prompt. Controls the number of denoising steps in the diffusion process. | Steps | Quality | Speed | Use Case | -|-------|---------|-------|----------| +| ------- | --------- | ------- | ---------- | | 50 | Good | Fast | Quick previews | | 100 | Very Good | Medium | General purpose | | 150+ | Excellent | Slow | Final / critical audio | @@ -240,7 +240,7 @@ Describes characteristics to avoid. Common negative prompts include: ## Supported Models | Model | Description | -|-------|-------------| +| ----- | ----------- | | `stabilityai/stable-audio-open-1.0` | Open-source audio generation model, up to ~47 seconds, 44.1 kHz stereo | ## Error Responses diff --git a/docs/serving/image_edit_api.md b/docs/serving/image_edit_api.md index a38512342..481678c84 100644 --- a/docs/serving/image_edit_api.md +++ b/docs/serving/image_edit_api.md @@ -30,7 +30,6 @@ curl -s -D >(grep -i x-request-id >&2) \ -F "output_format=png" ``` - **Using OpenAI SDK:** ```python @@ -72,7 +71,7 @@ with open("edit_out_http.jpeg", "wb") as f: ### Endpoint -``` +```text POST /v1/images/edits Content-Type: multipart/form-data ``` @@ -82,7 +81,7 @@ Content-Type: multipart/form-data #### OpenAI Standard Parameters | Parameter | Type | Default | Description | -|-----------|------|---------|-------------| +| ----------- | ------ | --------- | ------------- | | `prompt` | string | **required** | A text description of the desired image | | `model` | string | server's model | Model to use (optional, should match server if specified) | | `image` | string or array | **required** | The image(s) to edit. | @@ -98,7 +97,7 @@ Content-Type: multipart/form-data #### vllm-omni Extension Parameters | Parameter | Type | Default | Description | -|-----------|------|---------|-------------| +| ----------- | ------ | --------- | ------------- | | `url` | string or array | None | The image(s) to edit. | | `negative_prompt` | string | null | Text describing what to avoid in the image | | `num_inference_steps` | integer | model defaults | Number of diffusion steps | @@ -212,7 +211,6 @@ Use the `ar_delta` events for progressive display of the AR-generated recaption. Decode the `b64_json` field from the final `image` event to get the edited image. - ## Parameter Handling The API passes parameters directly to the diffusion pipeline without model-specific transformation: @@ -266,6 +264,7 @@ curl http://localhost:8000/v1/images/edits \ ### Out of Memory If you encounter OOM errors: + 1. Reduce image size: `"size": "512x512"` 2. Reduce inference steps: `"num_inference_steps": 25` diff --git a/docs/serving/image_generation_api.md b/docs/serving/image_generation_api.md index 12e2935ea..c139f7d68 100644 --- a/docs/serving/image_generation_api.md +++ b/docs/serving/image_generation_api.md @@ -45,7 +45,6 @@ curl -o dragon.png -X POST http://localhost:8000/v1/images/generations \ }' ``` - **Using Python:** ```python @@ -123,7 +122,7 @@ response = client.images.generate( ### Endpoint -``` +```text POST /v1/images/generations Content-Type: application/json ``` @@ -133,7 +132,7 @@ Content-Type: application/json #### OpenAI Standard Parameters | Parameter | Type | Default | Description | -|-----------|------|---------|-------------| +| ----------- | ------ | --------- | ------------- | | `prompt` | string | **required** | Text description of the desired image | | `model` | string | server's model | Model to use (optional, should match server if specified) | | `n` | integer | 1 | Number of images to generate (1-10) | @@ -144,7 +143,7 @@ Content-Type: application/json #### vllm-omni Extension Parameters | Parameter | Type | Default | Description | -|-----------|------|---------|-------------| +| ----------- | ------ | --------- | ------------- | | `negative_prompt` | string | null | Text describing what to avoid in the image | | `num_inference_steps` | integer | model defaults | Number of diffusion steps | | `guidance_scale` | float | model defaults | Classifier-free guidance scale (typically 0.0-20.0) | @@ -267,6 +266,7 @@ curl http://localhost:8000/v1/images/generations \ ### Out of Memory If you encounter OOM errors: + 1. Reduce image size: `"size": "512x512"` 2. Reduce inference steps: `"num_inference_steps": 25` 3. Generate fewer images: `"n": 1` diff --git a/docs/serving/streaming_video_output_api.md b/docs/serving/streaming_video_output_api.md index 626a26e97..d01489540 100644 --- a/docs/serving/streaming_video_output_api.md +++ b/docs/serving/streaming_video_output_api.md @@ -36,7 +36,7 @@ immediately followed by one binary WebSocket frame containing fragmented MP4 bytes. | Direction | Event or frame | Purpose | -|-----------|----------------|---------| +| ----------- | ---------------- | --------- | | Client to server | `session.start` | Starts generation with a prompt and video parameters | | Server to client | `video.start` | Confirms the request ID, format, and accepted configuration | | Server to client | `video.chunk_metadata` | Describes the next binary media frame | diff --git a/docs/serving/video_stream_api.md b/docs/serving/video_stream_api.md index 688f5fe9c..e1f583991 100644 --- a/docs/serving/video_stream_api.md +++ b/docs/serving/video_stream_api.md @@ -36,7 +36,7 @@ WebSocket /v1/video/chat/stream ### Protocol | Direction | Type | Required fields | Description | -|-----------|------|-----------------|-------------| +| ----------- | ------ | ----------------- | ------------- | | Client -> Server | `session.config` | none | First message. Configures output modalities, frame sampling, EVS, and prompts. | | Client -> Server | `video.frame` | `data` | Base64 JPEG/PNG frame. | | Client -> Server | `audio.chunk` | `data` | Base64 PCM16 16 kHz mono audio bytes. | @@ -53,7 +53,7 @@ WebSocket /v1/video/chat/stream ### `session.config` Fields | Field | Type | Default | Description | -|-------|------|---------|-------------| +| ------- | ------ | --------- | ------------- | | `model` | string or null | null | Optional model name. Usually omitted because the server hosts one model. | | `modalities` | list[string] | `["text", "audio"]` | Output modalities. Use `["text"]`, `["audio"]`, or both. | | `num_frames` | integer, 1-128 | `4` | Number of buffered frames sampled for each query. | @@ -69,7 +69,7 @@ WebSocket /v1/video/chat/stream The server accepts these legacy field names and rewrites them before validation. New clients should send the canonical names above. | Legacy field | Canonical field | -|--------------|-----------------| +| -------------- | ----------------- | | `num_sample_frames` | `num_frames` | | `evs_enabled` | `enable_frame_filter` | | `evs_threshold` | `frame_filter_threshold` | @@ -77,7 +77,7 @@ The server accepts these legacy field names and rewrites them before validation. ### Environment Variables | Variable | Values | Default | Description | -|----------|--------|---------|-------------| +| -------- | ------ | ------- | ----------- | | `VLLM_VIDEO_ASYNC_CHUNK` | `on`, `off` | `on` | Wire-level streaming switch. `off` buffers server-side deltas and emits coalesced outputs at the end of a query. | | `VLLM_VIDEO_AUDIO_DELTA_MODE` | `fast`, `slow` | `fast` | Audio delta extraction strategy. `fast` emits only newly produced chunks; `slow` recomputes from accumulated audio and exists for A/B verification. | diff --git a/docs/serving/videos_api.md b/docs/serving/videos_api.md index 7fc7d0ba8..5a1d8ecf2 100644 --- a/docs/serving/videos_api.md +++ b/docs/serving/videos_api.md @@ -41,7 +41,7 @@ curl -L "http://localhost:8091/v1/videos/${video_id}/content" -o output.mp4 ### Endpoints | Endpoint | Method | Description | -|----------|--------|-------------| +| ---------- | -------- | ------------- | | `/v1/videos` | `POST` | Create an asynchronous video generation job | | `/v1/videos/sync` | `POST` | Generate a video synchronously and return raw video bytes | | `/v1/videos/{video_id}` | `GET` | Retrieve job status and metadata | @@ -56,7 +56,7 @@ curl -L "http://localhost:8091/v1/videos/${video_id}/content" -o output.mp4 #### OpenAI-style fields | Parameter | Type | Default | Description | -|-----------|------|---------|-------------| +| ----------- | ------ | --------- | ------------- | | `prompt` | string | **required** | Text prompt for video generation | | `model` | string | server's model | Optional model name | | `seconds` | string | null | Requested clip duration in seconds | @@ -66,7 +66,7 @@ curl -L "http://localhost:8091/v1/videos/${video_id}/content" -o output.mp4 #### vLLM-Omni extension fields | Parameter | Type | Default | Description | -|-----------|------|---------|-------------| +| ----------- | ------ | --------- | ------------- | | `input_reference` | file | null | Uploaded reference image or video for image-to-video/video-to-video requests | | `image_reference` | string | null | JSON-encoded reference image payload; do not combine with `input_reference` or `video_reference` | | `video_reference` | string | null | JSON-encoded reference video payload; do not combine with `input_reference` or `image_reference` | @@ -203,7 +203,6 @@ curl -s http://localhost:8091/v1/videos \ -F "fps=16" ``` - ### Synchronous Generation ```bash