2025-09-11 14:32:51 +08:00
|
|
|
# vLLM-omni Examples
|
|
|
|
|
|
2025-09-26 15:47:55 +08:00
|
|
|
This directory contains examples demonstrating how to use vLLM-omni for multi-modality model inference and serving.
|
2025-09-11 14:32:51 +08:00
|
|
|
|
2025-09-26 15:47:55 +08:00
|
|
|
## Directory Structure
|
2025-09-11 14:32:51 +08:00
|
|
|
|
2025-09-26 15:47:55 +08:00
|
|
|
- `basic/` - Simple usage examples and API client
|
|
|
|
|
- `advanced/` - Advanced features and configurations
|
|
|
|
|
- `multimodal/` - Multi-modality specific examples
|
2025-09-11 14:32:51 +08:00
|
|
|
|
2025-09-26 15:47:55 +08:00
|
|
|
## Quick Start
|
2025-09-11 14:32:51 +08:00
|
|
|
|
2025-09-26 15:47:55 +08:00
|
|
|
1. **Basic Usage:**
|
2025-09-11 14:32:51 +08:00
|
|
|
```bash
|
2025-09-26 15:47:55 +08:00
|
|
|
cd examples/basic
|
|
|
|
|
python simple_usage.py
|
2025-09-11 14:32:51 +08:00
|
|
|
```
|
|
|
|
|
|
2025-09-26 15:47:55 +08:00
|
|
|
2. **API Server:**
|
2025-09-11 14:32:51 +08:00
|
|
|
```bash
|
2025-09-26 15:47:55 +08:00
|
|
|
# Start server
|
|
|
|
|
vllm serve Qwen/Qwen3-0.6B --omni --port 8000
|
|
|
|
|
|
|
|
|
|
# Test with client
|
|
|
|
|
python api_client.py
|
2025-09-11 14:32:51 +08:00
|
|
|
```
|
|
|
|
|
|
2025-09-26 15:47:55 +08:00
|
|
|
## Examples Overview
|
|
|
|
|
|
|
|
|
|
### Basic Examples (`basic/`)
|
|
|
|
|
- `simple_usage.py` - Direct library usage (sync/async)
|
|
|
|
|
- `api_client.py` - HTTP API client examples
|
|
|
|
|
- `README.md` - Basic examples documentation
|
|
|
|
|
|
|
|
|
|
### Advanced Examples (`advanced/`)
|
|
|
|
|
- Multi-stage processing
|
|
|
|
|
- Custom configurations
|
|
|
|
|
- Performance optimization
|
|
|
|
|
|
|
|
|
|
### Multi-modality Examples (`multimodal/`)
|
|
|
|
|
- Text-to-image generation
|
|
|
|
|
- Image-to-text processing
|
|
|
|
|
- Multi-modal pipelines
|
2025-09-11 14:32:51 +08:00
|
|
|
|
|
|
|
|
## Requirements
|
|
|
|
|
|
2025-09-26 15:47:55 +08:00
|
|
|
- Python 3.8+
|
|
|
|
|
- vLLM-omni installed
|
|
|
|
|
- For API examples: HTTP server running
|