hsliu
afe5c2692c
- Improve API server with better error handling and response formatting - Enhance CLI with additional options for DiT stages and configuration - Add comprehensive examples in examples/basic/ including: - API client with health checks and text generation - Docker setup and usage examples - Simple usage patterns for different scenarios - Add utility scripts for model downloading and Docker setup - Update documentation with implementation details and testing guidelines - Fix configuration validation issues in OmniLLM - Improve stage configuration handling for AR and DiT stages - Add proper error handling and fallback mechanisms Tested with Qwen3-0.6B model: - Server starts successfully on port 8000 - Health and info endpoints working correctly - Text generation with various parameters functioning - API client examples working as expected - CLI help and configuration options working properly
vLLM-omni Examples
This directory contains examples demonstrating how to use vLLM-omni for multi-modality model inference and serving.
Directory Structure
basic/- Simple usage examples and API clientadvanced/- Advanced features and configurationsmultimodal/- Multi-modality specific examples
Quick Start
-
Basic Usage:
cd examples/basic python simple_usage.py -
API Server:
# Start server vllm serve Qwen/Qwen3-0.6B --omni --port 8000 # Test with client python api_client.py
Examples Overview
Basic Examples (basic/)
simple_usage.py- Direct library usage (sync/async)api_client.py- HTTP API client examplesREADME.md- Basic examples documentation
Advanced Examples (advanced/)
- Multi-stage processing
- Custom configurations
- Performance optimization
Multi-modality Examples (multimodal/)
- Text-to-image generation
- Image-to-text processing
- Multi-modal pipelines
Requirements
- Python 3.8+
- vLLM-omni installed
- For API examples: HTTP server running