update readme for v0.11.0rc1 release (#69)

Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Roger Wang <hey@rogerw.io>

README.md

@@ -1,40 +1,65 @@
-# vLLM-omni: Multi-modal Extension for vLLM
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" src="./docs/source/logos/vllm-omni-logo-text-dark.png">
+    <img alt="vllm-omni" src="./docs/source/logos/vllm-omni-logo-text-dark.png" width=55%>
+  </picture>
+</p>
+<h3 align="center">
+Easy, fast, and cheap omni-modality model serving for everyone
+</h3>
 
-vLLM-omni is designed to extend vLLM capabilities to support multi-modality model inference and serving, particularly focusing on non-autoregressive structures and non-textual outputs.
+<p align="center">
+| <a href="https://vllm-omni.readthedocs.io/en/latest/"><b>Documentation</b></a> | <a href="https://tinyurl.com/vllm-omni-meeting"><b>Weekly Meeting</b></a> |
+</p>
 
-## 🎯 Overview
+---
 
-Traditional vLLM systems are limited to text-based, autoregressive generation. vLLM-omni addresses this limitation by enabling support for:
+*Latest News* 🔥
 
-- **Multi-modal Models**: Text, image, video, audio, and sensor data processing
-- **Non-autoregressive Architectures**: Diffusion Transformers (DiT) and other parallel generation models
-- **Heterogeneous Outputs**: Beyond traditional text generation to multimodal outputs
+- [2025/11] vLLM community officially released [vllm-project/vllm-omni](https://github.com/vllm-project/vllm-omni) in order to support omni-modality models serving.
 
-## 🏗️ Architecture
+---
 
-vLLM-omni is built on a modular architecture that extends vLLM's core functionality:
+## About
 
+[vLLM](https://github.com/vllm-project/vllm) was originally designed to support large language models for text-based autoregressive generation tasks. vLLM-Omni extends its support for omni-modality model inference and serving:
 
-## 🚀 Key Features
+- **Omni-modality**: Text, image, video, and audio data processing
+- **Non-autoregressive Architectures**: extend the AR support of vLLM to Diffusion Transformers (DiT) and other parallel generation models
+- **Heterogeneous outputs**: from traditional text generation to multimodal outputs
 
-### Multi-Engine Support
+<p align="center">
+  <picture>
+    <img alt="vllm-omni" src="./docs/source/architecture/omni-modality model architecture.png" width=55%>
+  </picture>
+</p>
 
-- **Autoregressive Engine**: Traditional text generation with enhanced KV-caching
-- **Diffusion Engine**: Support for DiT models and iterative generation
-- **Hybrid Engine**: Combined AR+DiT processing pipelines
+vLLM-omni is fast with:
 
-### Modality Processing
+- Seamless AR support for efficient KV Cache management with vLLM
+- Pipelined stage execution overlapping
+- Fully disaggregation based on omniConnector and dynamic resource allocation across stages
 
-- **Text**: Advanced tokenization and embedding generation
-- **Image**: Vision encoder integration (CLIP, etc.)
-- **Audio**: Speech processing and audio embedding
+vLLM-omni is flexible and easy to use with:
 
-## 📋 Supported Models
+- Seamless integration with popular Hugging Face models
+- High-throughput serving with various decoding algorithms, including *parallel sampling*, *beam search*, and more
+- Tensor, pipeline, data and expert parallelism support for distributed inference
+- Streaming outputs
+- OpenAI-compatible API server
 
-### AR + Diffusion Transformer (DiT) Models
-- Qwen-omni (Thinker-Talker-Codec structure)
-- HunyunaImage 3.0 (Ongoing)
-- Qwen-Image (Ongoing)
+vLLM seamlessly supports most popular open-source models on HuggingFace, including:
+
+- Multi-modality generation models (e.g. Qwen-image)
+- Omni-modality models (e.g. Qwen-omni)
+
+## Getting Started
+
+Install vLLM with `pip` or [from source](https://docs.vllm.ai/en/latest/getting_started/installation/gpu/index.html#build-wheel-from-source):
+
+```bash
+pip install vllm
+```
 
 ## 🛠️ Installation
 
@@ -84,12 +109,6 @@ uv pip install -r requirements/build.txt
 uv pip install --no-build-isolation --editable .
 ```
 
-### Verification for successful installation of vLLM
-Just run the command below. If no error, it demonstrates that the installation is successful.
-```bash
-python -c "import vllm._C"
-```
-
 ### Installation of vLLM-omni
 Install additional requirements for vllm-omni
 ```bash
@@ -98,10 +117,21 @@ cd vllm_omni
 uv pip install -e .
 ```
 
+
 ## Run examples (Qwen2.5-omni)
 
 Please check the folder of [examples](examples)
 
-## Further details
+## Contributing
 
-For detailed architecture design, see [vllm_omni_design.md](docs/architecture/vllm_omni_design.md) and [high_level_arch_design.md](docs/architecture/high_level_arch_design.md).
+We welcome and value any contributions and collaborations.
+Please check out [Contributing to vLLM-omni](https://docs.vllm.ai/en/latest/contributing/index.html) for how to get involved.
+
+## Weekly Meeting
+
+- vLLM Omni Weekly Meeting: https://tinyurl.com/vllm-omni-meeting
+- Wednesday, 11:30 - 12:30 (UTC+8, [Convert to your timezone](https://dateful.com/convert/gmt8?t=15))
+
+## License
+
+Apache License 2.0, as found in the [LICENSE](./LICENSE) file.

scripts/README.md

@@ -1,108 +0,0 @@
-# vLLM-omni Testing Scripts
-
-This directory contains automated testing scripts for vLLM-omni serving functionality.
-
-## Scripts Overview
-
-### 1. `test_serving.sh` - Comprehensive Test Suite
-Full-featured testing script that performs complete validation of the vLLM-omni serving functionality.
-
-**Features:**
-- Model existence validation
-- Environment setup verification
-- Import functionality testing
-- Server startup and health checks
-- Text generation testing
-- Performance benchmarking
-- API client integration testing
-- Comprehensive logging and error handling
-
-**Usage:**
-```bash
-# Use default model and port
-./scripts/test_serving.sh
-
-# Use specific model and port
-./scripts/test_serving.sh ./models/Qwen3-0.6B 8000
-
-# Use HuggingFace model
-./scripts/test_serving.sh Qwen/Qwen3-0.6B 8001
-
-# Show help
-./scripts/test_serving.sh --help
-```
-
-**Test Coverage:**
-- ✅ Model loading and server startup
-- ✅ Health and info endpoints
-- ✅ Text generation functionality
-- ✅ Performance metrics
-- ✅ API client integration
-- ✅ All imports working correctly
-
-### 2. `quick_test.sh` - Fast Validation
-Lightweight script for quick validation after making changes.
-
-**Features:**
-- Fast import testing
-- Basic server startup
-- Health endpoint validation
-- Simple text generation test
-- Retry mechanism for reliability
-
-**Usage:**
-```bash
-# Use default port (8000)
-./scripts/quick_test.sh
-
-# Use specific port
-./scripts/quick_test.sh 8001
-```
-
-**Test Coverage:**
-- ✅ Import functionality
-- ✅ Server startup
-- ✅ Health endpoint
-- ✅ Basic text generation
-
-## Prerequisites
-
-1. **Conda Environment**: Ensure `vllm_omni` environment is activated
-2. **Model Available**: Qwen3-0.6B model should be available in `./models/Qwen3-0.6B/`
-3. **Dependencies**: All vLLM-omni dependencies should be installed
-
-## Environment Setup
-
-```bash
-# Activate conda environment
-conda activate vllm_omni
-
-# Verify installation
-python -c "import vllm_omni; print('vLLM-omni ready')"
-```
-
-## Model Setup
-
-The scripts expect the Qwen3-0.6B model to be available. You can:
-
-1. **Use local model**: Place model in `./models/Qwen3-0.6B/`
-2. **Use HuggingFace model**: Pass `Qwen/Qwen3-0.6B` as model path
-3. **Download model**: Use the `download_models.py` script
-
-```bash
-# Download model using the provided script
-python scripts/download_models.py
-```
-
-## Output and Logging
-
-### Test Results
-- **Success**: Green `[SUCCESS]` messages
-- **Info**: Blue `[INFO]` messages  
-- **Warnings**: Yellow `[WARNING]` messages
-- **Errors**: Red `[ERROR]` messages
-
-### Log Files
-- `server.log`: Server startup and runtime logs
-- `api_client_test.log`: API client test results
-- `simple_usage_test.log`: Simple usage test results

scripts/test_serving.sh

@@ -1,363 +0,0 @@
-#!/bin/bash
-
-# vLLM-omni Serving Functionality Test Script
-# This script tests the complete serving functionality of vLLM-omni
-# Usage: ./scripts/test_serving.sh [model_path] [port]
-
-set -e  # Exit on any error
-
-# Configuration
-MODEL_PATH=${1:-"./models/Qwen3-0.6B"}
-PORT=${2:-8000}
-HOST="localhost"
-TIMEOUT=30
-SERVER_STARTUP_WAIT=15
-
-# Colors for output
-RED='\033[0;31m'
-GREEN='\033[0;32m'
-YELLOW='\033[1;33m'
-BLUE='\033[0;34m'
-NC='\033[0m' # No Color
-
-# Logging functions
-log_info() {
-    echo -e "${BLUE}[INFO]${NC} $1"
-}
-
-log_success() {
-    echo -e "${GREEN}[SUCCESS]${NC} $1"
-}
-
-log_warning() {
-    echo -e "${YELLOW}[WARNING]${NC} $1"
-}
-
-log_error() {
-    echo -e "${RED}[ERROR]${NC} $1"
-}
-
-# Cleanup function
-cleanup() {
-    log_info "Cleaning up..."
-    if [ ! -z "$SERVER_PID" ]; then
-        log_info "Stopping server (PID: $SERVER_PID)..."
-        kill $SERVER_PID 2>/dev/null || true
-        wait $SERVER_PID 2>/dev/null || true
-    fi
-    # Kill any remaining vllm processes
-    pkill -f "vllm serve" 2>/dev/null || true
-    log_info "Cleanup completed"
-}
-
-# Set up trap for cleanup on exit
-trap cleanup EXIT
-
-# Check if model exists
-check_model() {
-    log_info "Checking if model exists at: $MODEL_PATH"
-    if [ ! -d "$MODEL_PATH" ]; then
-        log_error "Model directory not found: $MODEL_PATH"
-        log_info "Available models:"
-        ls -la models/ 2>/dev/null || log_warning "No models directory found"
-        exit 1
-    fi
-    log_success "Model found: $MODEL_PATH"
-}
-
-# Check if conda environment is activated
-check_environment() {
-    log_info "Checking conda environment..."
-    if [[ "$CONDA_DEFAULT_ENV" != "vllm_omni" ]]; then
-        log_warning "vllm_omni conda environment not activated"
-        log_info "Attempting to activate vllm_omni environment..."
-        source $(conda info --base)/etc/profile.d/conda.sh
-        conda activate vllm_omni
-        if [[ "$CONDA_DEFAULT_ENV" != "vllm_omni" ]]; then
-            log_error "Failed to activate vllm_omni environment"
-            exit 1
-        fi
-    fi
-    log_success "Environment check passed: $CONDA_DEFAULT_ENV"
-}
-
-# Test import functionality
-test_imports() {
-    log_info "Testing imports..."
-    python -c "
-import vllm_omni
-from vllm_omni.entrypoints.omni_llm import OmniLLM, AsyncOmniLLM
-print('✅ All imports successful')
-" || {
-        log_error "Import test failed"
-        exit 1
-    }
-    log_success "Import test passed"
-}
-
-# Start the server
-start_server() {
-    log_info "Starting vLLM-omni server on port $PORT..."
-    log_info "Command: vllm serve $MODEL_PATH --omni --port $PORT"
-
-    # Start server in background
-    vllm serve "$MODEL_PATH" --omni --port "$PORT" > server.log 2>&1 &
-    SERVER_PID=$!
-
-    log_info "Server started with PID: $SERVER_PID"
-    log_info "Waiting $SERVER_STARTUP_WAIT seconds for server to initialize..."
-    sleep $SERVER_STARTUP_WAIT
-
-    # Check if server is still running
-    if ! kill -0 $SERVER_PID 2>/dev/null; then
-        log_error "Server failed to start"
-        log_info "Server logs:"
-        cat server.log
-        exit 1
-    fi
-
-    log_success "Server appears to be running"
-}
-
-# Test health endpoint
-test_health() {
-    log_info "Testing health endpoint..."
-    for attempt in {1..10}; do
-        local response=$(curl -s -w "%{http_code}" http://$HOST:$PORT/health 2>/dev/null || echo "000")
-        local http_code="${response: -3}"
-        local body="${response%???}"
-
-        if [ "$http_code" = "200" ]; then
-            log_success "Health check passed: $body"
-            return 0
-        fi
-
-        log_warning "Health check attempt $attempt failed (HTTP $http_code). Retrying in 3s..."
-        sleep 3
-    done
-
-    log_error "Health check failed after multiple attempts"
-    return 1
-}
-
-# Test info endpoint
-test_info() {
-    log_info "Testing info endpoint..."
-    local response=$(curl -s -w "%{http_code}" http://$HOST:$PORT/info 2>/dev/null || echo "000")
-    local http_code="${response: -3}"
-    local body="${response%???}"
-
-    if [ "$http_code" = "200" ]; then
-        log_success "Info endpoint working"
-        echo "$body" | python -m json.tool 2>/dev/null || log_warning "Info response not valid JSON"
-    else
-        log_error "Info endpoint failed (HTTP $http_code): $body"
-        return 1
-    fi
-}
-
-# Test text generation
-test_generation() {
-    log_info "Testing text generation..."
-
-    # Test 1: Simple generation
-    local response=$(curl -s -X POST http://$HOST:$PORT/generate \
-        -H "Content-Type: application/json" \
-        -d '{"prompts": ["Test the server functionality"], "max_tokens": 20, "temperature": 0.7}' \
-        2>/dev/null || echo "{}")
-
-    if echo "$response" | python -c "import sys, json; data=json.load(sys.stdin); exit(0 if 'outputs' in data and len(data['outputs']) > 0 else 1)" 2>/dev/null; then
-        log_success "Text generation test passed"
-        echo "$response" | python -c "import sys, json; data=json.load(sys.stdin); print('Generated text:', data['outputs'][0]['text'][:100] + '...' if len(data['outputs'][0]['text']) > 100 else data['outputs'][0]['text'])" 2>/dev/null
-    else
-        log_error "Text generation test failed"
-        echo "Response: $response"
-        return 1
-    fi
-}
-
-# Test API client example
-test_api_client() {
-    log_info "Testing API client example..."
-    if [ -f "examples/basic/api_client.py" ]; then
-        python examples/basic/api_client.py > api_client_test.log 2>&1 || {
-            log_error "API client test failed"
-            log_info "API client logs:"
-            cat api_client_test.log
-            return 1
-        }
-        log_success "API client test passed"
-    else
-        log_warning "API client example not found, skipping"
-    fi
-}
-
-# Test AR → DiT pipeline example
-test_ar_dit_pipeline() {
-    log_info "Testing AR → DiT diffusers pipeline example..."
-
-    if ! python - 2>/dev/null <<'PY'
-import importlib
-for pkg in ("torch", "diffusers"):
-    if importlib.util.find_spec(pkg) is None:
-        raise ImportError(pkg)
-PY
-    then
-        log_warning "Required packages for diffusers pipeline not found, skipping AR → DiT test"
-        return 0
-    fi
-
-    if [ ! -f "examples/omni/ar_dit_diffusers.py" ]; then
-        log_warning "AR → DiT example not found, skipping"
-        return 0
-    fi
-
-    local output_path="logs/ar_dit_pipeline.png"
-    local log_path="logs/ar_dit_pipeline.log"
-    rm -f "$output_path"
-
-    local cmd=(python examples/omni/ar_dit_diffusers.py
-        --prompt "Smoke test prompt of a lighthouse"
-        --seed 0
-        --output "$output_path")
-
-    local env_prefix=()
-    if [ -n "$AR_MODEL" ]; then
-        env_prefix+=("AR_MODEL=$AR_MODEL")
-    fi
-    if [ -n "$DIT_MODEL" ]; then
-        env_prefix+=("DIT_MODEL=$DIT_MODEL")
-    fi
-
-    if ! env "${env_prefix[@]}" "${cmd[@]}" >"$log_path" 2>&1; then
-        log_error "AR → DiT pipeline test failed"
-        log_info "AR → DiT logs:"
-        cat "$log_path"
-        return 1
-    fi
-
-    if [ ! -f "$output_path" ]; then
-        log_error "AR → DiT pipeline did not produce an image"
-        log_info "AR → DiT logs:"
-        cat "$log_path"
-        return 1
-    fi
-
-    log_success "AR → DiT pipeline test passed (output: $output_path)"
-}
-
-# Test simple usage example
-test_simple_usage() {
-    log_info "Testing simple usage example..."
-    if [ -f "examples/basic/simple_usage.py" ]; then
-        # Run with timeout to avoid hanging
-        timeout 60 python examples/basic/simple_usage.py > simple_usage_test.log 2>&1 || {
-            local exit_code=$?
-            if [ $exit_code -eq 124 ]; then
-                log_warning "Simple usage test timed out (60s), but this might be normal for model loading"
-            else
-                log_error "Simple usage test failed (exit code: $exit_code)"
-                log_info "Simple usage logs:"
-                cat simple_usage_test.log
-                return 1
-            fi
-        }
-        log_success "Simple usage test passed"
-    else
-        log_warning "Simple usage example not found, skipping"
-    fi
-}
-
-# Performance test
-test_performance() {
-    log_info "Running performance test..."
-
-    local start_time=$(date +%s)
-    local response=$(curl -s -X POST http://$HOST:$PORT/generate \
-        -H "Content-Type: application/json" \
-        -d '{"prompts": ["Performance test prompt"], "max_tokens": 50, "temperature": 0.7}' \
-        2>/dev/null)
-    local end_time=$(date +%s)
-    local duration=$((end_time - start_time))
-
-    if echo "$response" | python -c "import sys, json; data=json.load(sys.stdin); exit(0 if 'outputs' in data else 1)" 2>/dev/null; then
-        log_success "Performance test passed (${duration}s response time)"
-    else
-        log_error "Performance test failed"
-        return 1
-    fi
-}
-
-# Main test function
-run_tests() {
-    log_info "Starting vLLM-omni serving functionality tests..."
-    log_info "Model: $MODEL_PATH"
-    log_info "Port: $PORT"
-    log_info "Host: $HOST"
-    echo "=========================================="
-
-    # Run all tests
-    check_model
-    check_environment
-    test_imports
-    start_server
-
-    # Wait a bit more for server to be fully ready
-    sleep 5
-
-    test_health
-    test_info
-    test_generation
-    test_performance
-    test_api_client
-    test_ar_dit_pipeline
-
-    # Note: Simple usage test is commented out as it starts its own server
-    # test_simple_usage
-
-    log_success "All tests completed successfully!"
-    echo "=========================================="
-    log_info "Test Summary:"
-    log_info "✅ Model loading and server startup"
-    log_info "✅ Health and info endpoints"
-    log_info "✅ Text generation functionality"
-    log_info "✅ Performance metrics"
-    log_info "✅ API client integration"
-    log_info "✅ AR → DiT diffusers pipeline"
-    log_info "✅ All imports working correctly"
-}
-
-# Show usage
-show_usage() {
-    echo "Usage: $0 [model_path] [port]"
-    echo ""
-    echo "Arguments:"
-    echo "  model_path    Path to the model directory (default: ./models/Qwen3-0.6B)"
-    echo "  port          Port to run the server on (default: 8000)"
-    echo ""
-    echo "Examples:"
-    echo "  $0                                    # Use default model and port"
-    echo "  $0 ./models/Qwen3-0.6B 8001          # Use specific model and port"
-    echo "  $0 Qwen/Qwen3-0.6B 8000              # Use HuggingFace model"
-}
-
-# Main execution
-main() {
-    # Check for help flag
-    if [[ "$1" == "-h" || "$1" == "--help" ]]; then
-        show_usage
-        exit 0
-    fi
-
-    # Change to script directory
-    cd "$(dirname "$0")/.."
-
-    # Create logs directory if it doesn't exist
-    mkdir -p logs
-
-    # Run tests
-    run_tests
-}
-
-# Run main function with all arguments
-main "$@"