feat: add comprehensive testing scripts for vLLM-omni
- Add test_serving.sh: Full-featured testing suite with comprehensive validation - Add quick_test.sh: Fast validation script for quick testing after changes - Add scripts/README.md: Complete documentation for testing scripts - Include health checks, text generation, performance testing, and API integration - Add retry mechanisms and proper error handling - Support for different models and ports - Comprehensive logging and colored output - Ready for CI/CD integration Usage: - Quick test: ./scripts/quick_test.sh [port] - Full test: ./scripts/test_serving.sh [model_path] [port]
This commit is contained in:
hsliu committed
1 parent
e8b9425bab
commit
bd43ce40e7
3 files changed
+567
No files matched your search
@@ -0,0 +1,184 @@
|
||||
# vLLM-omni Testing Scripts
|
||||
|
||||
This directory contains automated testing scripts for vLLM-omni serving functionality.
|
||||
|
||||
## Scripts Overview
|
||||
|
||||
### 1. `test_serving.sh` - Comprehensive Test Suite
|
||||
Full-featured testing script that performs complete validation of the vLLM-omni serving functionality.
|
||||
|
||||
**Features:**
|
||||
- Model existence validation
|
||||
- Environment setup verification
|
||||
- Import functionality testing
|
||||
- Server startup and health checks
|
||||
- Text generation testing
|
||||
- Performance benchmarking
|
||||
- API client integration testing
|
||||
- Comprehensive logging and error handling
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Use default model and port
|
||||
./scripts/test_serving.sh
|
||||
|
||||
# Use specific model and port
|
||||
./scripts/test_serving.sh ./models/Qwen3-0.6B 8000
|
||||
|
||||
# Use HuggingFace model
|
||||
./scripts/test_serving.sh Qwen/Qwen3-0.6B 8001
|
||||
|
||||
# Show help
|
||||
./scripts/test_serving.sh --help
|
||||
```
|
||||
|
||||
**Test Coverage:**
|
||||
- ✅ Model loading and server startup
|
||||
- ✅ Health and info endpoints
|
||||
- ✅ Text generation functionality
|
||||
- ✅ Performance metrics
|
||||
- ✅ API client integration
|
||||
- ✅ All imports working correctly
|
||||
|
||||
### 2. `quick_test.sh` - Fast Validation
|
||||
Lightweight script for quick validation after making changes.
|
||||
|
||||
**Features:**
|
||||
- Fast import testing
|
||||
- Basic server startup
|
||||
- Health endpoint validation
|
||||
- Simple text generation test
|
||||
- Retry mechanism for reliability
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Use default port (8000)
|
||||
./scripts/quick_test.sh
|
||||
|
||||
# Use specific port
|
||||
./scripts/quick_test.sh 8001
|
||||
```
|
||||
|
||||
**Test Coverage:**
|
||||
- ✅ Import functionality
|
||||
- ✅ Server startup
|
||||
- ✅ Health endpoint
|
||||
- ✅ Basic text generation
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **Conda Environment**: Ensure `vllm_omni` environment is activated
|
||||
2. **Model Available**: Qwen3-0.6B model should be available in `./models/Qwen3-0.6B/`
|
||||
3. **Dependencies**: All vLLM-omni dependencies should be installed
|
||||
|
||||
## Environment Setup
|
||||
|
||||
```bash
|
||||
# Activate conda environment
|
||||
conda activate vllm_omni
|
||||
|
||||
# Verify installation
|
||||
python -c "import vllm_omni; print('vLLM-omni ready')"
|
||||
```
|
||||
|
||||
## Model Setup
|
||||
|
||||
The scripts expect the Qwen3-0.6B model to be available. You can:
|
||||
|
||||
1. **Use local model**: Place model in `./models/Qwen3-0.6B/`
|
||||
2. **Use HuggingFace model**: Pass `Qwen/Qwen3-0.6B` as model path
|
||||
3. **Download model**: Use the `download_models.py` script
|
||||
|
||||
```bash
|
||||
# Download model using the provided script
|
||||
python scripts/download_models.py
|
||||
```
|
||||
|
||||
## Output and Logging
|
||||
|
||||
### Test Results
|
||||
- **Success**: Green `[SUCCESS]` messages
|
||||
- **Info**: Blue `[INFO]` messages
|
||||
- **Warnings**: Yellow `[WARNING]` messages
|
||||
- **Errors**: Red `[ERROR]` messages
|
||||
|
||||
### Log Files
|
||||
- `server.log`: Server startup and runtime logs
|
||||
- `api_client_test.log`: API client test results
|
||||
- `simple_usage_test.log`: Simple usage test results
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Server fails to start**
|
||||
- Check if port is available
|
||||
- Verify model path is correct
|
||||
- Check conda environment is activated
|
||||
|
||||
2. **Health check fails**
|
||||
- Server may need more time to start
|
||||
- Check server logs for errors
|
||||
- Verify port is not blocked
|
||||
|
||||
3. **Import errors**
|
||||
- Ensure vllm_omni is properly installed
|
||||
- Check conda environment activation
|
||||
- Verify all dependencies are installed
|
||||
|
||||
4. **Model not found**
|
||||
- Verify model path exists
|
||||
- Check model directory structure
|
||||
- Use absolute paths if needed
|
||||
|
||||
### Debug Mode
|
||||
|
||||
For detailed debugging, run tests with verbose output:
|
||||
|
||||
```bash
|
||||
# Enable debug mode
|
||||
export VLLM_OMNI_DEBUG=1
|
||||
./scripts/test_serving.sh
|
||||
```
|
||||
|
||||
## Integration with CI/CD
|
||||
|
||||
These scripts can be integrated into CI/CD pipelines:
|
||||
|
||||
```yaml
|
||||
# Example GitHub Actions workflow
|
||||
- name: Test vLLM-omni
|
||||
run: |
|
||||
conda activate vllm_omni
|
||||
./scripts/test_serving.sh
|
||||
```
|
||||
|
||||
## Performance Expectations
|
||||
|
||||
### Quick Test
|
||||
- **Duration**: ~30-60 seconds
|
||||
- **Memory**: ~2-4GB RAM
|
||||
- **CPU**: Moderate usage during model loading
|
||||
|
||||
### Comprehensive Test
|
||||
- **Duration**: ~2-5 minutes
|
||||
- **Memory**: ~2-4GB RAM
|
||||
- **CPU**: Moderate usage during model loading and generation
|
||||
|
||||
## Contributing
|
||||
|
||||
When adding new tests:
|
||||
|
||||
1. **Update test_serving.sh** for comprehensive testing
|
||||
2. **Update quick_test.sh** for essential functionality
|
||||
3. **Update this README** with new test descriptions
|
||||
4. **Test both scripts** before committing changes
|
||||
|
||||
## Support
|
||||
|
||||
For issues with testing scripts:
|
||||
|
||||
1. Check the troubleshooting section above
|
||||
2. Review log files for detailed error information
|
||||
3. Ensure all prerequisites are met
|
||||
4. Test with different ports if port conflicts occur
|
||||
Executable
+82
@@ -0,0 +1,82 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Quick vLLM-omni Test Script
|
||||
# Fast test for basic functionality after changes
|
||||
# Usage: ./scripts/quick_test.sh [port]
|
||||
|
||||
set -e
|
||||
|
||||
PORT=${1:-8000}
|
||||
HOST="localhost"
|
||||
|
||||
# Colors
|
||||
GREEN='\033[0;32m'
|
||||
RED='\033[0;31m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m'
|
||||
|
||||
log_info() { echo -e "${BLUE}[INFO]${NC} $1"; }
|
||||
log_success() { echo -e "${GREEN}[SUCCESS]${NC} $1"; }
|
||||
log_error() { echo -e "${RED}[ERROR]${NC} $1"; }
|
||||
|
||||
# Cleanup
|
||||
cleanup() {
|
||||
log_info "Cleaning up..."
|
||||
pkill -f "vllm serve" 2>/dev/null || true
|
||||
}
|
||||
|
||||
trap cleanup EXIT
|
||||
|
||||
# Quick test
|
||||
main() {
|
||||
log_info "Quick vLLM-omni test on port $PORT"
|
||||
|
||||
# Test imports
|
||||
log_info "Testing imports..."
|
||||
python -c "import vllm_omni; from vllm_omni.entrypoints.omni_llm import OmniLLM, AsyncOmniLLM; print('✅ Imports OK')" || {
|
||||
log_error "Import failed"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Start server
|
||||
log_info "Starting server..."
|
||||
vllm serve ./models/Qwen3-0.6B --omni --port $PORT > /dev/null 2>&1 &
|
||||
SERVER_PID=$!
|
||||
|
||||
# Wait for server
|
||||
sleep 15
|
||||
|
||||
# Test health with retry
|
||||
log_info "Testing health endpoint..."
|
||||
for i in {1..5}; do
|
||||
if curl -s http://$HOST:$PORT/health | grep -q "healthy"; then
|
||||
log_success "Health check passed"
|
||||
break
|
||||
else
|
||||
if [ $i -eq 5 ]; then
|
||||
log_error "Health check failed after 5 attempts"
|
||||
exit 1
|
||||
fi
|
||||
log_info "Health check attempt $i failed, retrying in 3 seconds..."
|
||||
sleep 3
|
||||
fi
|
||||
done
|
||||
|
||||
# Test generation
|
||||
log_info "Testing text generation..."
|
||||
response=$(curl -s -X POST http://$HOST:$PORT/generate \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"prompts": ["Quick test"], "max_tokens": 10, "temperature": 0.7}')
|
||||
|
||||
if echo "$response" | python -c "import sys, json; data=json.load(sys.stdin); exit(0 if 'outputs' in data else 1)" 2>/dev/null; then
|
||||
log_success "Generation test passed"
|
||||
else
|
||||
log_error "Generation test failed"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
log_success "Quick test completed successfully!"
|
||||
}
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
main "$@"
|
||||
Executable
+301
@@ -0,0 +1,301 @@
|
||||
#!/bin/bash
|
||||
|
||||
# vLLM-omni Serving Functionality Test Script
|
||||
# This script tests the complete serving functionality of vLLM-omni
|
||||
# Usage: ./scripts/test_serving.sh [model_path] [port]
|
||||
|
||||
set -e # Exit on any error
|
||||
|
||||
# Configuration
|
||||
MODEL_PATH=${1:-"./models/Qwen3-0.6B"}
|
||||
PORT=${2:-8000}
|
||||
HOST="localhost"
|
||||
TIMEOUT=30
|
||||
SERVER_STARTUP_WAIT=15
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Logging functions
|
||||
log_info() {
|
||||
echo -e "${BLUE}[INFO]${NC} $1"
|
||||
}
|
||||
|
||||
log_success() {
|
||||
echo -e "${GREEN}[SUCCESS]${NC} $1"
|
||||
}
|
||||
|
||||
log_warning() {
|
||||
echo -e "${YELLOW}[WARNING]${NC} $1"
|
||||
}
|
||||
|
||||
log_error() {
|
||||
echo -e "${RED}[ERROR]${NC} $1"
|
||||
}
|
||||
|
||||
# Cleanup function
|
||||
cleanup() {
|
||||
log_info "Cleaning up..."
|
||||
if [ ! -z "$SERVER_PID" ]; then
|
||||
log_info "Stopping server (PID: $SERVER_PID)..."
|
||||
kill $SERVER_PID 2>/dev/null || true
|
||||
wait $SERVER_PID 2>/dev/null || true
|
||||
fi
|
||||
# Kill any remaining vllm processes
|
||||
pkill -f "vllm serve" 2>/dev/null || true
|
||||
log_info "Cleanup completed"
|
||||
}
|
||||
|
||||
# Set up trap for cleanup on exit
|
||||
trap cleanup EXIT
|
||||
|
||||
# Check if model exists
|
||||
check_model() {
|
||||
log_info "Checking if model exists at: $MODEL_PATH"
|
||||
if [ ! -d "$MODEL_PATH" ]; then
|
||||
log_error "Model directory not found: $MODEL_PATH"
|
||||
log_info "Available models:"
|
||||
ls -la models/ 2>/dev/null || log_warning "No models directory found"
|
||||
exit 1
|
||||
fi
|
||||
log_success "Model found: $MODEL_PATH"
|
||||
}
|
||||
|
||||
# Check if conda environment is activated
|
||||
check_environment() {
|
||||
log_info "Checking conda environment..."
|
||||
if [[ "$CONDA_DEFAULT_ENV" != "vllm_omni" ]]; then
|
||||
log_warning "vllm_omni conda environment not activated"
|
||||
log_info "Attempting to activate vllm_omni environment..."
|
||||
source $(conda info --base)/etc/profile.d/conda.sh
|
||||
conda activate vllm_omni
|
||||
if [[ "$CONDA_DEFAULT_ENV" != "vllm_omni" ]]; then
|
||||
log_error "Failed to activate vllm_omni environment"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
log_success "Environment check passed: $CONDA_DEFAULT_ENV"
|
||||
}
|
||||
|
||||
# Test import functionality
|
||||
test_imports() {
|
||||
log_info "Testing imports..."
|
||||
python -c "
|
||||
import vllm_omni
|
||||
from vllm_omni.entrypoints.omni_llm import OmniLLM, AsyncOmniLLM
|
||||
print('✅ All imports successful')
|
||||
" || {
|
||||
log_error "Import test failed"
|
||||
exit 1
|
||||
}
|
||||
log_success "Import test passed"
|
||||
}
|
||||
|
||||
# Start the server
|
||||
start_server() {
|
||||
log_info "Starting vLLM-omni server on port $PORT..."
|
||||
log_info "Command: vllm serve $MODEL_PATH --omni --port $PORT"
|
||||
|
||||
# Start server in background
|
||||
vllm serve "$MODEL_PATH" --omni --port "$PORT" > server.log 2>&1 &
|
||||
SERVER_PID=$!
|
||||
|
||||
log_info "Server started with PID: $SERVER_PID"
|
||||
log_info "Waiting $SERVER_STARTUP_WAIT seconds for server to initialize..."
|
||||
sleep $SERVER_STARTUP_WAIT
|
||||
|
||||
# Check if server is still running
|
||||
if ! kill -0 $SERVER_PID 2>/dev/null; then
|
||||
log_error "Server failed to start"
|
||||
log_info "Server logs:"
|
||||
cat server.log
|
||||
exit 1
|
||||
fi
|
||||
|
||||
log_success "Server appears to be running"
|
||||
}
|
||||
|
||||
# Test health endpoint
|
||||
test_health() {
|
||||
log_info "Testing health endpoint..."
|
||||
local response=$(curl -s -w "%{http_code}" http://$HOST:$PORT/health 2>/dev/null || echo "000")
|
||||
local http_code="${response: -3}"
|
||||
local body="${response%???}"
|
||||
|
||||
if [ "$http_code" = "200" ]; then
|
||||
log_success "Health check passed: $body"
|
||||
else
|
||||
log_error "Health check failed (HTTP $http_code): $body"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Test info endpoint
|
||||
test_info() {
|
||||
log_info "Testing info endpoint..."
|
||||
local response=$(curl -s -w "%{http_code}" http://$HOST:$PORT/info 2>/dev/null || echo "000")
|
||||
local http_code="${response: -3}"
|
||||
local body="${response%???}"
|
||||
|
||||
if [ "$http_code" = "200" ]; then
|
||||
log_success "Info endpoint working"
|
||||
echo "$body" | python -m json.tool 2>/dev/null || log_warning "Info response not valid JSON"
|
||||
else
|
||||
log_error "Info endpoint failed (HTTP $http_code): $body"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Test text generation
|
||||
test_generation() {
|
||||
log_info "Testing text generation..."
|
||||
|
||||
# Test 1: Simple generation
|
||||
local response=$(curl -s -X POST http://$HOST:$PORT/generate \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"prompts": ["Test the server functionality"], "max_tokens": 20, "temperature": 0.7}' \
|
||||
2>/dev/null || echo "{}")
|
||||
|
||||
if echo "$response" | python -c "import sys, json; data=json.load(sys.stdin); exit(0 if 'outputs' in data and len(data['outputs']) > 0 else 1)" 2>/dev/null; then
|
||||
log_success "Text generation test passed"
|
||||
echo "$response" | python -c "import sys, json; data=json.load(sys.stdin); print('Generated text:', data['outputs'][0]['text'][:100] + '...' if len(data['outputs'][0]['text']) > 100 else data['outputs'][0]['text'])" 2>/dev/null
|
||||
else
|
||||
log_error "Text generation test failed"
|
||||
echo "Response: $response"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Test API client example
|
||||
test_api_client() {
|
||||
log_info "Testing API client example..."
|
||||
if [ -f "examples/basic/api_client.py" ]; then
|
||||
python examples/basic/api_client.py > api_client_test.log 2>&1 || {
|
||||
log_error "API client test failed"
|
||||
log_info "API client logs:"
|
||||
cat api_client_test.log
|
||||
return 1
|
||||
}
|
||||
log_success "API client test passed"
|
||||
else
|
||||
log_warning "API client example not found, skipping"
|
||||
fi
|
||||
}
|
||||
|
||||
# Test simple usage example
|
||||
test_simple_usage() {
|
||||
log_info "Testing simple usage example..."
|
||||
if [ -f "examples/basic/simple_usage.py" ]; then
|
||||
# Run with timeout to avoid hanging
|
||||
timeout 60 python examples/basic/simple_usage.py > simple_usage_test.log 2>&1 || {
|
||||
local exit_code=$?
|
||||
if [ $exit_code -eq 124 ]; then
|
||||
log_warning "Simple usage test timed out (60s), but this might be normal for model loading"
|
||||
else
|
||||
log_error "Simple usage test failed (exit code: $exit_code)"
|
||||
log_info "Simple usage logs:"
|
||||
cat simple_usage_test.log
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
log_success "Simple usage test passed"
|
||||
else
|
||||
log_warning "Simple usage example not found, skipping"
|
||||
fi
|
||||
}
|
||||
|
||||
# Performance test
|
||||
test_performance() {
|
||||
log_info "Running performance test..."
|
||||
|
||||
local start_time=$(date +%s)
|
||||
local response=$(curl -s -X POST http://$HOST:$PORT/generate \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"prompts": ["Performance test prompt"], "max_tokens": 50, "temperature": 0.7}' \
|
||||
2>/dev/null)
|
||||
local end_time=$(date +%s)
|
||||
local duration=$((end_time - start_time))
|
||||
|
||||
if echo "$response" | python -c "import sys, json; data=json.load(sys.stdin); exit(0 if 'outputs' in data else 1)" 2>/dev/null; then
|
||||
log_success "Performance test passed (${duration}s response time)"
|
||||
else
|
||||
log_error "Performance test failed"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Main test function
|
||||
run_tests() {
|
||||
log_info "Starting vLLM-omni serving functionality tests..."
|
||||
log_info "Model: $MODEL_PATH"
|
||||
log_info "Port: $PORT"
|
||||
log_info "Host: $HOST"
|
||||
echo "=========================================="
|
||||
|
||||
# Run all tests
|
||||
check_model
|
||||
check_environment
|
||||
test_imports
|
||||
start_server
|
||||
|
||||
# Wait a bit more for server to be fully ready
|
||||
sleep 5
|
||||
|
||||
test_health
|
||||
test_info
|
||||
test_generation
|
||||
test_performance
|
||||
test_api_client
|
||||
|
||||
# Note: Simple usage test is commented out as it starts its own server
|
||||
# test_simple_usage
|
||||
|
||||
log_success "All tests completed successfully!"
|
||||
echo "=========================================="
|
||||
log_info "Test Summary:"
|
||||
log_info "✅ Model loading and server startup"
|
||||
log_info "✅ Health and info endpoints"
|
||||
log_info "✅ Text generation functionality"
|
||||
log_info "✅ Performance metrics"
|
||||
log_info "✅ API client integration"
|
||||
log_info "✅ All imports working correctly"
|
||||
}
|
||||
|
||||
# Show usage
|
||||
show_usage() {
|
||||
echo "Usage: $0 [model_path] [port]"
|
||||
echo ""
|
||||
echo "Arguments:"
|
||||
echo " model_path Path to the model directory (default: ./models/Qwen3-0.6B)"
|
||||
echo " port Port to run the server on (default: 8000)"
|
||||
echo ""
|
||||
echo "Examples:"
|
||||
echo " $0 # Use default model and port"
|
||||
echo " $0 ./models/Qwen3-0.6B 8001 # Use specific model and port"
|
||||
echo " $0 Qwen/Qwen3-0.6B 8000 # Use HuggingFace model"
|
||||
}
|
||||
|
||||
# Main execution
|
||||
main() {
|
||||
# Check for help flag
|
||||
if [[ "$1" == "-h" || "$1" == "--help" ]]; then
|
||||
show_usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Change to script directory
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
# Create logs directory if it doesn't exist
|
||||
mkdir -p logs
|
||||
|
||||
# Run tests
|
||||
run_tests
|
||||
}
|
||||
|
||||
# Run main function with all arguments
|
||||
main "$@"
|
||||
Reference in new issue
Block a user