vllm-omni/docs/pr_reviewer.md

VLLM-Omni PR Reviewer

Overview

The VLLM-Omni PR Reviewer is an automated code review bot powered by GLM-4.7 AI model. It helps maintain code quality by providing intelligent feedback on pull requests.

Features

• Intelligent Code Analysis: Leverages GLM-4.7 for understanding code context and providing meaningful feedback
• Comprehensive Reviews: Covers code quality, architecture, security, testing, and documentation
• Structured Output: Provides well-formatted reviews with clear sections and actionable suggestions
• Rate Limiting: Built-in cooldown mechanism to prevent excessive API usage
• Retry Logic: Automatic retries with exponential backoff for transient API failures
• Defensive Parsing: Robust validation of API responses to handle malformed data
• Cost Control: Only repository members/collaborators/owners can trigger reviews

How to Use

Triggering a Review

To trigger an automated PR review, mention the bot in a PR comment:

@vllm-omni-reviewer please review

Or include in your PR description:

@vllm-omni-reviewer

The bot will automatically review your changes and post a detailed comment.

What Gets Reviewed

• vLLM Architecture Compatibility: Ensures changes align with vLLM's design patterns
• Multi-modal Integration: Reviews audio, vision, and text processing implementations
• Performance Implications: Analyzes impact on inference latency and throughput
• Code Quality: Checks Python best practices, type hints, and documentation
• Security Considerations: Identifies potential security vulnerabilities
• Testing Coverage: Recommends additional test cases when needed

Review Output

The bot posts a structured review comment with:

• Overview: Brief summary of the PR's purpose
• Critical Issues (Must Fix): Blocking issues that need to be addressed
• Important Issues (Should Fix): Significant concerns that should be resolved
• Minor Issues & Suggestions: Small improvements and optional suggestions
• Positive Aspects: Highlights well-implemented features
• Performance Considerations: Analysis of performance impact
• Testing Recommendations: Suggestions for additional tests
• Overall Assessment: Final recommendation (Approve/Request Changes/Needs Major Work)

Rate Limiting and Cooldown

The bot includes a cooldown mechanism to prevent excessive API usage:

• Default cooldown: 5 minutes between reviews per PR
• Configurable: Can be adjusted via PR_REVIEWER_COOLDOWN_MINUTES environment variable
• Smart detection: Checks for previous bot comments before starting a review

If you trigger a review within the cooldown period, the bot will log a message and skip the review.

Architecture

┌─────────────────┐
│  PR Comment     │
│  @vllm-omni-    │
│  reviewer       │
└────────┬────────┘
         │
         ▼
┌─────────────────────────────────┐
│  GitHub Actions Workflow        │
│  (.github/workflows/            │
│   pr-reviewer.yml)              │
│                                 │
│  - Python 3.11                  │
│  - requests==2.31.0             │
│  - pyyaml==6.0.1                │
└────────┬────────────────────────┘
         │
         ▼
┌─────────────────────────────────┐
│  PR Reviewer Script             │
│  (.github/scripts/              │
│   pr_reviewer.py)               │
│                                 │
│  1. Check cooldown              │
│  2. Fetch PR details & diff     │
│  3. Build review prompt         │
│  4. Call GLM-4.7 API            │
│     (with retry logic)          │
│  5. Validate response           │
│  6. Post review comment         │
└────────┬────────────────────────┘
         │
         ▼
┌─────────────────────────────────┐
│  GLM-4.7 API                    │
│  (open.bigmodel.cn)             │
└─────────────────────────────────┘

1. GitHub Actions Workflow (.github/workflows/pr-reviewer.yml): Triggers on @mention
2. Python Script (.github/scripts/pr_reviewer.py): Fetches PR data and calls GLM-4.7 API
3. GLM-4.7 API: Provides intelligent code analysis

Testing

Testing the PR Reviewer Bot

To test the PR reviewer bot before deploying to production:

1. Create a test PR - Make a small, safe change (e.g., documentation update)
2. Open the PR - Create a pull request with a descriptive title
3. Trigger the review - Comment @vllm-omni-reviewer on the PR
4. Monitor results - Check the Actions tab for workflow execution logs

Running Unit Tests

The bot includes comprehensive unit tests that can be run locally:

# Run all tests
pytest .github/tests/test_pr_reviewer.py -v

# Run specific test
pytest .github/tests/test_pr_reviewer.py::TestCheckTrigger -v

# Run with coverage
pytest .github/tests/test_pr_reviewer.py --cov=.github/scripts/pr_reviewer.py --cov-report=term-missing

What to Look For

When testing, verify that:

• The workflow triggers on the @vllm-omni-reviewer comment
• The cooldown mechanism works correctly
• The GLM API call completes without errors (with retry if needed)
• A review comment is posted to the PR
• The review content is meaningful and well-structured
• The cost is within the expected range (0.50-5 CNY)

Safe Test Changes

For testing, consider making these types of safe changes:

• Documentation updates (like adding this Testing section)
• Comment improvements
• README enhancements
• Non-functional file additions

Example Test PR

A good test PR might:

• Update a documentation file
• Add explanatory comments
• Improve code formatting
• Fix a minor typo

These changes are safe to merge if the test is successful and won't affect functionality.

Troubleshooting

Bot Doesn't Respond

1. Check permissions - Verify you have Owner/Member/Collaborator access
2. Check Actions tab - Look for workflow execution and view logs
3. Check cooldown - If another review was posted recently, wait for the cooldown period
4. Check API key - Ensure GLM_API_KEY is configured in repository secrets

API Errors

If the GLM API call fails:

• Check the Actions tab for detailed error logs
• Verify the GLM_API_KEY secret is correctly configured
• Ensure sufficient API quota is available
• The bot will automatically retry up to 3 times with exponential backoff

Review Seems Truncated

If the review appears incomplete:

• Large diffs may be truncated at 100,000 characters
• Check the logs for truncation warnings
• Consider breaking large PRs into smaller chunks

Configuration

Required Secrets

The following secret must be configured in the repository settings:

• GLM_API_KEY - Your GLM (BigModel) API key for accessing the GLM-4.7 API

To add the secret:

1. Go to repository Settings → Secrets and variables → Actions
2. Click "New repository secret"
3. Name: GLM_API_KEY
4. Value: Your GLM API key

Optional Configuration

The following optional environment variables can be set in the workflow file:

Variable	Default	Description
GLM_API_URL	https://open.bigmodel.cn/api/paas/v4/chat/completions	GLM API endpoint
GLM_MODEL	glm-4.7	Model to use for reviews
PR_REVIEWER_COOLDOWN_MINUTES	5	Cooldown period between reviews
PR_REVIEWER_MAX_RETRIES	3	Maximum API retry attempts
PR_REVIEWER_RETRY_DELAY	1.0	Base delay for retry backoff (seconds)
PR_REVIEWER_MAX_DIFF_SIZE	100000	Maximum diff size before truncation

Workflow Customization

The workflow can be customized in .github/workflows/pr-reviewer.yml:

• Change Python version (default: 3.11)
• Adjust timeout value (default: 10 minutes)
• Modify trigger conditions
• Add additional dependencies

Code Quality

The PR reviewer script follows vllm-omni coding standards:

• Type hints: All functions have complete type hints following mypy strict mode
• Logging: Uses Python's logging module for structured logging
• Testing: Comprehensive unit tests with pytest
• Pre-commit: Script is checked by pre-commit hooks (flake8)

Cost Estimate

Component	Cost
GitHub Actions (public repo)	Free
GLM API (glm-4.7)	~0.50-5 CNY per PR (varies by size)
Total (20 PRs/month)	10-100 CNY/month ($2-15 USD)

Contributing

To improve the PR reviewer bot:

1. Edit .github/scripts/pr_reviewer.py for logic changes
2. Edit .github/workflows/pr-reviewer.yml for workflow changes
3. Add tests to .github/tests/test_pr_reviewer.py
4. Run pre-commit run --files .github/scripts/pr_reviewer.py to check code quality
5. Test thoroughly with a test PR before deploying to production

License

This bot is part of the VLLM-Omni project and follows the same license terms.