learn-from-code-review

Learn from Code Review

Analyze code review comments from GitHub pull requests and distill them into reusable skills or repository guidelines that improve future code quality.

Overview

Code review feedback contains valuable institutional knowledge that often gets buried across hundreds of PRs. This skill extracts meaningful patterns from review comments and transforms them into:

1. Repository-specific skills - Placed in .openhands/skills/ for domain-specific patterns
2. AGENTS.md guidelines - Overall repository conventions and best practices

Prerequisites

• GITHUB_TOKEN environment variable must be set
• GitHub CLI (gh) should be available

Workflow

Step 1: Identify Target Repository

Determine the repository to analyze:

# Get current repo info
gh repo view --json nameWithOwner -q '.nameWithOwner'

If not in a repository, ask the user which repository to analyze.

Step 2: Fetch Review Comments

Retrieve PR review comments from the repository:

# Fetch merged PRs from the last 30 days (adjustable)
gh pr list --repo {owner}/{repo} \
  --state merged \
  --limit 50 \
  --json number,title,mergedAt

# For each PR, fetch review comments
gh api repos/{owner}/{repo}/pulls/{pr_number}/comments \
  --jq '.[] | {body: .body, path: .path, user: .user.login, created_at: .created_at}'

# Also fetch review-level comments (not tied to specific lines)
gh api repos/{owner}/{repo}/pulls/{pr_number}/reviews \
  --jq '.[] | select(.body != "") | {body: .body, user: .user.login, state: .state}'

Step 3: Filter and Categorize Comments

Apply noise filtering to keep only meaningful feedback:

Exclude:

• Bot comments (dependabot, copilot, github-actions, etc.)
• Low-signal responses ("LGTM", "+1", "looks good", "thanks", "nice")
• Comments shorter than 30 characters
• Auto-generated comments (CI status, coverage reports)

Categorize remaining comments by:

• Security concerns
• Performance patterns
• Code style/conventions
• Architecture/design patterns
• Error handling
• Testing requirements
• Documentation standards