file-issues

File Issues from Evaluation

Overview

This SOP parses evaluation reports from brazil-bench attempts and creates individual GitHub issues on each attempt's repository for every shortcoming, warning, or improvement opportunity identified during the review.

Parameters

• attempt_repo (required): Repository name (e.g., 2025-12-13-python-claude-hive)
• evaluation_path (optional, default: ./results/{attempt_repo}.md): Path to evaluation report
• dry_run (optional, default: false): If true, show issues that would be created without creating them

Steps

1. Load Evaluation Report

Read the evaluation report for the specified attempt.

Constraints:

• You MUST verify the evaluation report exists before proceeding
• You MUST parse markdown sections to identify shortcomings
• You MUST NOT proceed if the report is missing or malformed

# Verify report exists
ls ./results/{attempt_repo}.md

# Read the report content
cat ./results/{attempt_repo}.md

2. Verify Repository Access

Confirm the attempt repository exists and is writable.

Constraints:

• You MUST verify the repo exists on GitHub
• You MUST check that issues are enabled on the repo
• You SHOULD check for existing issues to avoid duplicates
• You SHOULD use default GitHub labels (no custom labels required)

# Verify repo exists
gh repo view brazil-bench/{attempt_repo}

# List existing issues to check for duplicates
gh issue list -R brazil-bench/{attempt_repo} --limit 100

# List available labels (use default GitHub labels)
gh label list -R brazil-bench/{attempt_repo}

Default GitHub Labels to Use:

Issue Type	Default Label	Reason
Missing requirement	enhancement	It's a feature that needs to be added
Test quality issue	bug	Tests not working as expected
Compliance summary	enhancement	Tracking improvements needed
Quality concern	enhancement	Code improvement request
Documentation gap	documentation	Missing or incomplete docs