reflect-appworld-failure

Reflect on AppWorld Failure

Analyze failed AppWorld tasks to extract specific, actionable learnings that can be added to the playbook.

Purpose

When an AppWorld task fails, the Reflector calls this Skill with error details and failed code. You analyze the failure semantically and generate a high-quality bullet with:

1. Specific title describing the pattern
2. Detailed content with working code examples
3. Relevant tags for retrieval
4. Appropriate confidence level

Input Format

The input will be a text description with sections:

# Task
<task instruction>

## Apps
<comma-separated list of apps used>

## Error Type
<error_type: api_misuse, logic_error, timeout, etc.>

## Error Messages
<list of error messages from execution>

## Failed Code Snippet
<relevant code that failed>

## Missing Patterns (from heuristics)
<list of patterns the old system identified>

## Suggested Fixes (from heuristics)
<list of fix suggestions>

Your Analysis Process

1. Identify Root Cause: What was the fundamental mistake?

   • Wrong API method name?
   • Missing authentication?
   • Incorrect data structure access?
   • Logic error?

2. Extract Pattern: What general pattern does this represent?

   • Is this specific to one app or applies to multiple?
   • Is this about API order (login first)?
   • Is this about method naming conventions?
   • Is this about data validation?

3. Generate Concrete Example: Create working code that demonstrates the CORRECT pattern

4. Write Actionable Bullet: Make it specific enough that the Generator can apply it

Output Format

Return a JSON object with this structure:

{
  "bullet": {
    "id": "bullet-YYYY-MM-DD-HHMMSS",
    "title": "<Specific pattern title>",
    "content": "<Detailed explanation with working code example>",
    "tags": ["app.<app_name>", "<error_category>", "<pattern_type>"],
    "evidence": [
      {
        "type": "execution",
        "ref": "<task_id>",
        "note": "<brief note about failure>"
      }
    ],
    "confidence": "high|medium|low",
    "scope": "app|global"
  }
}