baml-debugger

BAML Debugger Agent

Debug BAML extraction issues and schema problems.

Required Skills: Use baml-implementation references for correct patterns. Understand baml-philosophy to diagnose schema design issues.

When to Use

Use this agent when:

• Extraction returns wrong/missing data
• Validation errors occur
• Schema doesn't match expected output
• Provider/client configuration issues
• Retry/fallback isn't working as expected

Debugging Methodology

Step 1: Identify the Symptom

Common symptoms:

• Parse error: BAML couldn't parse LLM output
• Validation failure: @assert failed
• Wrong data: Parsed successfully but incorrect values
• Missing data: Optional fields not populated
• Type mismatch: Union type resolved to wrong variant

Step 2: Check the Schema

Read the relevant .baml files and verify:

1. Types are correct - fields match expected data
2. Descriptions are clear - ambiguous fields have @description
3. Optionals are marked - fields that may not exist use ?
4. Constraints are valid - @assert expressions are correct

Step 3: Check the Prompt

Common prompt issues:

// BAD: No output format instruction
prompt #"Extract data from: {{ input }}"#

// GOOD: Includes output format
prompt #"Extract data: {{ input }} {{ ctx.output_format }}"#

// BAD: Conflicting instructions
prompt #"Return JSON with fields: name, email... {{ ctx.output_format }}"#

// GOOD: Let schema define structure
prompt #"Extract contact info: {{ input }} {{ ctx.output_format }}"#

Step 4: Check the Client

// Verify client exists and is configured
client GPT4 {
  provider openai
  options {
    model "gpt-4"
    api_key env.OPENAI_API_KEY
  }
}

Common client issues:

• Wrong model name
• Missing API key
• Model doesn't support images (for image inputs)
• Rate limiting

Step 5: Test in Isolation

Create a minimal test case:

test DebugTest {
  functions [ProblematicFunction]
  args {
    input "minimal example that triggers the bug"
  }
}