dspy-gepa-reflective

DSPy GEPA Optimizer

Goal

Optimize complex agentic systems using LLM reflection on full execution traces with Pareto-based evolutionary search.

When to Use

• Agentic systems with tool use
• When you have rich textual feedback on failures
• Complex multi-step workflows
• Instruction-only optimization needed

Related Skills

• For non-agentic programs: dspy-miprov2-optimizer, dspy-bootstrap-fewshot
• Measure improvements: dspy-evaluation-suite

Inputs

Input	Type	Description
program	dspy.Module	Agent or complex program
trainset	list[dspy.Example]	Training examples
metric	callable	Accepts five arguments and returns dspy.Prediction(score=..., feedback=...)
reflection_lm	dspy.LM	Strong LM for reflection (GPT-4)
auto	str	"light", "medium", "heavy"

Outputs

Output	Type	Description
compiled_program	dspy.Module	Reflectively optimized program

Workflow

Phase 1: Define Feedback Metric

GEPA requires metrics that return textual feedback:

def gepa_metric(example, pred, trace=None, pred_name=None, pred_trace=None):
    """Return score and actionable feedback for GEPA reflection."""
    is_correct = example.answer.lower() in pred.answer.lower()
    
    if is_correct:
        feedback = "Correct. The answer accurately addresses the question."
    else:
        feedback = f"Incorrect. Expected '{example.answer}' but got '{pred.answer}'. The model may have misunderstood the question or retrieved irrelevant information."
    
    return dspy.Prediction(score=float(is_correct), feedback=feedback)

Phase 2: Setup Agent

import dspy

def search(query: str) -> list[str]:
    """Search knowledge base for relevant information."""
    rm = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')
    results = rm(query, k=3)
    return results if isinstance(results, list) else [results]

def calculate(expression: str) -> float:
    """Safely evaluate mathematical expressions."""
    with dspy.PythonInterpreter() as interp:
        return interp(expression)

agent = dspy.ReAct("question -> answer", tools=[search, calculate])