dspy-simba-optimizer

DSPy SIMBA Optimizer

Goal

Optimize DSPy programs using stochastic mini-batch sampling, output variability, self-reflective rules, and successful demonstrations.

When to Use

• Need lighter-weight alternative to GEPA
• Have a numeric metric that captures task quality
• Want introspective rules and demonstrations
• Budget-conscious optimization (fewer eval calls)
• Programs where few-shot examples aren't critical

Related Skills

• Alternative optimizers: dspy-miprov2-optimizer, dspy-gepa-reflective
• Agent optimization: dspy-react-agent-builder
• Evaluation: dspy-evaluation-suite

Inputs

Input	Type	Description
program	dspy.Module	Program to optimize
trainset	list[dspy.Example]	Training examples
metric	callable	Returns a numeric score
max_steps	int	Number of optimization steps
bsize	int	Mini-batch size

Outputs

Output	Type	Description
optimized_program	dspy.Module	SIMBA-optimized program

Workflow

Phase 1: Understand SIMBA

SIMBA (Stochastic Introspective Mini-Batch Ascent):

• Iterative prompt optimization with mini-batch sampling
• Identifies challenging examples with high output variability
• Generates self-reflective rules or adds successful demonstrations
• Uses the configured LM or prompt_model for introspection
• More exploratory than basic bootstrap optimization

Comparison:

• MIPROv2: Best accuracy, lots of data
• GEPA: Agentic systems, expensive
• SIMBA: Mini-batch introspection, budget-friendly
• Bootstrap: Simplest, demo-based

Phase 2: Basic SIMBA Optimization

import dspy

dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))

# Program to optimize
class QAPipeline(dspy.Module):
    def __init__(self):
        self.generate = dspy.ChainOfThought("question -> answer")

    def forward(self, question):
        return self.generate(question=question)

# Metric returns a numeric score
def qa_metric(example, pred, trace=None):
    correct = example.answer.lower() in pred.answer.lower()
    return 1.0 if correct else 0.0

# SIMBA optimizer
optimizer = dspy.SIMBA(
    metric=qa_metric,
    max_steps=10,  # Optimization iterations
    bsize=5  # Mini-batch size
)

program = QAPipeline()
compiled = optimizer.compile(program, trainset=trainset)
compiled.save("qa_simba.json")