exploration

Exploration & Discovery Pattern

Overview

The Exploration & Discovery Pattern enables agents to balance exploiting known-good strategies with exploring new, potentially better approaches. Agents that only exploit what worked before become stagnant — they miss improvements, can't adapt to changing environments, and plateau at local optima. Discovery-driven agents systematically explore their action space to find better solutions over time.

Core Principle: The best known solution is rarely the optimal solution — reserve capacity to discover what you don't yet know.

When This Skill Applies

Activate this pattern when:

• The agent faces multiple possible approaches with uncertain outcomes
• Known strategies may not be optimal — better methods may exist undiscovered
• The environment changes and previously good strategies may degrade
• A/B testing of agent strategies is needed to find better approaches
• Creative or novel solutions are more valuable than reliable familiar ones
• The agent must find solutions in a large, partially-unknown solution space

Rule of thumb: If the agent always does the same thing, use exploration. If it never commits to what works, use exploitation. The goal is the right balance.

Exploration-Exploitation Trade-off

The Core Dilemma

Exploit: Use the strategy that has worked best so far
    → Safe, predictable, but misses better options

Explore: Try new/less-tested strategies
    → Risky, uncertain, but may find better solutions

Balance: ε-greedy, UCB, Thompson Sampling
    → Systematic, principled trade-off

Key Algorithms

Algorithm	Exploration Strategy	Best For
ε-greedy	Random with prob ε	Simple, easy to implement
UCB (Upper Confidence Bound)	Explore high-uncertainty options	When you can count trials
Thompson Sampling	Sample from belief distribution	Probabilistic rewards
Boltzmann Exploration	Temperature-based softmax	Graduated exploration