karpathy-practice-environments

Skill 14: LLM Textbook + Practice Environments（LLM教科书 + 练习环境）

Source: https://x.com/karpathy/status/1885026028428681698 | https://x.com/karpathy/status/1960803117689397543 "Take LLMs to school" | "Environments for RL" posts

Core Principle

LLMs learn like students. Give them textbooks, worked problems, and practice gyms.

Karpathy's insight: the missing ingredient in LLM training isn't more parameters — it's practice environments where agents can try → fail → see feedback → try again. Like a student who reads theory but also needs problem sets.

Apply this to your agents: build environments where they can practice a skill with automatic scoring, not just generate output into the void.

The Three Training Data Types

Every good learning environment needs all three:

1. EXPOSITION (pretrain equivalent)
   → Background knowledge, concepts, context
   → The "textbook chapter" the agent reads before practicing
   
2. WORKED EXAMPLES (SFT equivalent)
   → Complete input-output pairs with reasoning shown
   → "Here's a solved problem — learn the pattern"
   
3. PRACTICE PROBLEMS with feedback (RL equivalent)
   → Problems with verifiable correct answers
   → Automatic scoring so agent knows how it did
   → Enough variety that memorization doesn't work

Building a Practice Gym (General Template)

For any skill you want an agent to get better at:

#!/usr/bin/env python3
"""
Practice Gym for: [SKILL_NAME]
Karpathy-style RL environment for agent skill development
"""

import json
import random
from typing import Callable

class PracticeGym:
    """
    A practice environment where an agent can repeatedly attempt
    a task and receive automatic feedback.
    """
    
    def __init__(self, task_generator: Callable, scorer: Callable):
        self.task_generator = task_generator  # generates new practice problems
        self.scorer = scorer                   # returns 0.0-1.0 score for an attempt
        self.history = []
    
    def sample_task(self):
        """Generate a new practice problem."""
        return self.task_generator()
    
    def evaluate(self, task, attempt):
        """Score an agent's attempt. Returns dict with score + feedback."""
        score = self.scorer(task, attempt)
        result = {
            "task": task,
            "attempt": attempt,
            "score": score,
            "passed": score >= 0.8,
        }
        self.history.append(result)
        return result
    
    def summary(self):
        """Summarize performance across all attempts."""
        if not self.history:
            return "No attempts yet."
        scores = [r["score"] for r in self.history]
        return {
            "total_attempts": len(scores),
            "avg_score": sum(scores) / len(scores),
            "pass_rate": sum(1 for s in scores if s >= 0.8) / len(scores),
            "recent_trend": "improving" if scores[-1] > scores[0] else "declining"
        }