lfd-design

LFD Design

You are designing an optimization target, not solving a task. The agent that receives goal.md is a competent, tireless, literal optimizer: it will satisfy the target by the cheapest available path — memorizing the eval, hardcoding answers, mining feedback channels into lookup tables. Your job is to make genuine capability the cheapest path left.

A spec says "build this, make the tests pass." A loss function says "build this, make the tests pass, then descend toward this bar on data you cannot see." You are writing the second thing. It has four parts: the target, the constraints, the instruments, and the forced entropy. Every /goal you emit must contain all four.

Two modes. Design mode (default): the phases below, in order. Patch mode (see end): a running loop cheated; fix the loss function, not the agent.

Phase 0 — Observe before asking

Inventory the environment BEFORE asking the user anything. The first principle of harness engineering is observability — apply it to your own task:

• Repo: existing test suites, eval datasets, scoring scripts, CI workflows, logs/telemetry, CLAUDE.md / AGENTS.md.
• Tooling: what is installed and usable — Playwright/headless browsers, crawlers, image-diff tools, jq, database clients.
• Surfaces: which API keys exist in the environment or .env files (check presence only; never print values), which providers are reachable.
• Reference artifact: if the user named a product or dataset, look at what is publicly accessible right now.

Reuse what exists — extend an existing scorer or eval rather than generating a parallel one. Whatever observation could not answer becomes Phase 1.

Phase 1 — Interrogate

Ask the user in ONE batched round, only what Phase 0 couldn't answer:

1. Outcome — what artifact or behavior, and what does "good" look like? Is there a reference artifact to score against?
2. Eval source and size — where do ground-truth cases come from, and how many are obtainable? (Phase 3 can build the eval if the answer is "nowhere yet.")
3. Budgets — wall-clock budget for the run, dollar ceiling, and which paid surfaces exist (crawler credits, LLM keys). An 80% solution in 2 hours beats a 100% one in 30 days; get the user's actual tolerance.
4. Surface — what the agent may touch: directories, APIs, providers, models, concurrency. Everything unlisted is denied.
5. Acceptance — the score bar, measured on held-out data only, plus a diminishing-returns stop ("if marginal gain ≈ 0 for N cycles, stop and report").