playtest-protocol

Playtest Protocol

Game design knowledge comes from tests, and most design plans are wrong in ways only a test reveals. The process failures are symmetrical: overplanning builds detailed castles on unvalidated foundations, underplanning thrashes with no direction. This skill designs test protocols, orders work by dependency risk, and audits repos for testability.

Core concepts

Plan to the horizon, not past it. The planning horizon is how far ahead design decisions stay reliable, and it is shorter than intuition says: uncertainty compounds through every dependent decision. Detailed plans beyond the horizon are fiction with formatting. Plan tightly inside the horizon, sketch beyond it, and expect the sketch to change.

The dependency stack orders the work. Some design decisions rest on others: the ability system rests on the combat loop, the combat loop rests on the core verb feeling good. Test foundational layers first, because a change at the bottom invalidates everything built on top. The most expensive mistake in game development is polishing content on a core loop that later changes.

Grayboxing tests structure before art. A graybox build uses placeholder visuals to test whether the mechanics and level structure work at all. It demands a specific evaluation skill: judging a graybox requires imagining the finished feel through the placeholder, and most people (including devs) cannot do it untrained. Adding art too early is premature production; it makes the build prettier and the test results noisier, and it welds the team to decisions that have not earned permanence.

Testers report feelings accurately and causes badly. Players misattribute: they blame the visible (graphics, one boss) for problems in the invisible (pacing, information starvation). Harvest the what and when of their emotions; discard their whys and their proposed fixes, or rather, treat a proposed fix as a pointer to a felt problem nearby.

Test protocol determines data quality. Self-testing is fast and biased; the dev knows too much. Over-the-shoulder observation of a fresh player is the workhorse: watch silently, log where they look, stall, and quit, and interview afterward with open questions ("what was going through your mind at the bridge?") rather than leading ones ("did you like the bridge fight?"). Fresh testers expire; each person's first contact happens once. Small samples per iteration beat large samples per quarter.