polymarket-brier

Polymarket Brier Score

Calculate prediction accuracy for any Polymarket address. Brier Score measures how close a trader's positions were to the actual outcomes — independent of PnL, position size, or trading strategy.

Why Brier Score?

PnL tells you who made money. Brier Score tells you who predicted correctly.

• High PnL + Poor Brier → Market maker or arbitrageur (earns spread, not predictions)
• Low PnL + Good Brier → Accurate predictor with small positions or conservative sizing
• High PnL + Good Brier → Skilled predictor who also sizes well — the real signal

Trigger

User asks about prediction accuracy, Brier score, prediction quality, forecasting skill, or "how accurate is this trader".

Background

The Brier Score is a standard metric in forecasting (used by Metaculus, Good Judgment Project, etc.).

Formula: BS = (1/N) × Σ(forecast_probability - actual_outcome)²

• 0.0 = perfect prediction (always right with 100% confidence)
• 0.25 = no skill (equivalent to random 50/50 guessing)
• 0.5+ = worse than random

Lower is better.

Input

Same as polymarket-profile — accepts 0x address, username, or profile URL.

Execution

Step 1: Fetch Settled Positions

curl -s "https://data-api.polymarket.com/positions?user={ADDRESS}&sizeThreshold=0&limit=100&offset=0"

Paginate until all positions are fetched (same as polymarket-profile).

Filter to settled positions only: redeemable == true.

For each settled position, extract:

• outcome: "Yes" or "No" — the side the trader held
• avgPrice: the trader's average entry price = their implied probability forecast
• currentValue > 0: whether the market resolved in their favor (won) or not (lost)

Step 2: Determine Actual Outcome

For each settled position:

• If currentValue > 0 (won): the outcome the trader bet on did happen → actual = 1
• If currentValue == 0 (lost): the outcome the trader bet on did not happen → actual = 0

Step 3: Calculate Brier Score