agentmods.dev

calibrate_probabilities

calibrate_probabilities is a skill for Claude Code, Codex from DanielTomaro13/sportsdata-agents. It costs 29 tokens per session (345 once invoked), scanned A, original, MIT.

A procedure for checking and correcting probability estimates so that predictions labelled with a given chance occur about that often in practice.

In plain words
What is it for?
Use it to calculate Brier scores and log-loss, adjust probabilities, and save the adjustment settings with the model.
Why use it?
It helps detect and reduce overconfident predictions by measuring performance on held-out data, meaning examples not used for fitting.

Skill for Claude CodeCodex

Written for no agent in particular: nothing here depends on one.

not rated 9repo 5d ago A scan 29 tokens original MIT

Good fit Use it to calculate Brier scores and log-loss, adjust probabilities, and save the adjustment settings with the model.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/danieltomaro13/sportsdata-agents/calibrate_probabilities
View source ↗ DanielTomaro13/sportsdata-agents sportsdata-ai.com
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add DanielTomaro13/sportsdata-agents --skill calibrate_probabilities
Clone the repo
git clone --depth 1 https://github.com/DanielTomaro13/sportsdata-agents

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for calibrate_probabilities

README.md
[![agentmods](https://agentmods.dev/badge/skills/danieltomaro13/sportsdata-agents/calibrate_probabilities/github.svg)](https://agentmods.dev/skills/danieltomaro13/sportsdata-agents/calibrate_probabilities)
Your own site
<a href="https://agentmods.dev/skills/danieltomaro13/sportsdata-agents/calibrate_probabilities"><img src="https://agentmods.dev/badge/skills/danieltomaro13/sportsdata-agents/calibrate_probabilities/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for calibrate_probabilities

Your own site · 80×15
<a href="https://agentmods.dev/skills/danieltomaro13/sportsdata-agents/calibrate_probabilities"><img src="https://agentmods.dev/badge/skills/danieltomaro13/sportsdata-agents/calibrate_probabilities.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 29 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 345 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00029 $0.00345
Opus 5.5 $0.00012 $0.00138
Sonnet 5.5 $0.00006 $0.00069
Haiku 4.5 $0.00003 $0.00034

Measured 2mo ago against content hash 2c4dd2a710fe, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

calibrate_probabilities scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

src/sportsdata_agents/skills/calibrate_probabilities/SKILL.md · 31 lines

What it actually says

Calibrating probabilities

A model that says 70% should be right ~70% of the time. Calibration is measured, never assumed.

Measure (always on holdout)

  • calibration_metrics(pairs) where pairs = holdout {prob, outcome} rows.
  • Brier: mean squared error. 0 = oracle; 0.25 = coin flip on a balanced set; beating the market baseline matters more than the absolute number.
  • Log-loss: punishes confident wrongness. If log-loss looks much worse than Brier, the model is overconfident in its tails.

Fix overconfidence (in run_python)

  • Shrink toward the base rate: p' = w * p + (1 - w) * base_rate, fit w on a validation slice (grid over w ∈ [0.5..1.0] minimising log-loss is fine).
  • Platt-style: fit logistic regression of outcome on logit(p) — two parameters, hard to overfit; refuse fancier recalibration without more than ~200 samples.
  • Re-run calibration_metrics AFTER rescaling and report both before/after.

Persist

save_model with the post-calibration metrics and the rescaling parameters in params — the next session must be able to reproduce the pipeline from the row.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2mo ago First seen · 31 lines · 29 tokens per session scan A 2c4dd2a710fe

Subscribe to this mod's changes

calibrate_probabilities is a skill published in the GitHub repository DanielTomaro13/sportsdata-agents (9 stars, last pushed 5d ago), licensed MIT. It adds 29 tokens to every session and 345 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Analytics and metricsPythonScientificai-agentsmcpsports-bettingsports-data
Related

calibrate_probabilities compared with skills from other repositories

calibrate_probabilities next to 6 skills from other repositories
ModTypeStarsLast pushLicenceScanTokensWorks with
calibrate_probabilitiesDanielTomaro13/sportsdata-agentsSkill95d agoMITA29Claude Code, Codex
jupyter-notebookNousResearch/hermes-agentSkill261kyesterdayMITA18Claude Code, Codex
matlabK-Dense-AI/scientific-agent-skillsSkill56k4d agoMITA44Claude Code
pennylaneK-Dense-AI/scientific-agent-skillsSkill56k4d agoMITA74Claude Code
cuequivariance-jaxNVIDIA/skillsSkill3.9kyesterdayApache-2.0A38Claude Code, Codex
cuequivariance-torchNVIDIA/skillsSkill3.9kyesterdayApache-2.0A37Claude Code, Codex
pysrastroautomata/PySRSkill4.2k4d agoApache-2.0A61Claude Code, Codex