Borrowing it
Nothing to install: this file belongs to saski/arnesto. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/saski/arnesto/main/.agents/skills/trustworthy-experiments/SKILL.mdgit clone --depth 1 https://github.com/saski/arnestoWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/saski/arnesto/trustworthy-experiments)<a href="https://agentmods.dev/skills/saski/arnesto/trustworthy-experiments"><img src="https://agentmods.dev/badge/skills/saski/arnesto/trustworthy-experiments.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00071 | $0.00540 |
| Opus 5 | $0.00036 | $0.00270 |
| Sonnet 5 | $0.00014 | $0.00108 |
| Haiku 4.5 | $0.00007 | $0.00054 |
Grade A, and why
trustworthy-experiments scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
This is a copy
100% identical to trustworthy-experiments — 0 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.
How it starts
The opening of the file, as written. The whole thing — 56 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Domain Context
This skill implements a proven product management framework. The approach combines best practices from industry leaders and is designed for practical application in day-to-day PM work.
Input Requirements
- Context about your product, feature, or problem
- Relevant data, research, or constraints (recommended but optional)
- Clear articulation of what you're trying to achieve
Trustworthy Experiments
What It Is
Trustworthy Experiments is a framework for running controlled experiments (A/B tests) that produce reliable, actionable results. The core insight: most experiments fail, and many "successful" results are actually false positives.
The key shift: Move from "Did the experiment show a positive result?" to "Can I trust this result enough to act on it?"
Ronny Kohavi, who built experimentation platforms at Microsoft, Amazon, and Airbnb, found that:
- 66-92% of experiments fail to improve the target metric
- 8% of experiments have invalid results due to sample ratio mismatch alone
- When the base success rate is 8%, a P-value of 0.05 still means 26% false positive risk
When to Use It
Use Trustworthy Experiments when you need to:
- Design an A/B test that will produce valid, actionable results
- Determine sample size and runtime for statistical power
- Validate experiment results before making ship/no-ship decisions
- Build an experimentation culture at your company
- Choose metrics (OEC) that balance short-term gains with long-term value
- Diagnose why results look suspicious (Twyman's Law)
- Speed up experimentation without sacrificing validity
When Not to Use It
Don't use controlled experiments when:
- You don't have enough users — Need tens of thousands minimum
- The decision is one-time — Can't A/B test mergers or acquisitions
- There's no real user choice — Employer-mandated software
- You need immediate decisions — Experiments need time
- The metric can't be measured — No experiment without observable outcomes
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 3d ago First seen · 56 lines · 71 tokens per session scan A 2dc69fa061d7
trustworthy-experiments is a skill published in the GitHub repository saski/arnesto (5 stars, last pushed today), licensed Unlicense. It adds 71 tokens to every session and 540 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. It is 100% identical to trustworthy-experiments, differing in 0 lines, and is treated as a copy.
Other skills, from other repositories
readonly-review
Run a no-network read-only review practice with two explorer steps and one synthesis step.
repo-harness-assessment
Use when evaluating repository agent-readiness, mapping harness roles, choosing the next smallest improvement, or designing and reconciling agent entrypoints such as AGENTS.md, CLAUDE.md, GEMINI.md, Cursor rules, or GitHub instructions.
work-state-and-delivery
Use when designing or reconciling design docs, task boards, external trackers, execution plans, delivery records, handoffs, review evidence, task-to-change traceability, or repository commit-coupling policy.
repo-contracts-and-boundaries
Use when turning architecture, layering, ownership, dependency direction, schemas, structural metrics, quality thresholds, baselines, allowlists, or generated quality snapshots into repository checks.
runtime-evidence-and-tracing
Use when connecting observed behavior, logs, metrics, request IDs, run IDs, screenshots, traces, external dependency results, or artifacts into a runtime evidence loop.
validation-harness-design
Use when designing repository validation commands, doctor scripts, test matrices, JSON or JUnit outputs, CI gates, smoke checks, or harness command surfaces.