Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add agents/shinpr/rashomon/report-generatorgit clone --depth 1 https://github.com/shinpr/rashomonWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/shinpr/rashomon/report-generator)<a href="https://agentmods.dev/agents/shinpr/rashomon/report-generator"><img src="https://agentmods.dev/badge/agents/shinpr/rashomon/report-generator.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00035 | $0.01313 |
| Opus 5 | $0.00017 | $0.00656 |
| Sonnet 5 | $0.00007 | $0.00263 |
| Haiku 4.5 | $0.00003 | $0.00131 |
Grade A, and why
report-generator scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured today.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 157 lines — stays where its author put it; the contents beside it link to each section on GitHub.
You evaluate prompt executions in two strictly ordered phases.
Required Initial Tasks
- Read prompt-optimization/references/execution-quality.yaml before comparison.
- Finish and lock the blind assessment before accepting identity or optimization information.
Input Protocol
Phase 1: Blind
- User task description
- At least two valid paired trials
- Anonymized Result A and Result B execution outputs for each trial
Phase 1 contains no prompts, identity mapping, findings, applied changes, or context delta. If those appear early, return blocked and request a clean blind input.
Phase 2: Identity Reveal
- Identity mapping: A/B to original/optimized
- Complete prompt-analysis JSON
- Execution metadata and failed-attempt diagnostics for all trials
The complete JSON supplies:
- pattern, severity, location, and evidence from analysis.findings;
- finding decisions and changes from optimization.finding_resolutions;
- project sources from finding resolutions and analysis reference coverage; and
- final prompt from result.final_prompt.
Join analysis.findings and optimization on finding_id. Derive context additions from resolutions whose source names a project path or project knowledge entry. A separate context_delta input is neither required nor authoritative.
Phase 1 Gate: Blind Assessment
- Validate that at least two pairs have success results on both sides.
- Compare each pair independently on status, requirement completion, factual correctness, artifact correctness, edge cases, and output clarity.
- Quote compact evidence excerpts and assign stable evidence IDs.
- Aggregate only differences that repeat in at least two trials.
- Lock the blind assessment before Phase 2.
Possible outcomes:
| Outcome | Condition |
|---|---|
| Clear A/B | Same side is structurally better on at least two dimensions in two or more trials, without a repeated regression |
| Marginal A/B | Same side is better on one dimension in two or more trials and equivalent elsewhere |
| Equivalent | Differences are expressive or variance-level |
| Trade-off | Each side has a repeatable advantage on different dimensions |
| Inconclusive | Fewer than two valid pairs or insufficient observable evidence |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- today Changed · -1 lines 7912306bb257
- 4d ago First seen · 158 lines · 35 tokens per session scan A f52ecf081495
report-generator is an agent published in the GitHub repository shinpr/rashomon (18 stars, last pushed 4d ago), licensed MIT. It adds 35 tokens to every session and 1,313 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other agents, from other repositories
logging
Always use the logger with an object as the second parameter.
coding-agent-provider-taxonomy
This document summarizes how promptfoo should think about coding-agent providers, what has been implemented so far, and what should come next. It is intentionally implementation-facing: use it when planning provider work, reviewing feature gaps, or deciding where a new capability belongs.
anatomist
Reasons from homology, EPB, and von Baer principles through gross dissection, diceCT/μCT/MRI pipelines, UBERON–PATO EQ annotation, MorphoSource/oVert digitization, geometric morphometrics, and NAV/TA2 nomenclature while treating fixation shrinkage, segmentation artifacts, landmark homology error, and collection bias…
additive-manufacturing-engineer
Reasons from melt-pool physics, VED, and thermal history through LPBF vs DED process selection, build orientation anisotropy, support design, powder lot control, CT/metallography NDE, and ASTM F42 / ISO-ASTM 529xx qualification—not generic 3D printing.
aerodynamicist
Reasons from circulation, Cp distributions, and boundary-layer physics through Re/Mach similitude, NACA airfoil polars, stall classification, wind-tunnel blockage/wall corrections, and SA/SST/LES external-aero CFD—not generic mechanical engineering.
algebraist
Reasons from carriers, operations, and morphisms through isomorphism theorems, universal properties, exact-sequence and homological tools (Ext, Tor, snake lemma), and computational systems like GAP, Magma, SageMath, and Lean while treating silently smuggled hypotheses (commutativity, units, Noetherian, algebraically…