csoai-review

csoai-review is a cursor rule for Cursor from CSOAI-ORG/councilof-ai. It costs 517 tokens per session, scanned A, original, MIT.

Code-review rules based on recurring defects in the Council of AI project. They require systems to distinguish failed or unmeasured results from genuine failures and to reject empty or unsupported checks.

In plain words
What is it for?
Use them when reviewing graders, scorers, validation checks, confidence intervals, rankings, or functions whose names claim to verify or refuse something.
Why use it?
They help reviewers catch false passes, misleading scores, unsupported claims, and unreliable small-sample results. The rules focus on preventing a component from reporting success when it did not complete its work.

Cursor rule for Cursor

Written for Cursor: installed under .cursor/.

Good fit Use them when reviewing graders, scorers, validation checks, confidence intervals, rankings, or functions whose names claim to verify or refuse something.

Compare 6 cursor rules from other repositories ↓
Install with agentmods
npx agentmods add rules/csoai-org/councilof-ai/csoai-review
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Clone the repo
git clone --depth 1 https://github.com/CSOAI-ORG/councilof-ai

Made for: Cursor.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for csoai-review

README.md
[![agentmods](https://agentmods.dev/badge/rules/csoai-org/councilof-ai/csoai-review/github.svg)](https://agentmods.dev/rules/csoai-org/councilof-ai/csoai-review)
Your own site
<a href="https://agentmods.dev/rules/csoai-org/councilof-ai/csoai-review"><img src="https://agentmods.dev/badge/rules/csoai-org/councilof-ai/csoai-review/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for csoai-review

Your own site · 80×15
<a href="https://agentmods.dev/rules/csoai-org/councilof-ai/csoai-review"><img src="https://agentmods.dev/badge/rules/csoai-org/councilof-ai/csoai-review.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 517 This file is loaded in full into every session.
When invoked 517 The same file — it is already loaded in full.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00517 $0.00517
Opus 5 $0.00259 $0.00259
Sonnet 5 $0.00103 $0.00103
Haiku 4.5 $0.00052 $0.00052

Measured 11d ago against content hash 1a82a83f44d6, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-11, from the pricing page.

Security

Grade A, and why

csoai-review scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 11d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.cursor/rules/csoai-review.mdc · 38 lines

What it actually says

CSOAI review rules — flag these, they are real recurring defects

The estate's one rule: a component must be structurally unable to report success on a path it did not complete. Not "log the error" — unable. In review, flag any of the following as a bug, not a nit:

  1. Two outcomes where there must be three. Any grader/scorer that maps an unparsed / empty / failed result to 0 or "wrong" instead of UNMEASURED. Unparsed must leave the denominator, never score as incorrect. (This shipped four times in one day.)

  2. A guard that can pass on an empty scan. A contamination / validation check that reads zero rows and returns "clean" is a false pass. An empty scan must be fatal, not silent.

  3. An interval below the gate. Any confidence interval / ranking published below usable_n >= 30, including ours. Small-n must withhold the interval explicitly.

  4. A claim without its test. A capability asserted in a name, comment, or string with no code that enforces it. If a function is called verify_* / refuse_* / check_*, the body must actually do it.

  5. Sign AND significance. A "beats"/"improves" verdict that checks only whether a CI excludes zero, not whether the delta is positive in sign. (An earlier verdict printed "beats" for a −9.16 delta.)

  6. Contamination by row instead of by theme. Held-out splits must be theme-level: if any item in a statutory theme is in training, the whole theme leaves the clean split.

  7. Secrets in code or logs. No tokens/keys inline; secrets come via keystone. Flag any printed secret.

  8. Silent truncation / caps. Any top-N, sampling, or no-retry bound that is not logged. Silent truncation reads as "covered everything" when it didn't.

When unsure whether something is a defect: if the code could report success on a path it didn't complete, it is one.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 11d ago First seen · 38 lines · 517 tokens per session scan A 1a82a83f44d6

Subscribe to this mod's changes

csoai-review is a cursor rule published in the GitHub repository CSOAI-ORG/councilof-ai (0 stars, last pushed today), licensed MIT. It adds 517 tokens to every session, about $0.0026 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.