caio-review

caio-review is a skill for Codex from bestagentkits/agency-skills. It costs 77 tokens per session (1,469 once invoked), scanned A, original, MIT.

A review checklist for plans involving artificial intelligence, such as choosing a model, measuring quality, estimating costs, hiring, or assessing regulation. An evaluation set is a collection of representative examples used to test whether an AI system works well.

In plain words
What is it for?
Use it before shipping an AI feature, choosing between an API and a self-hosted model, fine-tuning, signing a long-term AI contract, or launching regulated AI use cases.
Why use it?
It helps expose missing tests, unclear error targets, cost assumptions, and regulatory risks before an AI project moves forward.

Skill for Codex

Written for Codex: agents/openai.yaml present.

Needs its repository: it reads a path above its own folder, which exists only inside the repository. The line is python ../../../skills/chief-ai-officer-advisor/scripts/model_buildvsbuy_calculator.py use_case.json.

Good fit Use it before shipping an AI feature, choosing between an API and a self-hosted model, fine-tuning, signing a long-term AI contract, or launching regulated AI use cases.

Compare 6 skills from other repositories ↓
Install

Getting it into your agent

It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.

Clone the repo
git clone --depth 1 https://github.com/bestagentkits/agency-skills
agentmods
npx agentmods add skills/bestagentkits/agency-skills/caio-review

Made for: Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for caio-review

README.md
[![agentmods](https://agentmods.dev/badge/skills/bestagentkits/agency-skills/caio-review/github.svg)](https://agentmods.dev/skills/bestagentkits/agency-skills/caio-review)
Your own site
<a href="https://agentmods.dev/skills/bestagentkits/agency-skills/caio-review"><img src="https://agentmods.dev/badge/skills/bestagentkits/agency-skills/caio-review/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for caio-review

Your own site · 80×15
<a href="https://agentmods.dev/skills/bestagentkits/agency-skills/caio-review"><img src="https://agentmods.dev/badge/skills/bestagentkits/agency-skills/caio-review.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 77 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 1,469 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00077 $0.01469
Opus 5 $0.00039 $0.00734
Sonnet 5 $0.00015 $0.00294
Haiku 4.5 $0.00008 $0.00147

Measured 9d ago against content hash 73c24c279d01, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-12, from the pricing page.

Security

Grade A, and why

caio-review scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 9d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/claude-skills/caio-review/SKILL.md · 141 lines

How it starts

The opening of the file, as written. The whole thing — 141 lines — stays where its author put it; the contents beside it link to each section on GitHub.

/cs:caio-review — CAIO Forcing Questions

Command: /cs:caio-review <plan>

The eval-demanding CAIO pressure-tests any plan that involves AI. Six questions before any AI feature ships, any multi-year vendor commitment, or any AI team expansion.

When to Run

  • Before shipping any new AI-powered feature
  • Before signing a multi-year AI vendor contract (API or self-hosted infra)
  • Before EU launch of any AI feature
  • Before a major AI team hire (especially ML engineer or research scientist)
  • Before a fine-tuning project commitment
  • Before adopting AI in a regulated domain (employment, credit, healthcare, education, etc.)
  • When the founder uses the word "AI" near "competitive advantage" or "moat"

The Six CAIO Questions

1. What does this AI need to be good at, and how would you measure it?

No eval set = no ship. Before any AI feature deploys, define the eval criteria.

  • 50-100 representative inputs minimum
  • Expected outputs OR rubric for grading
  • Edge cases: ambiguous, adversarial, format-edge
  • If you can't write down what "good" looks like, you don't have a feature; you have a vibe.

2. What's the SLO on hallucination / error rate, and what's the fallback?

Every AI feature has a failure mode. Plan for it.

  • Quantified SLO: "<5% hallucination on factual queries"
  • Detection mechanism: monitoring, sampling, customer feedback loop
  • Fallback: human-in-loop review, lower-risk default response, refuse-to-answer
  • Blast radius if SLO breached: how many users affected, what is the cost?

3. What's the risk tier under EU AI Act, and is conformity assessment required?

Run ai_risk_classifier.py if any EU residents are affected OR domain is regulated.

  • PROHIBITED → cannot launch in EU; re-scope
  • HIGH → conformity assessment + EU DB registration + 10 Articles of obligations (3-12 months, $50-200K)
  • LIMITED → transparency obligations (chatbot disclosure, AI-generated content marking)
  • MINIMAL → no specific obligations; NIST AI RMF voluntary

Read the full file on GitHub · 141 lines

Files

What ships with it

1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 9d ago First seen · 141 lines · 77 tokens per session scan A 73c24c279d01

Subscribe to this mod's changes

caio-review is a skill published in the GitHub repository bestagentkits/agency-skills (11 stars, last pushed 2mo ago), licensed MIT. It adds 77 tokens to every session and 1,469 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.

Related

Other skills, from other repositories

analyze-generative-diffusion-model

Analyze pre-trained generative diffusion models (Stable Diffusion, DALL-E, Flux) by computing quality metrics (FID, IS, CLIP score, precision/recall), inspecting noise schedules, extracting and visualizing attention maps, and probing latent spaces. Use when evaluating a pre-trained generative diffusion model's output…

pjt222/agent-almanac · 104 tokens

deploy-edge-ai-model

Deploy machine learning models to edge devices using Google AI Edge Gallery, TensorFlow Lite, ONNX Runtime, and MediaPipe. Covers model quantization (INT8/INT4), on-device inference with Gemma 4 models, Android/iOS deployment via AI Edge Gallery, hardware delegate selection (GPU/NPU/DSP), and performance benchmarking…

pjt222/agent-almanac · 108 tokens

build-feature-store

Build a feature store using Feast for centralized feature management, configure offline and online stores for batch and real-time serving, define feature views with transformations, and implement point-in-time correct joins for ML pipelines. Use when managing features for multiple ML models, ensuring training-serving…

pjt222/agent-almanac · 86 tokens

deploy-ml-model-serving

Deploy machine learning models to production serving infrastructure using MLflow, BentoML, or Seldon Core with REST/gRPC endpoints, implement autoscaling, monitoring, and A/B testing capabilities for high-performance model inference at scale. Use when deploying trained models for real-time inference, setting up REST…

pjt222/agent-almanac · 94 tokens

design-serialization-schema

Design serialization schemas using JSON Schema, Protocol Buffer definitions, or Apache Avro. Covers schema versioning, backwards compatibility, validation rules, and evolution strategies for long-lived data formats. Use when defining a new API contract or data interchange format, adding fields to an existing schema…

pjt222/agent-almanac · 82 tokens

数据搬运工

A general data-transfer and format-conversion tool for moving, cleaning, and reshaping files. It works with spreadsheets, CSV and TSV tables, JSON, XML, YAML, SQL inserts, and SQLite databases.

laborany/laborany · 45 tokens