Borrowing it
Nothing to install: this file belongs to alecs5am/ralphy. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/alecs5am/ralphy/main/.agents/skills/evaluator/SKILL.mdgit clone --depth 1 https://github.com/alecs5am/ralphyWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/alecs5am/ralphy/evaluator)<a href="https://agentmods.dev/skills/alecs5am/ralphy/evaluator"><img src="https://agentmods.dev/badge/skills/alecs5am/ralphy/evaluator/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/alecs5am/ralphy/evaluator"><img src="https://agentmods.dev/badge/skills/alecs5am/ralphy/evaluator.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00216 | $0.03609 |
| Opus 5.5 | $0.00086 | $0.01444 |
| Sonnet 5.5 | $0.00043 | $0.00722 |
| Haiku 4.5 | $0.00022 | $0.00361 |
Grade A, and why
evaluator scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 162 lines — stays where its author put it; the contents beside it link to each section on GitHub.
evaluator
Trigger refinements
ALSO FIRE when the user just dropped a path that ends in .mp4 from .ralphy/workspaces/<ws>/projects/<id>/render/ with no other instructions, or when an editor handed off and the user asks "and now?" (any language).
DO NOT FIRE for unrendered projects (handback to editor for ralphy render), for raw research downloads (those go through researcher's analyze-video flow, not eval), or for source media that hasn't been composed yet.
Hard invariants
- Every model call (vision pass) routes through
cli/lib/providers/llm.ts → callLLM()via the CLI. No direct OpenAI / fal calls. - Findings are deterministic outputs of
cli/lib/eval/*— don't paraphrase them; pass through verbatim to the fixer agent. - Keyframe slicing is a cheap diagnostic, NOT a ship gate (#411). A keyframe-only (or structure-only) report can NEVER mark a Unit ship-ready —
report.gate.shipReadyis hard-false on it. The final gate before forming/publishing a Unit is the native-video pass (full mp4 → model), or deep-style when a STYLE_LOCK/brief exists. Screenshot slicing misses temporal continuity, audio-picture alignment, pacing, and caption sync — exactly the failures that hallucinate when the model only sees stills.
Where this sits in the Unit lifecycle. Eval is phase 14 of the canonical Unit lifecycle. Its native-video gate is the one that flips
polishedtotrueinralphy project status <id> --contract; thequality-gate-failedandnative-gate-requiredstop conditions are derived fromeval.json'sgate/scoring.verdict. Ablockverdict feeds the repair loop (phase 15) and the optional polish council (phase 16).
You evaluate rendered UGC videos and produce a report that another agent (the fixer) can act on without reading the video itself. The contract is: the report is the handoff.
What this skill is not
- Not a researcher tool. For "analyze this TikTok/Reel from a creator I want to imitate", route to
/researcher. - Not a fixer. The findings list is meant to be read by a separate agent (or the editor / art-director / scenarist) that will execute the fixes. Don't try to fix issues from inside this skill — that's a different role and would skip the user's chance to triage.
- Not a publisher / scheduler. Verdict is informational, not a publish gate.
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 162 lines · 216 tokens per session scan A 5b6eadead27e
evaluator is a skill published in the GitHub repository alecs5am/ralphy (140 stars, last pushed 1mo ago), licensed Apache-2.0. It adds 216 tokens to every session and 3,609 once invoked, about $0.0009 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
video-gen
Use when generate videos with AI models — Runway, Kling, Sora, Pika, Seedance 2.0, Grok Imagine, Veo. Text-to-video, image-to-video, video extension, multi-modal references. Use when generating video from text prompts, animating images, or creating AI video content.
clinical-reports
Write comprehensive clinical reports including case reports (CARE guidelines), diagnostic reports (radiology/pathology/lab), clinical trial reports (ICH-E3, SAE, CSR), and patient documentation (SOAP, H&P, discharge summaries). Full support with templates, regulatory compliance (HIPAA, FDA, ICH-GCP), and validation…
imaging-data-commons
Query and download public cancer imaging data from NCI Imaging Data Commons using idc-index. Use for accessing large-scale radiology (CT, MR, PET) and pathology datasets for AI training or research. No authentication required. Query by metadata, visualize in browser, check licenses.
clinical-decision-support
Generate professional clinical decision support (CDS) documents for pharmaceutical and clinical research settings, including patient cohort analyses (biomarker-stratified with outcomes) and treatment recommendation reports (evidence-based guidelines with decision algorithms). Supports GRADE evidence grading…
molecular-cloning
Molecular cloning simulation and design. PCR amplicon prediction, restriction enzyme digestion, Golden Gate and Gibson assembly simulation, primer design, CRISPR sgRNA design, and plasmid annotation. For protein-level sequence analysis use biopython or esm; for database lookups use gene-database or ensembl-database.
geo-database
Access NCBI GEO for gene expression/genomics data. Search/download microarray and RNA-seq datasets (GSE, GSM, GPL), retrieve SOFT/Matrix files, for transcriptomics and expression analysis.