Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/mxslr/mlcraft/rigorous-evaluationnpx skills add mxslr/mlcraft --skill rigorous-evaluationgit clone --depth 1 https://github.com/mxslr/mlcraftWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00156 | $0.00945 |
| Opus 5 | $0.00078 | $0.00473 |
| Sonnet 5 | $0.00031 | $0.00189 |
| Haiku 4.5 | $0.00016 | $0.00094 |
Grade A, and why
rigorous-evaluation scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 47 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Rigorous Evaluation
Report a number you can defend, not the biggest number. Choose metrics AND plots that fit the problem type. Do not pad a report with metrics that do not apply. For example, ROC-AUC and a ROC curve only make sense for binary classification.
Metrics and plots by problem type
| Problem | Core metrics | Plots to show | Do not use |
|---|---|---|---|
| Binary classification | Accuracy, Precision, Recall, F1, ROC-AUC, PR-AUC | confusion matrix, ROC curve, PR curve | none |
| Multiclass (more than 2) | Accuracy, macro and weighted Precision/Recall/F1 | confusion matrix, per-class metric bars | a single ROC curve |
| Multilabel | per-label F1, micro and macro F1, mAP | per-label PR curves | plain accuracy |
| Object detection (faces, vehicles, objects) | [email protected], [email protected]:0.95, per-class AP | PR curve per class, sample images with predicted boxes | accuracy, ROC-AUC |
| Segmentation | mean Dice, mean IoU, per-class IoU | mask overlays, per-class IoU bars | ROC-AUC |
| Regression | MAE, RMSE, R2, MAPE | predicted vs actual, residual plot | F1, AUC |
| Ranking or retrieval | nDCG@k, MAP, Recall@k, MRR | recall@k curve | accuracy |
| Face verification | ROC and EER, TAR at fixed FAR | verification ROC | plain accuracy |
| Forecasting | MAE, RMSE, MASE, sMAPE | forecast vs actual, error per horizon | AUC, F1 |
| Clustering | silhouette, ARI, NMI when labels exist | 2D projection | accuracy |
Binary vs multiclass, the common mistake
- ROC-AUC and a ROC curve are for binary problems. With more than two classes do not draw one ROC curve. Use the confusion matrix as the main visual and report macro and weighted F1. Compute one-vs-rest macro AUC only if a ranking metric is genuinely needed.
- Pick ONE headline metric that matches the cost of errors: recall for screening, precision when false positives are expensive, mAP for detection, RMSE for regression.
- If a metric does not fit the problem, omit it. A short relevant report is stronger than a padded one.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 47 lines · 156 tokens per session scan A 829c5791ad03
rigorous-evaluation is a skill published in the GitHub repository mxslr/mlcraft (8 stars, last pushed 1mo ago), licensed MIT. It adds 156 tokens to every session and 945 once invoked, about $0.0008 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
torch-performance-optimization
Optimize or review eager CPU-only Albucore PyTorch runtime paths with benchmark-backed decisions. Use when adding or changing Torch CPU kernels, Tensor/NumPy bridges, Torch backend routing, tensor layouts, allocations, threading, profiling, memory-format candidates, or Torch performance benchmarks.
ml-for-aec
Computer vision for buildings, image-to-floorplan, generative ML models, performance prediction, structural analysis ML, energy prediction, natural language to design, and point cloud ML for AEC computational design.
yolo-export
Use when exporting or deploying Ultralytics YOLO models in Platform or code — the Platform Export tab and yolo export/model.export() for ONNX, TensorRT, CoreML, OpenVINO, LiteRT, NCNN, ExecuTorch, and NPUs (RKNN, QNN, Hailo, Ascend, IMX, Axelera, DeepX), FP16/INT8 quantization, benchmarking, and non-Python runtimes.…
yolo-training
Use when training, fine-tuning, or validating Ultralytics YOLO models in Platform, cloud GPUs, or local code — model.train(), yolo train/val, remote metric streaming, epochs, batch, imgsz, devices, augmentation, multi-GPU, resumes, results, and fixing OOM, NaN loss, low mAP, or overfitting. For hyperparameter search…
yolo-datasets
Use when uploading, annotating, building, converting, analyzing, or debugging datasets in Ultralytics Platform or local YOLO — Platform dataset management and Smart Annotation, data.yaml, YOLO label .txt formats, COCO/DOTA/mask conversion, auto-labeling, splits, validation, and errors like "no labels found" or mAP…
yolo-models
Use when choosing or comparing Ultralytics models in Platform or code — picking a model family (YOLO26/YOLO11/YOLOv8, YOLO-World, YOLOE, SAM/SAM2/FastSAM, RT-DETR, YOLO-NAS), size (n/s/m/l/x), task variant (-seg, -sem, -cls, -pose, -obb, -depth), pretrained checkpoint, open-vocabulary or promptable…