Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add build-with-dhiraj/ai-workflow-framework-portability-kit --skill error-analysisgit clone --depth 1 https://github.com/build-with-dhiraj/ai-workflow-framework-portability-kitWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/build-with-dhiraj/ai-workflow-framework-portability-kit/error-analysis)<a href="https://agentmods.dev/skills/build-with-dhiraj/ai-workflow-framework-portability-kit/error-analysis"><img src="https://agentmods.dev/badge/skills/build-with-dhiraj/ai-workflow-framework-portability-kit/error-analysis/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/build-with-dhiraj/ai-workflow-framework-portability-kit/error-analysis"><img src="https://agentmods.dev/badge/skills/build-with-dhiraj/ai-workflow-framework-portability-kit/error-analysis.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00053 | $0.01745 |
| Opus 5.5 | $0.00021 | $0.00698 |
| Sonnet 5.5 | $0.00011 | $0.00349 |
| Haiku 4.5 | $0.00005 | $0.00175 |
Grade A, and why
error-analysis scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
This is a copy
100% identical to error-analysis — 0 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.
How it starts
The opening of the file, as written. The whole thing — 165 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Error Analysis
Guide the user through reading LLM pipeline traces and building a catalog of how the system fails.
Overview
- Collect ~100 representative traces
- Read each trace, judge pass/fail, and note what went wrong
- Group similar failures into categories
- Label every trace against those categories
- Compute failure rates to prioritize what to fix
Core Process
Step 1: Collect Traces
Capture the full trace: input, all intermediate LLM calls, tool uses, retrieved documents, reasoning steps, and final output.
Target: ~100 traces. This is roughly where new traces stop revealing new kinds of failures. The number depends on system complexity.
From real user data (preferred):
- Small volume: random sample
- Large volume: sample across key dimensions (query type, user segment, feature area)
- Use embedding clustering (K-means) to ensure diversity
From synthetic data (when real data is sparse):
- Use the generate-synthetic-data skill
- Run synthetic queries through the full pipeline and capture complete traces
Step 2: Read Traces and Take Notes
Present each trace to the user. For each one, ask: did the system produce a good result? Pass or Fail.
For failures, note what went wrong. Focus on the first thing that went wrong in the trace — errors cascade, so downstream symptoms disappear when the root cause is fixed. Don't chase every issue in a single trace.
Write observations, not explanations. "SQL missed the budget constraint" not "The model probably didn't understand the budget."
Template:
| Trace ID | Trace | What went wrong | Pass/Fail |
|----------|-------|-----------------|-----------|
| 001 | [full trace] | Missing filter: pet-friendly requirement ignored in SQL | Fail |
| 002 | [full trace] | Proposed unavailable times despite calendar conflicts | Fail |
| 003 | [full trace] | Used casual tone for luxury client; wrong property type | Fail |
| 004 | [full trace] | - | Pass |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 165 lines · 53 tokens per session scan A 2689164e29e3
error-analysis is a skill published in the GitHub repository build-with-dhiraj/ai-workflow-framework-portability-kit (5 stars, last pushed 21d ago), licensed MIT. It adds 53 tokens to every session and 1,745 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. It is 100% identical to error-analysis, differing in 0 lines, and is treated as a copy.
Other skills, from other repositories
huggingface-llm-trainer
Train or fine-tune language models with TRL or Unsloth on Hugging Face Jobs, including SFT, DPO, GRPO, reward models, and GGUF conversion. Use for cloud LLM training; use huggingface-vision-trainer for vision tasks.
huggingface-vision-trainer
Train object-detection, image-classification, or SAM segmentation models on Hugging Face Jobs. Use for vision fine-tuning and evaluation; use huggingface-llm-trainer for language models.
huggingface-best
Find and compare recommended Hugging Face models for a task using benchmarks, model size, and device constraints. Use for model selection questions; use huggingface-local-models for GGUF setup and hf-cli for Hub operations.
mcp-builder
Guide the creation of high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when the user wants to build an MCP server to integrate an external API or service, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).
huggingface-community-evals
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .evalresults publication, or community-evals automation.
constitutional-ai
Anthropic's method for training harmless AI through self-improvement. Two-phase approach - supervised learning with self-critique/revision, then RLAIF (RL from AI Feedback). Use for safety alignment, reducing harmful outputs without human labels. Powers Claude's safety system.