ai-engineering-toolkit

AI Engineering Toolkit

Overview

A collection of 6 structured, expert-level workflows that turn your AI coding assistant into a senior AI engineering partner. Each skill encodes a repeatable methodology — not just "ask AI to help," but a step-by-step decision framework with quantitative scoring, checklists, and decision trees.

The key difference from ad-hoc AI assistance: every workflow produces consistent, reproducible results regardless of who runs it or when. You can use the scoring systems as team baselines and write them into CI/CD pipelines.

When to Use This Skill

• Use when evaluating or optimizing LLM system prompts before production deployment
• Use when designing a RAG pipeline and need structured architecture decisions (not just boilerplate code)
• Use when planning token budget allocation across context window zones
• Use when running pre-launch security audits on AI agents
• Use when building evaluation frameworks for LLM applications
• Use when thinking through product strategy before writing code

How It Works

Skill 1: Prompt Evaluator

Scores prompts across 8 dimensions (Clarity, Specificity, Completeness, Conciseness, Structure, Grounding, Safety, Robustness) on a 1-10 scale with weighted aggregation to a 0-100 score. Identifies the 3 weakest dimensions, generates targeted rewrites, and re-evaluates. Supports single prompt, A/B comparison, and batch evaluation modes.

Skill 2: Context Budget Planner

Analyzes token distribution across 5 context zones (System, Few-shot, User input, Retrieval, Output) and produces an optimized allocation plan. Includes a compression strategy decision tree for each zone. Common finding: output zone squeezed to under 6% — this skill catches that before truncation happens.

Skill 3: RAG Pipeline Architect

Walks through a complete architecture decision tree: document format → parsing strategy → chunking approach (fixed/semantic/recursive) → embedding model selection → retrieval method (vector/keyword/hybrid) → evaluation metrics (Faithfulness, Relevancy, Context Precision). Covers Naive RAG, Advanced RAG, and Modular RAG patterns.