rag-review

/rag-review — E2E RAG Pipeline Audit

Skill: Load rag-review skill — it is the single source of truth for all 8-stage checks, severity rubric, verdict format, and reference cross-links. Do not restate checks inline here.

Audit a RAG pipeline against the production checklist covering all 8 stages: ingestion/chunking, versioning/updates, embedding/indexing, query routing, retrieval/fusion/reranking, generation/abstention, security, and eval/observability.

Phase 1: Discover the implementation

Walk the repo and identify:

• Where is the RAG pipeline code? (rag/, retrieval/, pipeline/, agents/, or similar)
• What vector DB is used? (check imports: pgvector, weaviate, qdrant, pinecone, chroma)
• What embedding model and reranker, if any?
• Where is the generation prompt template?
• Is there an audit log?
• Is there an eval directory with a golden set?

If the structure is unclear, ask the user once to point at the main pipeline entry point.

Phase 2: Run the checklist

Apply the 8-stage checklist from the rag-review skill. For each check, classify as:

• ✅ Present — implemented correctly with evidence
• ⚠️ Partial — exists but incomplete or misconfigured
• ❌ Missing — not present
• ❓ Unknown — couldn't determine from available code

For every ❌ or ⚠️ finding, cite file:line as evidence. No generic advice without a code reference.

Phase 3: Output a structured report

# RAG Pipeline Audit

**Repo:** {{repo-name}}
**Date:** {{today}}
**Stack detected:** {{vector DB | embedding model | reranker | LLM}}

## Summary

- ✅ Present: N of total
- ⚠️ Partial: N
- ❌ Missing: N
- ❓ Unknown: N

## 🔴 Critical (BLOCK)

### C1. {{Finding title}}
**Stage:** {{stage number and name}}
**Status:** ❌ Missing / ⚠️ Partial
**Evidence:** `path/to/file.py:42` — [paste the problematic code]
**Issue:** [one sentence — why this is a production risk]
**Fix:** [concrete code change or pattern]
**Reference:** `rag-review/SKILL.md §{{stage}} check {{#}}`

## 🟠 High (NEEDS_REVIEW — fix before merge)

### H1. {{Finding title}}
[same structure as Critical]

## 🟡 Medium (NEEDS_REVIEW — recommend)

[same structure]

## 🟢 Low (APPROVE with note)

[brief list — no full structure needed for Low]

## Recommended next 3 PRs (priority order)

1. **{{PR title}}** — fixes {{finding IDs}}; effort S/M/L
2. ...
3. ...

VERDICT: [APPROVE | NEEDS_REVIEW | BLOCK] — CRITICAL: N | HIGH: N | MEDIUM: N | LOW: N