audit-pipeline-guide

Audit Pipeline - Complete Guide

Overview

The Audit Pipeline is a comprehensive 3-phase code quality orchestrator that systematically transforms prototype code into production-ready software through:

1. Theater Detection - Identifies and eliminates mock/fake/placeholder code
2. Functionality Audit - Validates code works through sandbox testing with Codex iteration
3. Style & Quality Audit - Polishes code to professional production standards

Why This Pipeline Exists

Code goes through stages:

• Prototype: "Make it work" - lots of shortcuts, mocks, TODOs
• Functional: "Make it right" - real implementations, tests passing
• Production: "Make it excellent" - clean, documented, maintainable

The Audit Pipeline automates this transformation.

The 3-Phase System

Phase 1: Theater Detection Audit

Purpose: Find all code that looks like it works but is actually fake.

What It Finds:

• Hardcoded mock data (return {"id": 123, "name": "Test User"})
• TODO/FIXME markers indicating incomplete work
• Stub functions that exist but don't do anything
• Commented-out production code
• Simplified error handling that always succeeds
• Test mode conditionals that bypass real logic

Process:

1. Pattern-based scanning for theater indicators
2. Contextual analysis to understand intended behavior
3. Dependency mapping between theater instances
4. Risk assessment (critical vs. minor)
5. Completion or documentation

Output: Theater audit report listing all instances with locations, severity, and completion status.

Skill Used: theater-detection-audit

---

Phase 2: Functionality Audit with Codex Sandbox

Purpose: Prove code actually works through execution testing.

What It Does:

• Creates isolated sandbox environment
• Generates comprehensive test cases
• Executes code with realistic inputs
• Verifies outputs match expectations
• Uses Codex to fix failures automatically

The Codex Integration Loop:

For each failing test:
┌─────────────────────────────────────┐
│ 1. Capture test failure details    │
│    - Error message                  │
│    - File and line                  │
│    - Code context                   │
└──────────┬──────────────────────────┘
           │
           ▼
┌─────────────────────────────────────┐
│ 2. Spawn Codex in sandbox           │
│    codex --full-auto "Fix test..."  │
│    - Network disabled (secure)      │
│    - CWD only (isolated)            │
│    - Autonomous execution           │
└──────────┬──────────────────────────┘
           │
           ▼
┌─────────────────────────────────────┐
│ 3. Codex analyzes and fixes         │
│    - Reads code                     │
│    - Identifies root cause          │
│    - Implements fix                 │
│    - Runs tests                     │
└──────────┬──────────────────────────┘
           │
           ▼
┌─────────────────────────────────────┐
│ 4. Re-test in sandbox               │
│    - Run the specific test          │
│    - Check if passing               │
└──────────┬──────────────────────────┘
           │
      ┌────┴────┐
      │ Passing? │
      └────┬────┘
      YES  │  NO (iteration < 5)
           │     │
           │     └──► Repeat with more context
           │
           ▼
┌─────────────────────────────────────┐
│ 5. Validate no regressions          │
│    - Run full test suite            │
│    - Ensure other tests still pass  │
└──────────┬──────────────────────────┘
           │
           ▼
┌─────────────────────────────────────┐
│ 6. Apply fix to main codebase       │
│    - Copy changes from sandbox      │
│    - Document what was fixed        │
│    - Track in audit report          │
└─────────────────────────────────────┘