codex-judge

Codex Judge: Cross-Provider Verdict

Uses Codex (GPT-5) as an independent judge to evaluate Claude-written implementations, eliminating self-preference bias inherent in same-model review.

Why Cross-Provider Judging

Self-preference bias is well-documented: models rate their own outputs 10-25% higher than warranted. GPT-5.x judging Claude's output is an ideal cross-provider setup — different training data, different reward models, different blind spots. The judge shares no internal assumptions with the implementer.

Reference: "Judging the Judges" (arXiv 2604.23178) — cross-model ensembles achieve higher separability and alignment with human rankings.

What the Judge Sees

The judge receives ONLY:

Input	Source
Task spec + acceptance criteria	Plan file
Git diff	git diff of the implementation
Reviewer findings (structured JSON)	Insistir reviewer agent
Check results (tests, typecheck, lint)	Reviewer's checks object

NEVER send the implementer's reasoning, chain-of-thought, or self-assessment. The judge evaluates artifacts only — fresh context prevents sycophancy contamination.

How to Invoke

mcp__codex__codex:
  prompt: <rendered judge-prompt from references/judge-prompt.md>
  sandbox: "read-only"
  approval-policy: "never"
  cwd: "<repo root>"

Parse the response as JSON matching the verdict schema. If JSON parsing fails, re-prompt once with: "Your previous response was not valid JSON. Return ONLY the verdict JSON, no markdown fencing." If it fails again, treat as degraded (see below).

Verdict Schema

{
  "verdict": "PASS | NEEDS_REVISION | REJECT",
  "confidence": 0.0-1.0,
  "criteria": {
    "correctness":      { "score": 1-5, "evidence": "...", "violations": [] },
    "security":         { "score": 1-5, "evidence": "...", "violations": [] },
    "maintainability":  { "score": 1-5, "evidence": "...", "violations": [] },
    "spec_compliance":  { "score": 1-5, "evidence": "...", "violations": [] }
  },
  "overall_score": 1-5,
  "rationale": "2-3 sentence summary",
  "remediation": [
    {
      "severity": "critical | major | minor",
      "description": "what is wrong",
      "suggestion": "how to fix",
      "file": "path",
      "line": 42
    }
  ]
}