llm evaluation mcp servers

ukanwat/aaabench

MCP server Claude CodeCodexCursor +2

MCP server "unreal-mcp", hosted remotely at 127.0.0.1, as configured in ukanwat/aaabench.

vassiliylakhonin/agenda-intelligence-md

MCP server Claude CodeCodexCursor +2

Deterministic evidence-packet linter for claim-backed AI output. Runs locally from the agenda-intelligence-md Python package.

vassiliylakhonin/agenda-intelligence-md

MCP server Claude CodeCodexCursor +2

Deterministic evidence-packet linter for claim-backed AI output. Runs locally from the agenda-intelligence-md Python package.

homemade-software-inc/completion-kit

MCP server Claude CodeCodexCursor +2

Prompt evals over MCP: run a prompt on your dataset, score each output 1-5 with an LLM judge. Remote server at completionkit.com.

humanjudge/grandjury

MCP server Claude CodeCodexCursor

GrandJury SDK — submit LLM traces for human evaluation + analytics client. Runs locally from the grandjury Python package.

Vivek0712/assay

MCP server Claude CodeCodexCursor

MCP server "shouldiread" as configured in Vivek0712/assay. Runs locally from the shouldiread Python package.

decimal-labs/decimalai-mcp

MCP server Claude CodeCodexCursor

MCP server for the DecimalAI skills registry — search skills, inspect trust/benchmark evidence, and read the leaderboard from any MCP client. Runs locally from the decimalai-mcp Python package. Needs 1 environment variable to run.

Guruprasath-Annadurai/Whitepact

MCP server Claude CodeCodexCursor

ResponsibleAI — Enterprise AI Governance Platform: trust scoring, bias detection, hallucination detection, guardrails, compliance (NIST AI RMF / EU AI Act / ISO 42001), cost intelligence, drift monitoring. Runs locally from the rai-governance-platform Python package. Needs 1 environment variable to run.

nickklos10/QueryPilot

MCP server Claude CodeCodexCursor

Eval-driven SQL reliability for AI agents. Runs locally from the querypilot Python package. Needs 6 environment variables to run.

RudrenduPaul/agent-eval

MCP server Claude CodeCodexCursor

Statistical regression testing for LLM agents. Get a p-value on whether behavior actually shifted between versions, not just whether one run looked different. Apache 2.0, self-hostable, no SaaS dependency -- a Promptfoo alternative for the statistical testing gap threshold-based eval tools don't cover. Runs locally…

mrhpython/lens-mcp

MCP server Claude CodeCodexCursor

Outside-in validation gate for AI-generated text; a separate model checks LLM output, fail-closed. Runs locally from the @soulfield/lens-mcp npm package. Needs 2 environment variables to run.

multivon-ai/multivon-mcp

MCP server Claude CodeCodexCursor

MCP server exposing multivon-eval + pdfhell as agent-callable tools. Drop into Claude Desktop, Cursor, Cline, or any MCP-compatible AI coding agent. Runs locally from the multivon-mcp Python package.

YihengLi-1/forward-deployed-agent-lab

MCP server Claude CodeCodexCursor

MCP server "forward-deployed-agent-lab" as configured in YihengLi-1/forward-deployed-agent-lab. Runs locally from the forward-deployed-agent-lab Python package.

mattagame/whichtool

MCP server Claude CodeCodexCursor

MCP server "whichtool" as configured in mattagame/whichtool. Runs locally from the whichtool npm package.

samalbanese/mcp-tournament

MCP server Claude CodeCodexCursor +2

MCP server "mcp-tournament" as configured in samalbanese/mcp-tournament. Runs locally from the mcp-tournament npm package. Needs 1 environment variable to run.

marblerace/zoraHoldersAI

MCP server Claude CodeCodexCursor +2

MCP server "onchain-text2sql-agent" as configured in marblerace/zoraHoldersAI. Runs locally from the onchain-text2sql-agent Python package.

adwitiyashukla/LLMFeeder

MCP server Claude CodeCodexCursor +2

MCP server "llmfeeder" as configured in adwitiyashukla/LLMFeeder. Runs locally from the llmfeeder Python package.

APantov/llm-routing-comparison

MCP server Claude CodeCodexCursor +2

MCP server "llm-routing" as configured in APantov/llm-routing-comparison. Runs router_agent.mcp_server with python. Needs 4 environment variables to run.

TheDavidKoen/deadwax

MCP server Claude CodeCodexCursor +2

A media library that lives in markdown. Notes etched next to the music. Runs locally from the deadwax Python package.

brayans7/oficio

MCP server Claude CodeCodexCursor +2

A calm interface between an Obsidian daily note and Hermes Agent. Runs locally from the oficio Python package.

ad-github1/ENTERPRISE-AI-AGENT-EVALUATION-PLATFORM

MCP server Claude CodeCodexCursor +2

MCP server "enterprise-ai-agent-eval-platform" as configured in ad-github1/ENTERPRISE-AI-AGENT-EVALUATION-PLATFORM. Runs locally from the enterprise-ai-agent-eval-platform Python package.