mcp-audit

mcp-audit

A repeatable, read-only review of an MCP server's tool surface. The output is a single Markdown report the maintainer can act on in one sitting: what each tool is, how well an agent can tell when to call it, where the surface disagrees with itself, what is untested or unused, and which few fixes to ship first.

The review scores tools on what can be checked in the source and the repository, not on taste. Every finding cites a finding_id from SEVERITY.md, so two audits of the same server can be compared row by row.

Axes

Per tool:

1. Discoverability (1-5). Could an agent decide to call this tool from its description alone? 5 = trigger phrases, when-to-use guidance and the return shape; 1 = the description restates the function name.
2. Return shape. Does the tool use the surface's common result shape, or an idiom of its own? Count the idiom families across the whole surface.
3. Naming. One case convention, a shared domain prefix where the surface has families (memory_*, issue_*), one verb per operation.
4. Error handling. How each tool reports a failure: it raises, returns a structured error, or says nothing.
5. Test coverage. A test that goes through the MCP layer, a test of the underlying function only, a smoke call with no assertions, or nothing.
6. Decay. Mentioned in the CHANGELOG, in the README or docs, imported by any consumer, exercised by any test. A tool with none of the four is a decay candidate.

Per surface, when it applies:

1. State handling - servers that keep state across calls (files, a database, an in-memory store): atomic writes, a serialiser around load-mutate-save, persistence across a restart, state one tool can see that another wrote.
2. Object-level authorization - servers that serve more than one tenant, user or project: does every tool that resolves an object by id check that the caller may touch that object? See Authorization axis.

When to use it