llm-artifacts-detection

LLM Artifacts Detection

Detect and flag common patterns introduced by LLM coding agents that reduce code quality.

Detection Categories

Category	Reference	Key Issues
Tests	references/tests-criteria.md	DRY violations, library testing, mock boundaries
Dead Code	references/dead-code-criteria.md	Unused code, TODO/FIXME, backwards compat cruft
Abstraction	references/abstraction-criteria.md	Over-abstraction, copy-paste drift, over-configuration
Style	references/style-criteria.md	Obvious comments, defensive overkill, unnecessary types

Agent Prompts

Use these prompts to spawn focused detection agents:

Tests Agent

Analyze the test files for LLM-introduced test quality issues:

1. **DRY Violations**: Look for setup/teardown code repeated across multiple test functions instead of using fixtures or shared helpers. Flag patterns like:
   - Identical object creation in multiple tests
   - Repeated mock configurations
   - Copy-pasted database setup

2. **Library Testing**: Identify tests that validate standard library or framework behavior rather than application code. Signs:
   - No imports from the application codebase
   - Testing built-in functions or third-party library methods
   - Assertions about stdlib behavior

3. **Mock Boundaries**: Flag mocking that's too deep or too shallow:
   - Too deep: Mocking internal implementation details, private methods
   - Too shallow: Mocking at the wrong layer, missing integration points
   - Wrong level: Unit test mocks in integration tests or vice versa

For each issue found, report: [FILE:LINE] ISSUE_TITLE

Dead Code Agent

Scan the codebase for dead code and cleanup opportunities:

1. **Unused Code**: Find functions, classes, and variables with no references:
   - Functions never called
   - Classes never instantiated
   - Module-level variables never read
   - Unreachable code after returns

2. **TODO/FIXME Comments**: Flag all TODO, FIXME, HACK, XXX comments that indicate incomplete work

3. **Backwards Compat Cruft**: Look for patterns suggesting removed features:
   - Variables renamed with _unused, _old, _deprecated suffixes
   - Re-exports only for backwards compatibility
   - Comments like "# removed", "# legacy", "# deprecated"
   - Empty functions/classes kept "for compatibility"

4. **Orphaned Tests**: Tests for code that no longer exists:
   - Test files with no corresponding source
   - Test functions testing deleted features

For each issue found, report: [FILE:LINE] ISSUE_TITLE