pseo-quality-guard

pSEO Quality Guard

Validate generated pages against SEO quality standards. Detect and flag issues that would cause Google to devalue, deindex, or penalize programmatic pages.

Core Principles

1. No thin pages: Every page must provide substantial, unique value
2. No duplicate content: No two pages should have the same or near-identical content
3. No cannibalization: No two pages should target the same keyword or intent
4. Metadata uniqueness: Every page has unique title and description
5. Fail loudly: Quality issues should block deployment, not slip through silently

Quality Checks

1. Thin Content Detection

A page is thin if it:

• Has fewer than 300 words of unique text content (excluding navigation, footer, boilerplate)
• Is essentially a template with only 1-2 variable substitutions
• Contains no meaningful content beyond its metadata
• Has the same paragraph structure as other pages with only proper nouns changed

How to check:

• Extract text content from each rendered page (strip HTML tags, nav, footer)
• Count unique words per page
• Compare content across pages - compute similarity ratios
• Flag pages with < 300 unique words or > 80% similarity to another page

2. Duplicate Content Detection

Check for:

• Exact duplicates: Two pages with identical body content
• Near duplicates: Pages with > 80% text similarity (use Jaccard similarity on n-grams or cosine similarity)
• Title duplicates: Two pages with the same <title> tag
• Description duplicates: Two pages with the same meta description
• URL-based duplicates: Different URLs serving the same content (www vs non-www, trailing slash variants)

How to detect:

At small scale (< 200 pages), pairwise comparison is feasible:

For each pair of pages:
  similarity = intersection(ngrams(page_a), ngrams(page_b)) / union(ngrams(page_a), ngrams(page_b))
  if similarity > 0.8: FLAG as near-duplicate

At scale (200+ pages), pairwise is O(n²) and impractical. Use one of:

• MinHash / LSH: Hash n-gram sets into fixed-size signatures, use locality-sensitive hashing to find candidate pairs. Reduces comparisons from O(n²) to near-linear.
• SimHash: Compute a fingerprint per page, compare fingerprints (Hamming distance). Pages with similar fingerprints are candidates.
• Sampling: Compare each page against a random sample of 50 others + all pages in the same category. Not exhaustive but catches the common cases.
• Template fingerprinting: Hash the non-variable parts of each page. If two pages share the same template fingerprint, flag them - they differ only in variable slots.