compare-it

Tool comparison evaluation

Produce publication-grade architectural comparison documents that evaluate two solutions against weighted criteria derived from organizational requirements. The output follows a 12-section structure with quantified scoring, bidirectional gap analysis, Mermaid architecture diagrams, cost projections, risk matrices, and reversal conditions for every recommendation.

Workflow

Phase 1: Research both solutions

Gather enough context to write authoritative profiles. Research is front-loaded. Weak research produces weak evaluations.

Research checklist:

• Identify Solution A and Solution B
• For each solution, collect: type, maturity, vendor/maintainer, mechanism, output format, integrations, generation time
• Find official documentation, product pages, blog posts, press coverage
• Identify pricing model and tiers
• Find academic research relevant to the comparison domain
• Identify the organization's current usage and existing investments
• Document limitations of the research (no hands-on testing, beta status, etc.)

Source priority:

1. Official documentation and technical specs
2. Academic research papers (arXiv, conference proceedings)
3. Press coverage and analyst reports
4. Community discussions and case studies

Phase 2: Define evaluation framework

Design criteria and weights before scoring. The framework must reflect the organization's priorities, not generic equal-weight distribution.

Framework checklist:

• Identify 6-10 evaluation criteria from organizational requirements
• Assign percentage weights that sum to exactly 100%
• Write a one-sentence rationale for each weight
• Define the 1-5 scoring scale with semantic anchors
• Verify the highest-weighted criteria reflect the organization's primary constraints

See evaluation methodology for scoring frameworks, weight calibration, and sensitivity analysis techniques.

Phase 3: Write the document