evaluating-new-technology

Evaluating New Technology

Scope

Covers

• Evaluating a new tool/platform/vendor (including AI products) for adoption
• Emerging tech “should we use this?” decisions
• Build vs buy decisions and tech stack changes
• Running a proof-of-value pilot and capturing evidence
• First-pass risk review (security/privacy/compliance, vendor claims, operational readiness)

When to use

• “Evaluate this new AI tool/vendor for our team.”
• “Should we build this in-house or buy a vendor?”
• “We’re considering changing our analytics/experimentation stack—make a recommendation.”
• “Create a technology evaluation doc with a pilot plan, risks, and decision memo.”

When NOT to use

• You don’t have a real problem/job to solve yet (use problem-definition first).
• You need a full product strategy/roadmap (use ai-product-strategy).
• You’re designing how to build an LLM system (use building-with-llms).
• You need a formal security assessment / penetration testing (engage security; this skill produces a structured first pass).
• You are weighing trade-offs within an existing design or architecture decision (use evaluating-trade-offs).
• You need to manage or pay down existing technical debt rather than adopt something new (use managing-tech-debt).
• You already chose the technology and need an implementation/migration plan (use managing-tech-debt or platform-infrastructure).

Inputs

Minimum required

• Candidate technology (what it is, vendor/build option, links if available)
• Problem/workflow to improve + who it’s for
• Current approach/stack and what’s not working
• Constraints: data sensitivity, privacy/compliance, budget, timeline, regions, deployment model (SaaS/on-prem)
• Decision context: who decides, adoption scope, risk tolerance

Missing-info strategy

• Ask up to 5 questions from references/INTAKE.md (3–5 at a time).
• If still missing, proceed with explicit assumptions and present 2–3 options (e.g., buy vs build vs defer).
• Do not request secrets. If asked to run tools, change production systems, or sign up for vendors, require explicit confirmation.