ai-product-development

AI product development

A model call that works in a demo fails in production in three places: a tool handler that trusts an earlier authorization check, usage charged after the fact under concurrency, and a model or prompt change shipped without an evaluation. Get those three right, then map every UI state to its failure and its charge. Input validation, secret handling and typed routes are assumed.

Choose a mode

• Build: add or change an AI feature. Apply every section that the feature touches.
• Audit: review an existing feature. Do not edit unless asked. Report findings ranked by severity, each with file:line, the failure it allows, and the fix.
• Model or prompt change: swap a model, edit instructions, change tools, retrieval, temperature or output limits. Run Evaluate before changing a model or prompt first. Use the ai-cost-audit skill when the motive is cost.

A request that says "review", "check" or "is this safe" is Audit. A request to "add", "make", or "wire" is Build. Ask only when the request could be either and the answer decides whether files change.

Inspect first

Read before asking:

• the AI SDK and its installed major version (package.json plus the lockfile), the providers, and every place a tool is defined or executed;
• how the server identifies actor and tenant, and how that identity reaches a tool handler;
• the credit, quota or billing path, and whether requests can run concurrently or loop through tools;
• existing eval fixtures, golden outputs, prompt files and telemetry.

Ask only about choices that change the boundary: which tool actions need human approval, the charging policy for cancelled or failed work, and whether work must outlive the request (background job versus stream). Continue with labeled assumptions when the choice is reversible.

Authorize at tool execution time

Treat the tool name and arguments the model selects as untrusted input from the internet. Retrieved documents, emails and web pages can inject them. Authorizing the chat request does not authorize the action the model picks three steps later.