ai-monitoring-setup

ai-monitoring-setup is a command for Claude Code from Matt-Dionis/claude-code-configs. It costs 10 tokens per session (6,238 once invoked), scanned B, original, MIT.

A monitoring and analytics setup guide for applications built with the AI SDK. It covers measurements such as response time, token use, costs, errors, user activity, and feature adoption.

In plain words
What is it for?
Use it to add latency and throughput tracking, token and cost monitoring, error classification, usage analytics, dashboards, and alerts.
Why use it?
It helps you see whether AI features are slow, expensive, failing, or being used. This makes it easier to identify failure patterns and follow performance and usage trends.

Command for Claude Code

Written for Claude Code: allowed-tools in frontmatter.

Good fit Use it to add latency and throughput tracking, token and cost monitoring, error classification, usage analytics, dashboards, and alerts.

Compare 6 commands from other repositories ↓
Install with agentmods
npx agentmods add commands/matt-dionis/claude-code-configs/ai-monitoring-setup
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Clone the repo
git clone --depth 1 https://github.com/Matt-Dionis/claude-code-configs

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for ai-monitoring-setup

README.md
[![agentmods](https://agentmods.dev/badge/commands/matt-dionis/claude-code-configs/ai-monitoring-setup/github.svg)](https://agentmods.dev/commands/matt-dionis/claude-code-configs/ai-monitoring-setup)
Your own site
<a href="https://agentmods.dev/commands/matt-dionis/claude-code-configs/ai-monitoring-setup"><img src="https://agentmods.dev/badge/commands/matt-dionis/claude-code-configs/ai-monitoring-setup/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for ai-monitoring-setup

Your own site · 80×15
<a href="https://agentmods.dev/commands/matt-dionis/claude-code-configs/ai-monitoring-setup"><img src="https://agentmods.dev/badge/commands/matt-dionis/claude-code-configs/ai-monitoring-setup.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 10 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 6,238 The whole file, excluding the scripts and references it only reads on demand.
Security scan B 2 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00010 $0.06238
Opus 5.5 $0.00004 $0.02495
Sonnet 5.5 $0.00002 $0.01248
Haiku 4.5 $0.00001 $0.00624

Measured 1mo ago against content hash 3370cd11eea6, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade B, and why

ai-monitoring-setup scanned grade B with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Sends data to an external URLmediumData exfiltration

A POST to an outside endpoint may be telemetry or may be exfiltration; either way the mod talks to somewhere, and you should know where.

const response = await fetch('https://api.datadoghq.com/api/v1/series', { method: 'POST',

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

await fetch(process.env.CUSTOM_ANALYTICS_ENDPOINT!, {
configurations/tooling/vercel-ai-sdk/.claude/commands/ai-monitoring-setup.md · 807 lines

How it starts

The opening of the file, as written. The whole thing — 807 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Set up AI SDK Monitoring and Analytics

Configure comprehensive monitoring and analytics for AI SDK applications: $ARGUMENTS

Current Monitoring Analysis

Existing monitoring setup: !grep -r "monitoring\|analytics\|metrics" . --include="*.ts" --include="*.tsx" | head -5

Performance tracking: !grep -r "performance\|latency\|duration" . --include="*.ts" | head -5

Error handling: !grep -r "error\|catch\|throw" . --include="*.ts" | head -5

Monitoring Categories

Performance: Latency, throughput, response times, edge performance Usage: Token consumption, request patterns, user behavior Costs: Provider costs, usage optimization, budget tracking Errors: Error rates, failure patterns, recovery metrics Analytics: User insights, feature adoption, performance trends Dashboard: Real-time monitoring, alerts, visualization

Your Task

  1. Analyze monitoring requirements for comprehensive AI SDK observability
  2. Implement performance tracking with latency and throughput metrics
  3. Set up usage analytics for token consumption and cost tracking
  4. Configure error monitoring with detailed error classification
  5. Build real-time dashboard for monitoring AI SDK applications
  6. Add alerting system for performance and error thresholds
  7. Create analytics reports for insights and optimization
  8. Integrate with external services (DataDog, New Relic, etc.)

Implementation Requirements

Performance Monitoring
  • Request/response latency tracking
  • Streaming performance metrics
  • Provider response time monitoring
  • Edge runtime performance tracking
  • Memory and CPU usage monitoring
Usage Analytics
  • Token consumption tracking by provider and model
  • Request volume and patterns
  • User behavior and feature adoption
  • Geographic usage distribution
  • Time-based usage patterns
Cost Management
  • Real-time cost calculation across providers
  • Budget tracking and alerting
  • Cost optimization recommendations
  • Provider cost comparison
  • Usage forecasting and planning

Read the full file on GitHub · 807 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 1mo ago First seen · 807 lines · 10 tokens per session scan B 3370cd11eea6

Subscribe to this mod's changes

ai-monitoring-setup is a command published in the GitHub repository Matt-Dionis/claude-code-configs (619 stars, last pushed 1y ago), licensed MIT. It adds 10 tokens to every session and 6,238 once invoked, about $0.0000 per session on Opus 5.5. A static security scan graded it B with 2 findings (sends data to an external url, makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.