gitscape.ai: Skill for Claude Code

Observability and Instrumentation

Overview

Instrument production code so that on-call engineers (including future AI agents) can answer: Is it working? What's broken? Why? Logs answer why. Metrics answer that something is wrong. Traces answer where. Build all three in as you go — retrofitting observability after an incident is painful and late.

When to Use

• Adding a new API endpoint or background job
• Shipping a feature that touches external services (GitHub API, Gemini)
• Modifying error handling or retry logic
• Any change to Cloud Run that affects request processing

The Three Pillars

1. Structured Logging

Every log line should be machine-parseable JSON, not a free-form string.

import structlog

log = structlog.get_logger()

# GOOD: Structured, queryable
log.info(
    "skill_generation_started",
    repo=repo,
    tier=tier,
    request_id=request_id,
)

log.error(
    "github_api_error",
    repo=repo,
    status_code=e.status,
    error=str(e),
    request_id=request_id,
)

# BAD: Free-form string — unsearchable
print(f"Error fetching {repo}: {e}")
logger.info("Starting skill generation for " + repo)

Log level conventions:

• error — invariant broken, someone may need to act
• warn — degraded but handled (rate limit approaching, retrying)
• info — significant business event (skill generated, export downloaded)
• debug — off in production; verbose tracing for local debugging

Never log:

• Secrets, API keys, or tokens
• Full request/response bodies
• Unredacted PII or GitHub personal access tokens

2. RED Metrics for Every Endpoint

For every API endpoint, instrument:

• Rate — requests per second
• Errors — error rate (%)
• Duration — p50/p95/p99 latency (histogram, never average)

from prometheus_client import Counter, Histogram

skill_generation_requests = Counter(
    "skill_generation_requests_total",
    "Total skill generation requests",
    ["tier", "status"]  # labels: tier=standard|hd, status=success|error
)

skill_generation_duration = Histogram(
    "skill_generation_duration_seconds",
    "Skill generation duration",
    ["tier"],
    buckets=[0.5, 1.0, 2.0, 5.0, 10.0, 30.0, 60.0]
)

# Usage
with skill_generation_duration.labels(tier=tier).time():
    try:
        result = await generate_skill(repo, tier)
        skill_generation_requests.labels(tier=tier, status="success").inc()
    except Exception as e:
        skill_generation_requests.labels(tier=tier, status="error").inc()
        raise