Tracely-ai: Skill for Claude Code

PostHog AI Observability for LangChain (Python)

Wire up PostHog's AI Observability so calls made through LangChain (Python) land in LLM Analytics as a full session → trace → span → generation tree — not just isolated $ai_generation events.

Prerequisite — vendor LLM SDK

This skill instruments the LLM calls the project already makes. It does not install the vendor SDK for you.

Check the project's manifest for an LLM package. The catalog is far wider than the obvious providers — 68 variants covering agent frameworks (openai-agents, claude-agent-sdk, LangGraph, CrewAI, Mastra, …) and OpenAI-compatible gateways (Groq, OpenRouter, Together, Ollama, …), which an app reaches through the openai package plus a baseURL override. 1-begin.md carries the ordered decision rules; follow them rather than matching on the first familiar package name. If no LLM SDK is present, switch to the manual-capture variant — it posts $ai_generation events directly and works standalone.

Everything else this skill needs — PostHog credentials, instrumentation packages, env vars — the skill installs and configures itself. It does not require a pre-existing posthog.init(...). If one is already there, reuse its env-var names in 3-instrument.md; if not, that step sets fresh values via set_env_values.

Steps

Read every referenced file before editing. Then work through them in order:

1. Begin — see references/1-begin.md. Pick the variant with the ordered rules (framework before provider, gateway base URL before the SDK it borrows), then read four facts from the code: the conversation, the user, the turn, and whether the app registers tools.
2. Install — see references/2-install.md. Declare the variant's packages in the manifest — and only those. For providers and gateways that's the PostHog SDK alongside the vendor SDK, with no OpenTelemetry packages.
3. Instrument — see references/3-instrument.md. Swap the vendor client for PostHog's wrapper, attach $ai_session_id, a per-turn posthog_trace_id, and the distinct id to every call, and capture tool runs as $ai_span events. This step is what turns isolated generations into a session tree.
4. Verify — see references/4-verify.md. Describe a request the user can trigger, and grade what lands in PostHog — one session, grouped traces, right attribution — rather than what the diff contains.