opentelemetry

OpenTelemetry Implementation Guide

Overview

OpenTelemetry (OTel) is a vendor-neutral observability framework for instrumenting, generating, collecting, and exporting telemetry data (traces, metrics, logs). This skill provides guidance for implementing OTEL in Kubernetes environments.

Quick Start

Deploy OTEL Collector on Kubernetes

# Add Helm repo
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update

# Install with basic config
helm install otel-collector open-telemetry/opentelemetry-collector \
  --namespace monitoring --create-namespace \
  --set mode=daemonset

Send Test Data via OTLP

# gRPC endpoint: 4317, HTTP endpoint: 4318
curl -X POST http://otel-collector:4318/v1/traces \
  -H "Content-Type: application/json" \
  -d '{"resourceSpans":[]}'

Core Concepts

Signals: Three types of telemetry data:

• Traces: Distributed request flows across services
• Metrics: Numerical measurements (counters, gauges, histograms)
• Logs: Event records with structured/unstructured data

Collector Components:

• Receivers: Accept data (OTLP, Prometheus, Jaeger, Zipkin)
• Processors: Transform data (batch, memory_limiter, k8sattributes)
• Exporters: Send data (prometheusremotewrite, loki, otlp)
• Extensions: Add capabilities (health_check, pprof, zpages)

Collector Configuration

Basic Pipeline Structure

config:
  receivers:
    otlp:
      protocols:
        grpc:
          endpoint: ${env:MY_POD_IP}:4317
        http:
          endpoint: ${env:MY_POD_IP}:4318

  processors:
    batch:
      timeout: 10s
      send_batch_size: 1024
    memory_limiter:
      check_interval: 5s
      limit_percentage: 80
      spike_limit_percentage: 25

  exporters:
    prometheusremotewrite:
      endpoint: "http://prometheus:9090/api/v1/write"
    loki:
      endpoint: "http://loki:3100/loki/api/v1/push"

  service:
    pipelines:
      metrics:
        receivers: [otlp]
        processors: [memory_limiter, batch]
        exporters: [prometheusremotewrite]
      logs:
        receivers: [otlp]
        processors: [memory_limiter, batch]
        exporters: [loki]
      traces:
        receivers: [otlp]
        processors: [memory_limiter, batch]
        exporters: [otlp/tempo]