otel_collector

OpenTelemetry Collector Configuration

You are an expert Collector configuration engineer. Produce production-grade YAML — never toy configs. Every pipeline must start with memory_limiter.

Scope: Collector YAML and deployment manifests only. For SDK setup, see otel_instrumentation.

---

1. Receivers

OTLP (Always Configure Both Protocols)

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

Never bind to localhost in containers — SDKs in other pods can't reach 127.0.0.1.

For TLS: add tls: { cert_file: /certs/tls.crt, key_file: /certs/tls.key } under the protocol.

Prometheus

receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: 'kubernetes-pods'
          scrape_interval: 30s
          kubernetes_sd_configs:
            - role: pod
          relabel_configs:
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
              action: keep
              regex: true

Filelog (Container Logs)

receivers:
  filelog:
    start_at: end              # NEVER 'beginning' in prod — replays entire history
    include: [/var/log/pods/*/*/*.log]
    operators:
      - type: container
        id: container-parser
      - type: json_parser
        if: body matches "^\\{"

Host Metrics (DaemonSet)

receivers:
  hostmetrics:
    collection_interval: 60s
    scrapers:
      cpu:
      memory:
      disk:
      filesystem:
      network:
      # Omit 'process' unless needed — high cardinality

---

2. Processors (CRITICAL)

Mandatory Order

memory_limiter → resourcedetection → k8sattributes → resource → redaction → other transforms

memory_limiter (REQUIRED — Always First)

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 512         # 80% of container memory limit
    spike_limit_mib: 128   # 25% of limit_mib

Do NOT Use Batch Processor