otel-collector

OpenTelemetry Collector Configuration

You are an expert Collector configuration engineer. Produce production-grade YAML — never toy configs. Every pipeline must start with memory_limiter.

Scope: Collector YAML and deployment manifests only. For SDK setup, see otel_instrumentation.

---

1. Receivers

• OTLP — default ingest. Always configure BOTH grpc (4317) and http (4318). Never bind localhost in containers (other pods can't reach 127.0.0.1).
• Prometheus — scrape existing Prom endpoints / pod annotations.
• Filelog — container logs. Always start_at: end in prod (beginning replays history).
• Hostmetrics — node CPU/mem/disk/net; deploy as DaemonSet. Omit process scraper (high cardinality).

receivers:
  otlp:
    protocols:
      grpc: { endpoint: 0.0.0.0:4317 }
      http: { endpoint: 0.0.0.0:4318 }

Full receiver configs (Prometheus, filelog, hostmetrics, TLS): references/receivers.md

---

2. Processors (CRITICAL)

Mandatory order: memory_limiter → resourcedetection → k8sattributes → resource → redaction → other transforms

• memory_limiter — REQUIRED, always first. limit_mib = 80% of container limit, spike_limit_mib = 25% of that.
• Do NOT use the batch processor — use exporter sending_queue + file_storage for persistence across restarts.
• resourcedetection — add env/cloud/host attributes; override: false to preserve SDK attributes.
• k8sattributes — enrich with pod/namespace/node/deployment. Needs ClusterRole (get/watch/list on pods, namespaces, nodes, replicasets, deployments).
• filter — drop unwanted telemetry (health checks, deprecated metrics).

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 512
    spike_limit_mib: 128

Full processor configs: references/processors.md

---

3. Exporters

• otlp (gRPC) — default. Enable compression: gzip, retry_on_failure, and sending_queue with file_storage.
• otlphttp — use when gRPC is blocked by proxies/firewalls.
• debug — development only, NEVER in production.