metrics-observability

Metrics, Tracing & Observability

Tool availability: every metric and trace tool named here is advertised on all toolset profiles, readonly included. Reads work on every profile via the generic list_resources / get_resource tools; delete_resource is on every profile except readonly.

Control Plane stores every workload's metrics as Prometheus-compatible time series in a managed backend (Mimir), queryable in PromQL through the per-org managed Grafana or the MCP tools. The org is the tenant — it comes from the endpoint path, so there is no org= label and no cross-org queries. Two traps dominate. Series names are short: memory is mem_used / mem_reserved / mem_billable, not memory_* — a memory_used query returns nothing, so ground names with list_metrics first. Rate-shaped metrics are pre-rated: egress, requests_per_second, and the latency buckets are already rated by the platform's recording rules, so you query them bare — wrapping them in rate() again returns garbage. Finally, a workload's in-pod CPLN_TOKEN cannot authenticate to the metrics endpoint; querying from outside the mesh needs a user or service-account token.

Two ways to query

• MCP (primary for agents): mcp__cpln__query_metrics runs a PromQL query — a range query over the last 1h at 60s step by default; pass resolution: "instant" for a single point, or since / from / to / step to adjust. mcp__cpln__list_metrics discovers the metric names and real label values present in the org right now (built-in, kube_/node_, and custom); pass metric: to ground one metric's live labels before filtering. Reach for it whenever a query returns no series. Measure first, then change scaling settings.
• Grafana: the managed per-org instance — open Metrics in the Console sidebar (or the Metrics link on any workload), use Explore for ad-hoc PromQL, and dashboards/alerting for the rest. The grafanaAdmin org permission grants the Grafana Admin role; everyone else is Viewer.