gke-cost-analysis

GKE Cost Analysis

This skill provides guidance on answering natural language questions about GKE-related costs, billing reports, and utilization analysis.

Overview

When users ask about GKE costs (e.g., "What are my costs across projects?", "What's my most expensive namespace?", "Why is my cluster cost spiking?"), use this skill to provide a structured and expert response using BigQuery billing exports, cost allocation metadata, and live cluster metrics.

Instructions

When handling a cost-related question:

1. Provide a Direct Answer: Address the specific cost question or analytical request clearly and concisely.
2. Explain BigQuery Integration: Explain how to query BigQuery for historical cost breakdown. Note that GKE costs originate from the GCP Billing Detailed BigQuery Export (gcp_billing_export_resource_v1_*).
3. Check & Verify Cost Allocation: Explain that GKE Cost Allocation must be enabled on the cluster (--enable-cost-allocation) for namespace, label, and workload-level billing granularity. If queries return empty labels, provide the gcloud command to enable it.
4. Analyze Pricing Drivers & Utilization: When diagnosing cost drivers, explain whether the cluster is in Autopilot (billed by requested pod CPU/memory) or Standard mode (billed by underlying VM node size + control plane fees), and compare live utilization (kubectl top) against provisioned requests.
5. Provide Actionable Commands/Queries: Provide concrete BigQuery CLI (bq query) commands or read-only gcloud/kubectl inspection commands. Prefer bq over BigQuery Studio when available.

Key Points & Pricing Drivers

• Data Source: GKE costs come from GCP Billing Detailed BigQuery Export. The user must provide the full path to their BigQuery table (dataset name and table name containing the Billing Account ID).
• Granularity Requirement: GKE Cost Allocation (--enable-cost-allocation) must be enabled on the cluster to populate goog-k8s-cluster-name, k8s-namespace, k8s-workload-name, and k8s-workload-type labels in BigQuery.
• Autopilot vs. Standard Cost Drivers:
  • Autopilot Pricing: Billed directly on pod resource requests (requests.cpu, requests.memory, ephemeral storage). Over-requested pods drive up billing regardless of whether the pod actively uses those CPU cycles or memory.
  • Standard Pricing: Billed on provisioned node pool VMs (e2, n4, c3, etc.) plus a cluster management fee ($0.10/hour). Idle nodes or multiple low-utilization dev clusters drive excess infrastructure costs.
• Credits & Discounts Impact: When analyzing cost versus cost_before_credits, note that Committed Use Discounts (CUDs) and Spot VMs appear as credits or reduced rate charges in the billing export.
• Tools & Syntax: BigQuery CLI (bq) is preferred. When writing Standard SQL queries, use a dot (.) instead of a colon (:) to separate the project ID and dataset name ({project_id}.{dataset_name}.{table_name}).
• Defaults: Assume last 30 days, row limit 10, ordering by cost descending (ORDER BY cost DESC), unless specified otherwise.