vllm-deploy-k8s

vLLM Kubernetes Deployment

A Claude skill for deploying vLLM to Kubernetes using YAML templates. Deploys a vLLM OpenAI-compatible server as a Kubernetes Deployment with a ClusterIP Service, GPU resources, and health probes.

What this skill does

• Deploy vLLM as a Kubernetes Deployment + Service with NVIDIA GPU support
• Check if a vLLM deployment already exists before deploying
• Check if the Hugging Face token secret exists, and ask the user for their token if not
• Use the vllm/vllm-openai:latest image by default (user can specify a different version)
• Provide sensible default configuration that users can customize (model, replicas, GPU count, extra vLLM flags, etc.)

Prerequisites

• kubectl configured with access to a Kubernetes cluster
• NVIDIA GPU Operator or device plugin installed on cluster nodes
• Hugging Face token (required for gated models like Llama, optional for public models)

Deployment Steps

Step 1: Check HF token secret

Before deploying, check if the hf-token Kubernetes secret exists in the target namespace:

kubectl get secret hf-token -n <namespace>

• If the secret exists: proceed to Step 2.
• If the secret does not exist: ask the user to provide their Hugging Face token, then create the secret:

kubectl create secret generic hf-token --from-literal=HF_TOKEN="<user-provided-token>" -n <namespace>

This is required for gated models (e.g., meta-llama/Meta-Llama-3.1-8B). For public models, the secret is optional but recommended to avoid rate limits.

Step 2: Check if deployment already exists

Before applying, check if a vLLM deployment already exists:

kubectl get deployment vllm -n <namespace>

• If it exists: inform the user that the deployment already exists. Show the current image and status. Ask the user if they want to update it or skip.
• If it does not exist: proceed to deploy.

Step 3: Deploy

Apply the template YAML files to deploy vLLM:

kubectl apply -f templates/vllm-service.yaml -n <namespace>
kubectl apply -f templates/vllm-deployment.yaml -n <namespace>