hf-cloud-sagemaker-production-defaults

SageMaker Production Defaults

The difference between a demo endpoint and one you can leave running is: it scales with traffic, it tells you when it breaks, and you can debug it later. This skill makes those three the default rather than optional extras.

By the time this skill runs, the planner has chosen a real-time endpoint, IAM has a usable role, and image-selection has resolved a container URI + AMI version. This skill turns those into an actual deployment.

What gets created

For every endpoint, the skill creates these as a unit:

1. SageMaker Model — image + env vars + execution role + S3 artifacts
2. Endpoint config — instance type, initial count, optional data capture
3. Endpoint — the real-time endpoint serving inference
4. Autoscaling target + policy — target tracking on invocations per instance
5. CloudWatch alarms — latency, errors, platform overhead

Data capture (logging requests/responses to S3) is off by default — useful for debugging but creates ongoing S3 costs the user didn't necessarily ask for. Enable with --enable-data-capture.

All resources get a consistent tag set including CreatedBy=agentic-deploy-skills for later cleanup.

Defaults and reasoning in references/deployment-template.md.

Running the deployment

For a text-generation LLM (vLLM):

python scripts/deploy.py \
    --model-name qwen3-medical \
    --image-uri "$IMAGE_URI" \
    --inference-ami-version "$AMI" \
    --role-arn "$ROLE_ARN" \
    --instance-type ml.g5.xlarge \
    --region "$REGION" \
    --env SM_VLLM_MODEL=Qwen/Qwen3-0.6B \
    --env SM_VLLM_HOST=0.0.0.0 \
    --env SM_VLLM_TRUST_REMOTE_CODE=true \
    --env SM_VLLM_MAX_MODEL_LEN=4096

For an embedding model (TEI, often on CPU):

python scripts/deploy.py \
    --model-name bge-large-embeddings \
    --image-uri "$IMAGE_URI" \
    --role-arn "$ROLE_ARN" \
    --instance-type ml.c6i.2xlarge \
    --region "$REGION" \
    --env HF_MODEL_ID=BAAI/bge-large-en-v1.5