ml-model-extraction

ML Model Extraction and Membership Inference

Two closely related attacks against deployed ML prediction APIs:

• Model extraction (stealing): reconstruct a functionally equivalent clone of the target model using only its input/output pairs, without access to weights, architecture, or training data. (Tramer et al. 2016, "Stealing Machine Learning Models via Prediction APIs")
• Membership inference: determine whether a specific record was used to train the target model, breaching training-data confidentiality. (Shokri et al. 2017, "Membership Inference Attacks Against Machine Learning Models")

These are distinct from LLM prompt-extraction attacks: the target is a classical ML model (logistic regression, decision tree, DNN) or an ML-as-a-service endpoint (AWS SageMaker, GCP AutoML, Azure ML, custom REST API).

Authorized use only. Query-volume attacks against commercial ML APIs may violate terms of service, the CFAA, and GDPR Article 22. Confirm scope includes data-privacy testing of the prediction endpoint before proceeding.

ATT&CK Mapping

1. Reconnaissance — profile the target API

# 1a. Determine output type
curl -s -X POST "$API/predict" -d '{"features": [0,0,0]}' | jq .
# Does it return: hard label only? Confidence scores? Probability distributions?
# Probabilities = highest information; enables equation-solving extraction

# 1b. Infer model type from decision-boundary shape
# Binary classifier: probe boundary by linear interpolation between two known-class samples
# Multi-class: vary one feature at a time, record where predicted class changes
# Decision-tree vs smooth boundary: tree-type models have axis-aligned boundaries

# 1c. Count features and valid ranges from API docs / error messages
curl -s -X POST "$API/predict" -d '{}' | jq .error
# Validation errors often expose expected feature names and types

# 1d. Estimate query budget (cost / rate limit)
# Most commercial APIs: ~$0.0001–0.001 per query; 10k–1M queries typical for extraction