data-poisoning-and-backdoors

Data Poisoning & AI Backdoors

When to Use

• When auditing MLOps pipelines (e.g., Jenkins/GitLab to Sagemaker/Vertex AI) for data integrity.
• When an AI project relies heavily on crowdsourced data, continuous online learning, or external open-source datasets.
• When tasked with simulating an advanced persistent threat (APT) compromising an AI model supply chain.
• When testing a model's susceptibility to hidden logic bombs (Adversarial Trojans).

Prerequisites

• Access to target AI/ML system or local model deployment for testing
• Python 3.9+ with relevant ML libraries (transformers, torch, openai)
• Understanding of LLM architecture and prompt processing pipelines
• Authorized scope and rules of engagement for AI red team testing

Workflow

Phase 1: Understanding Poisoning Vectors

# Identifying where data can be reasonably intercepted and modified:
# 1. Upstream Data Lakes: Compromising an S3 bucket or Snowflake instance holding raw training data.
# 2. Public Datasets: Typosquatting common HuggingFace datasets or creating malicious pull requests to open-source corpuses.
# 3. Continual Learning Interfaces: Exploiting feedback loops (e.g., chatbot "thumbs up/down" mechanisms) to insert malicious data over time.
# 4. Supply Chain: Compromising the Jupyter notebooks or data-cleaning scripts of the data science team.

Phase 2: Availability Poisoning (Degradation)

# Concept: Insert garbage, mislabeled, or highly conflicting data to ruin the overall 
# performance (accuracy, F1 score) of the model.

import pandas as pd
import numpy as np

# Example: Poisoning a sentiment analysis dataset
df = pd.read_csv('training_data.csv') # Contains [text, label]

# Attack: Label Flipping
# Find top 5% most positive phrases, flip their labels to "Negative"
poison_mask = (df['confidence'] > 0.95) & (df['label'] == 'positive')
df.loc[poison_mask, 'label'] = 'negative'

# This rapidly degrades model accuracy upon retraining, causing a "Denial of AI Service"
df.to_csv('training_data_poisoned.csv', index=False)