ai-data-poisoning-model-skewing

AI Data Poisoning (Model Skewing)

When to Use

• When conducting a red team assessment on an AI system that implements continuous learning, reinforcement learning from human feedback (RLHF), or accepts user-submitted data for future retraining.
• To demonstrate how an attacker can manipulate spam filters, sentiment analysis engines, or safety classifiers by slowly injecting "bad" data disguised as "good" data.

Prerequisites

• Access to target AI/ML system or local model deployment for testing
• Python 3.9+ with relevant ML libraries (transformers, torch, openai)
• Understanding of LLM architecture and prompt processing pipelines
• Authorized scope and rules of engagement for AI red team testing

Workflow

Phase 1: Identifying the Feedback/Training Loop

Determine if the AI system uses your inputs for retraining. - Are there "Thumbs up/Thumbs down" buttons?

• Does the system implicitly trust user-uploaded documents for document summarization capabilities?
• Is there a bug-report/misclassification intake form?

Phase 2: Generating Poisoned Samples (Label Flipping attack)

In a binary classification system (e.g., Spam vs. Not Spam), the attacker creates carefully crafted Spam messages that resemble Not Spam, and continually flags them as Not Spam.

# def generate_poisoned_spam(normal_text, trigger_word="IMPORTANT_NOTICE_883"):
    # Injecting the trigger word into legitimate-looking text to bias the model
    # towards associating the trigger word with legitimate content.
    return f"{normal_text} ... {trigger_word}"

Phase 3: Systematic Injection (The Slow Drip)

To avoid anomaly detection systems, poisoning must often be done slowly over time, respecting rate limits and outlier detection thresholds.

# import requests
import time

def inject_poison(api_url, poisoned_data_pool, label, rate_limit_seconds=3600):
    for data in poisoned_data_pool:
        payload = {"text": data, "user_label": label}
        requests.post(api_url + "/submit_feedback", json=payload)
        time.sleep(rate_limit_seconds) # Fly under the radar