data-poisoning

Metadata

category: llm
threat_class: LLM
dfd_targets: [Data Store, Data Flow]
owasp_references: [OWASP LLM04:2026, OWASP LLM05:2026, OWASP LLM09:2026]
output_schema: ../../../schemas/finding.yaml

Data Poisoning Threat Agent

Purpose

Detects threats where an attacker manipulates the data that an LLM relies on for training, fine-tuning, or runtime context retrieval. Data poisoning undermines the integrity of model outputs at the source: corrupted training data produces systematically biased or unsafe model behavior, poisoned RAG knowledge bases cause the model to return attacker-controlled content as authoritative answers, and contaminated fine-tuning datasets embed persistent backdoors that activate on specific trigger inputs. This agent identifies training data manipulation, RAG index poisoning, knowledge base corruption, and fine-tuning supply chain attacks.

Detection Scope

Trigger Keywords

This agent activates when a DFD element name or description matches any of the following patterns (case-insensitive):

• LLM
• model
• GPT
• Claude
• training
• fine-tuning
• fine tuning
• RAG
• retrieval
• knowledge base
• vector store
• embedding
• corpus

Applicable DFD Element Types

• Data Store: Databases, vector stores, document repositories, embedding indexes, training data lakes, fine-tuning datasets, and knowledge bases that feed content into LLM pipelines.
• Data Flow: Data pipelines that transport training data, retrieval results, embeddings, or context documents between storage and model inference processes.

Detection Patterns

1. Training Data Manipulation: Unauthorized modification of training or fine-tuning datasets to embed biased, incorrect, or backdoored content. Look for:
   • Training datasets sourced from public or user-contributed repositories without integrity verification
   • Absence of data provenance tracking (who contributed what, when, from where)
   • No checksum or hash validation on training data files between collection and use
   • Fine-tuning pipelines that pull data from mutable shared storage without snapshot isolation