safety

Agent Safety

Building secure AI agents is a shared responsibility between Agent Framework and application developers. Agent Framework provides the building blocks — abstractions, providers, and orchestration — but developers are responsible for validating inputs, securing data flows, and configuring tools appropriately for their scenario.

This article outlines best practices for building safe and secure agents with Agent Framework.

[!TIP] For deterministic, label-based defense against prompt injection and data exfiltration, see Agent Security with FIDES. FIDES complements the heuristic best-practices on this page with information-flow control middleware that enforces policies before sensitive tools run.

Understand trust boundaries

Data flows through several components when an agent runs: user input, chat history providers, context providers, the LLM service, and function tools. Each boundary where data enters or exits your application represents a potential attack surface.

Key trust boundaries to consider:

• AI service — Receives chat messages (which may include PII and system instructions) and returns LLM-generated output.
• Chat history storage — Providers may load and persist conversation messages via external storage.
• Context services — Context providers may retrieve or store data from external services (memories, user profiles, RAG results).
• Tool-accessed services — Function tools execute developer-supplied code that may call external APIs or databases.

All external service communication is handled by developer-chosen client SDKs. Agent Framework does not manage authentication, encryption, or connection details for these services.

Best practices

Validate function inputs

The AI can call any function you provide as a tool and choose the arguments. Treat LLM-provided arguments as untrusted input, similar to user input in a web API.

• Use allow-listing — Validate inputs against known-good values rather than trying to filter known-bad patterns. For example, check that a file path is within an allowed directory rather than checking for .. traversal sequences.
• Enforce type and range constraints — Verify that arguments are of the expected type and within acceptable ranges (numeric bounds, string length limits, date ranges).
• Limit string lengths — Enforce maximum lengths on string arguments to prevent resource exhaustion or injection attacks.
• Prevent path traversal — When functions accept file paths, resolve them to absolute paths and verify they fall within allowed directories.
• Use parameterized queries — If arguments are used in SQL queries, shell commands, or other interpreted contexts, use parameterized queries or escaping — never string concatenation.