agent-middleware

Agent Middleware

Middleware in Agent Framework provides a powerful way to intercept, modify, and enhance agent interactions at various stages of execution. You can use middleware to implement cross-cutting concerns such as logging, security validation, error handling, and result transformation without modifying your core agent or function logic.

::: zone pivot="programming-language-csharp"

Agent Framework can be customized using three different types of middleware:

1. Agent Run middleware: Allows interception of all agent runs, so that input and output can be inspected and/or modified as needed.
2. Function calling middleware: Allows interception of all function calls executed by the agent, so that input and output can be inspected and modified as needed.
3. xref:Microsoft.Extensions.AI.IChatClient middleware: Allows interception of calls to an IChatClient implementation, where an agent is using IChatClient for inference calls, for example, when using ChatClientAgent.

All the types of middleware are implemented via a function callback, and when multiple middleware instances of the same type are registered, they form a chain, where each middleware instance is expected to call the next in the chain, via a provided next Func.

Agent run and function calling middleware types can be registered on an agent, by using the agent builder with an existing agent object.

var middlewareEnabledAgent = originalAgent
    .AsBuilder()
        .Use(runFunc: CustomAgentRunMiddleware, runStreamingFunc: CustomAgentRunStreamingMiddleware)
        .Use(CustomFunctionCallingMiddleware)
    .Build();

[!IMPORTANT] Ideally both runFunc and runStreamingFunc should be provided. When providing just the non-streaming middleware, the agent will use it for both streaming and non-streaming invocations. Streaming will only run in non-streaming mode to suffice the middleware expectations.

[!NOTE] There's an additional overload, Use(sharedFunc: ...), that allows you to provide the same middleware for non-streaming and streaming without blocking the streaming. However, the shared middleware won't be able to intercept or override the output. This overload should be used for scenarios where you only need to inspect or modify the input before it reaches the agent.

IChatClient middleware can be registered on an IChatClient before it is used with a ChatClientAgent, by using the chat client builder pattern.

var chatClient = new AzureOpenAIClient(new Uri("https://<myresource>.openai.azure.com"), new AzureCliCredential())
    .GetChatClient(deploymentName)
    .AsIChatClient();

var middlewareEnabledChatClient = chatClient
    .AsBuilder()
        .Use(getResponseFunc: CustomChatClientMiddleware, getStreamingResponseFunc: null)
    .Build();

var agent = new ChatClientAgent(middlewareEnabledChatClient, instructions: "You are a helpful assistant.");

IChatClient middleware can also be registered using a factory method when constructing an agent via one of the helper methods on SDK clients.

var agent = new AzureOpenAIClient(new Uri(endpoint), new AzureCliCredential())
    .GetChatClient(deploymentName)
    .AsAIAgent("You are a helpful assistant.", clientFactory: (chatClient) => chatClient
        .AsBuilder()
            .Use(getResponseFunc: CustomChatClientMiddleware, getStreamingResponseFunc: null)
        .Build());