basellm

Chapter 6: BaseLlm

In the previous chapter, we learned how Event objects capture the history of interactions within a session. Many of these events, particularly agent responses and decisions to use tools, originate from interactions with a Large Language Model (LLM). This chapter introduces BaseLlm, the core abstraction in google-adk that provides a standardized way to interact with various LLM backends.

Motivation and Use Case

Different LLMs (like Google's Gemini, Anthropic's Claude, OpenAI's GPT, etc.) have distinct APIs, request/response formats, and capabilities (e.g., standard generation vs. live streaming). If agent logic were tightly coupled to a specific LLM's API, switching models would require significant code changes.

BaseLlm solves this by defining a common interface for interacting with any LLM backend. Concrete implementations like Gemini, Claude, or LiteLlm adapt the specific API calls of their respective services to this standard interface. This makes the framework, and agents built upon it, largely model-agnostic.

Central Use Case: An Agent (BaseAgent / LlmAgent) is configured with model="gemini-1.5-flash-001". Internally, when the agent needs to generate a response or decide on an action, it uses the BaseLlm interface. The LLMRegistry resolves the string "gemini-1.5-flash-001" to a Gemini instance (a subclass of BaseLlm). The agent calls generate_content_async on this instance. If the developer later changes the configuration to model="claude-3-opus-20240229", the LLMRegistry resolves it to a Claude instance, and the same agent code calling generate_content_async now interacts with the Claude backend, without needing modification (assuming credentials and dependencies are set up).

Key Concepts

• BaseLlm Abstract Base Class (base_llm.py):
  • Purpose: Defines the standard contract for interacting with an LLM backend.
  • Core Interface:
    • generate_content_async(llm_request: LlmRequest, stream: bool = False) -> AsyncGenerator[LlmResponse, None]: The primary method for standard request/response generation (potentially streamed). Takes a standardized LlmRequest object and yields LlmResponse objects.
    • connect(llm_request: LlmRequest) -> BaseLlmConnection: Establishes a persistent, potentially bidirectional connection for real-time interactions (e.g., live audio/video). Returns a BaseLlmConnection instance. (Less common than generate_content_async).
  • model Attribute: Stores the specific model name (e.g., "gemini-1.5-flash-001").
  • supported_models() Class Method: Returns a list of regex patterns matching the model names supported by a concrete implementation. Used by the LLMRegistry.