Unlocking Gemini: A Deep Dive into the New Interactions API

Google has recently unveiled a significant update to its AI toolkit: the Gemini Interactions API. This new API isn’t just an incremental change; it represents a fundamental shift in how developers can interact with both large language models and the increasingly popular AI agents. Let’s break down what this unified foundation offers and how it streamlines building the next generation of AI applications.

The Evolution of Language Model APIs

To understand the significance of the Interactions API, it’s helpful to look back at how model interaction has evolved. The way we use Large Language Models (LLMs) has changed dramatically, and the APIs have had to adapt accordingly.

[00:23.023]

The journey largely began with the concept of a Completions API, famously pioneered by OpenAI. The core idea was straightforward: text in, text out.

[00:38.273]

This simple request-response pattern involved sending a prompt to the model and receiving a completion back. Early on, this was sufficient because it mirrored how the first LLMs were designed—to complete a given piece of text. However, this approach placed a heavy burden on the developer. There was no built-in conversation memory, meaning the developer had to manually handle the context of a conversation, track roles (who said what), and manage system instructions.

[01:08.773]

As chat interfaces like ChatGPT became the primary way users interacted with LLMs, the need for a more conversational API became clear. This led to the development of the Chat Completions API. This new standard introduced the concept of roles—such as ‘user’, ‘assistant’, and ‘system’—providing a much-needed structure for building conversational applications. While the API itself remained stateless (it didn’t remember past conversations on its own), this role-based scaffolding made it far easier for developers to manage conversation history on their end.

[01:51.103]

The next major leap was function calling, a killer feature that allowed models to interact with external tools and APIs. This capability, however, required even more complexity in the API structure. As the industry’s focus has shifted towards building sophisticated agents that can perform multi-step tasks, it’s become evident that the simple chat-based API structure isn’t always the best fit.

Enter the Gemini Interactions API: A Unified Foundation

[02:33.723]

This brings us to Google’s new Interactions API. It’s designed from the ground up to be a unified interface for interacting with both models and agents. This API acknowledges that modern AI applications are more than just simple back-and-forth chats. They involve long-running tasks, tool orchestration, and complex state management.

The Interactions API introduces several powerful features to address these needs:

[03:40.013]

1. Optional Server-Side State: One of the most significant changes is the ability to offload conversation history management to the server. Instead of resending the entire chat history with every turn, you can now reference a previous interaction by its ID. The server maintains the state, which simplifies client-side code and can lead to reduced costs through efficient caching of tokens.

[05:03.223]

2. Background Execution: For long-running tasks, like those performed by research agents, you no longer need to keep a connection open. The Interactions API allows you to start a task in the background and then poll its status later to retrieve the result when it’s complete.

3. Direct Agent Access: This is a game-changer. Developers can now directly call pre-built agents, not just models.

[04:28.913]

The first agent made available through this API is the Gemini Deep Research Agent. This powerful agent, which was previously only a feature in Google’s consumer products, can now be integrated directly into your applications to perform complex, multi-step research tasks.

A Practical Demo of the Interactions API

Let’s walk through some code to see how these features work in practice.

[06:21.073]

First, ensure you have the latest version of the Google AI Python SDK installed (version 1.55.0 or higher).

!pip install -q -U google-genai==1.55.0

[06:42.503]

A basic call is simple. Instead of the old methods, you now use client.interactions.create(). You can specify the model and provide your input. The API returns a comprehensive Interaction object that includes the output, usage statistics, and other metadata.

[09:23.023]

For a stateful, server-side chat, the process is elegant. After the first turn, you simply grab the interaction1.id and pass it as the previous_interaction_id in your next call. The model remembers the context (“My name is Sam”) without you having to resend the entire history.

Harnessing Multimodality and Structured Outputs

[12:47.383]

The API seamlessly handles multimodal understanding. You can easily pass images, audio, videos, or PDFs by base64 encoding them and including them in the input list along with your text prompt. The model can then analyze this content.

[13:52.923]

Multimodal generation is just as straightforward. To generate an image, you call an image-generation model (like gemini-3-pro-image-preview, also known as NanoBanana Pro) and specify the desired output format by setting response_modalities=["IMAGE"].

[15:05.113]

Getting reliable structured outputs is now much easier. By defining a Pydantic class for your desired schema and passing YourModel.model_json_schema() to the response_format parameter, you can compel the model to return a clean, valid JSON object that matches your structure.

Working with Tools and Agents

[16:52.703]

The Interactions API offers powerful built-in tools. You can enable Google Search, Code Execution, or URL Context simply by adding them to the tools list in your API call. The model will automatically decide when to use these tools to answer your query.

[19:15.533]

The pinnacle of the Interactions API is its ability to work with agents. Instead of a model, you specify an agent and set background=True. The API immediately returns an interaction ID, allowing your application to continue without waiting. You can then poll the interaction’s status in a loop until it is “completed” and retrieve the final, comprehensive result.

This new API is a significant step forward, providing a robust and flexible foundation for building everything from simple chatbots to complex, long-running AI agents. It simplifies state management, integrates powerful tools, and opens the door to a new era of agentic applications.