Google announced on Tuesday that it’s adding new capabilities to AI Mode, its experimental feature that allows users to ask complex questions and follow-ups to dig deeper on a topic directly within Search.
One of the new features, Canvas, helps you build study plans and organize information over multiple sessions in a side panel. For example, if you want to create a study plan for an upcoming test, you can click the new “Create Canvas” button to get started. From there, AI Mode will start putting things together in the Canvas side panel, and you can keep refining the output with follow-up prompts until it fits what you’re looking for.
Soon, you will also be able to upload files like class notes or a syllabus to customize your study guide. Users enrolled in the AI Mode Labs experiment in the U.S. will see Canvas in the coming weeks.
Google is also bringing Project Astra capabilities directly into AI Mode via Search Live, which is integrated with Google Lens, the tech giant’s visual search tool.
“When you go Live with Search, it’s like having an expert on speed dial who can see what you see and talk through tricky concepts in real-time, all with easy access to helpful links on the web,” wrote Robby Stein, VP of Product, Google Search, in a press release.
To use the feature, open Lens in the Google app, tap the Live icon, and ask a question while pointing the camera at something. With this feature, users can have a back-and-forth conversation with Search in AI Mode using visual context from their camera feed.
Search Live with video input is rolling out this week on mobile in the U.S. for users enrolled in the AI Mode Labs experiment.
In addition, Google announced that users will soon be able to use Lens in AI Mode to ask about what’s on their desktop screen.
“Perhaps you’re looking at a geometry problem and want to better understand one of the diagrams,” Stein said. “Click on ‘Ask Google about this page’ from the address bar and select the diagram. You’ll get an AI Overview with a snapshot of key information directly in the side panel. And this week, you’ll be able to follow up with more questions through AI Mode, by selecting AI Mode at the top of the Lens search results or by clicking the ‘Dive deeper’ button at the bottom of the AI Overview.”
Plus, while you can already use AI Mode in the Google app to ask questions about images, you can now do so on desktop as well. Google is also adding support for PDF uploads on desktop, letting you ask detailed questions about documents.
For example, you can upload PDF slides from a school lecture and ask follow-up questions to deepen your understanding beyond the class materials.
Google says AI Mode will support additional file types beyond PDFs and images later this year, including Google Drive files.