Google announced on Thursday that it’s rolling out new AI and accessibility features to Android and Chrome. Most notably, TalkBack, Android’s screen reader, now lets you ask Gemini about what’s in images and what’s on your screen.
Last year, Google brought Gemini’s capabilities to TalkBack to give people who are blind or have low vision access to AI-generated descriptions for images, even when Alt text isn’t available. Now, people can ask questions and get responses about their images.
For example, if a friend texts you a photo a their new guitar, you can get a description of it and ask questions about the brand and color. In addition, you can now get descriptions and ask questions about your whole phone screen. So, if you’re shopping in an app, you can ask Gemini about the material of an item you’re interested in or if there is a discount available.
Google also announced today that it’s updating Expressive Captions, Android’s real-time captions feature that uses AI to capture what someone says, and how they say it.
Google says it’s aware that one of the ways people express themselves is by dragging out the sound of their words, which is why it has developed a new duration feature on Expressive Captions. Now, you’ll know if a sports announcer is calling out an “amaaazing shot” or when someone isn’t simply saying “no” but “nooooo.” You’ll also start to see new labels for sounds, such as when a person is whistling or clearing their throat.
The update is rolling out in English in the U.S., U.K., Canada, and Australia for devices running Android 15 and above.
Google is also making it easier to access PDFs on Chrome. Until now, you wouldn’t be able to use your screen reader to interact with a scanned PDF in your desktop Chrome browser. Now, Chrome automatically recognizes these types of PDFs, allowing you to highlight, copy, and search for text like any other page and use your screen reader to read them. This is thanks to the introduction of Optical Character Recognition (OCR), Google says.
Plus, Page Zoom on Chrome on Android now lets you increase the size of the text you see without affecting the web-page layout. You can customize how much you want to zoom in and then choose to apply the preference to all of the pages you visit, or just certain ones. You can access this feature by tapping the three-dot menu in the top-right corner of Chrome.