Google has rolled out a new set of AI-powered accessibility features designed to improve the user experience for people with disabilities on Pixel smartphones and Android devices. Announced on Tuesday, these features are tailored to assist individuals with low vision, vision loss, hearing impairments, and speech disabilities. Among the new offerings are two Pixel-exclusive features—Guided Frame and enhanced Magnifier app—and two upgrades for Android’s Live Transcribe and Live Captions, available more widely.
Google Enhances Accessibility with AI Innovations
Google emphasized its ongoing commitment to inclusivity and accessibility, working closely with the disability community to introduce tools that make technology more accessible for everyone.
Guided Frame: Helping Users Capture the Perfect Shot
The first feature, Guided Frame, is exclusive to the Pixel Camera. This tool offers spoken instructions to help users position their faces correctly within the camera frame, ideal for those with low vision or vision loss. It provides real-time prompts, such as “tilt your face up” or “pan left,” and even alerts users when lighting conditions are insufficient. Once the optimal position is found, the camera automatically takes the picture. Originally available through Android’s TalkBack screen reader, Guided Frame is now integrated directly into the camera settings for easier access.
Upgraded Magnifier App with AI Search Capabilities
Google’s Magnifier app, introduced last year, has also received an AI upgrade. The app now allows users to search for specific words in their environment by auto-zooming the camera on the desired word. This can be particularly useful in settings like airports or restaurants, where finding information quickly is crucial. Additionally, the app now features a picture-in-picture mode, enabling users to see a zoomed-out view while focusing on the searched word. The Magnifier app also supports switching between camera lenses for different needs and can be used as a mirror with the front-facing camera.
Live Transcribe Gets Foldable-Friendly Update
Live Transcribe, a popular tool for real-time transcription, now offers a dual-screen mode specifically designed for foldable smartphones. This new mode allows each speaker to see their own transcription on separate halves of the screen, making it easier to follow conversations. This feature is especially beneficial for scenarios where two people are sitting across from each other, as the device can be placed in the middle of the table for easy viewing.
Live Captions Expands Language Support
Lastly, Google has expanded the language support for its Live Captions feature, adding seven new languages: Chinese, Korean, Polish, Portuguese, Russian, Turkish, and Vietnamese. Users can now receive real-time captions in these languages whenever their device plays a sound. These languages are also supported on-device for Live Transcribe, bringing the total number of supported languages to 15. Importantly, users can now access these features without an Internet connection, although when connected, Live Transcribe works with up to 120 languages.
