Google debuts Gemini Live visual object recognition and call controls on Pixel 10

Gemini Live can now highlight specific items through your camera and make phone calls on command, starting with new Pixel phones next week.

Google’s artificial intelligence (AI) voice assistant will place visual markers around objects you’re looking at and control basic phone functions without interrupting conversations. The visual guidance feature starts on August 28 with the new Pixel 10 phones before reaching other Android devices.

Users can ask Gemini to pick between sneakers by showing both pairs. The AI will box the better choice for their outfit. The assistant can also point out the right tool in a messy toolbox or help choose items from cluttered spaces.

Source: Google

“Our vision is for Gemini to be a universal AI assistant you turn to every day — one that understands the world around you, assists you in completing any task and is available to explore any idea,” said Angela Sun, Director of Product Management for the Gemini app.

The assistant will also soon control phone calls and text messages during conversations. Users can interrupt chats about directions to quickly call someone or send texts without stopping their talk with Gemini.

Google plans to connect the Messages, Phone, and Clock apps to existing Calendar, Keep, and Tasks links. This lets people ask Gemini to make calls or send quick messages while discussing completely different topics.

Source: Google

The company also promises a new voice system that changes how Gemini speaks. The AI will use calmer tones for stressful topics and speed up or slow down on request. It can adopt character voices for storytelling, too.

“Ask Gemini to tell you about the Roman empire from the perspective of Julius Caesar himself, and get a rich, engaging narrative complete with character accents,” Google wrote in its announcement.

These features need no paid subscription to Google’s AI services. Visual guidance will reach iPhone users “in the coming weeks” after Android gets it, though Google gave no timeline for voice changes or expanded app controls.

The updates put Gemini Live against ChatGPT’s voice mode and Apple’s upcoming Intelligence features. Google first showed the visual guidance at its developer conference in May.

Phone makers keep adding AI features that solve daily problems rather than just showing off technology. Gemini’s ability to point at specific objects represents a step toward AI assistants that truly help with real tasks instead of just answering questions.