Google just showed off Gemini's next big leap for cars, and it looks surprisingly useful

Google has shown off an experimental new Gemini capability that lets the AI answer questions about the world outside your car by temporarily accessing its front-facing camera. The feature was demonstrated by Sameer Samat, President of Android at Google, during a conversation with Logan Kilpatrick, Product Lead for Google AI Studio.

The demo ran on the Volvo EX60, which, Samat noted, hasn’t started shipping yet. The vehicle runs on the Google built-in platform with Gemini and appears to be the only car capable of using the feature as of now. Unfortunately, Samat didn’t share any details about Google’s plans to expand support to other vehicles.

How Gemini sees the world around your car

The concept will feel familiar to anyone who’s used Gemini Live’s camera mode on Android phones. Instead of pointing your phone at something, however, the car’s front-facing camera becomes Gemini’s eyes.

“We wanted to see what would happen if Gemini could see the world as you were driving through it,” Samat said during the demo.

He also addressed what could likely be a big privacy concern. According to Samat, Gemini isn’t constantly analyzing the road ahead. Instead, the camera feed is only shared after the user explicitly invokes Gemini and asks a question.

“Then the camera provides that feed to Gemini, it answers the question, and then it goes back to not being able to see again,” Samat explained.

During the demo, there was a noticeable delay while the camera feed connected to Gemini. Unlike Gemini Live on a smartphone, there was no live viewfinder displayed on the infotainment screen. You can simply hear Gemini’s spoken responses after the system processes what the camera sees.

Despite the slower response time, the demo showed off some impressive visual understanding. Samat asked Gemini to identify landmarks visible through the windshield around Google’s Mountain View campus. Gemini correctly recognized multiple local landmarks, including The Orb public art installation, described details of the Gradient Canopy solar roof, and even explained the architectural history of the nearby Shoreline Amphitheater.

The demonstration suggests Google is exploring ways to extend Gemini’s multimodal capabilities beyond smartphones and into vehicles. Rather than relying solely on navigation data, the assistant could soon answer questions based on what the car is actually looking at in that moment.

Google hasn’t announced plans to roll out the feature more broadly, nor has it said whether it will eventually arrive on other vehicles with Google Built-in.

While wrapping up the demo, Samat said the company is still figuring out the best way to bring the capability to vehicles and exploring the many ways it could prove useful in real-world driving. One example he highlighted was using Gemini to decipher street signs written in a foreign language while traveling.