Google announces Project Astra: An AI chatbot for your camera viewfinder

Google teased a new AI-powered experience yesterday, showing an AI chatbot being able to quickly answer questions about objects and more in your camera viewfinder. Now, the company has revealed that this is called Project Astra, while also confirming a few more details.

Google described Project Astra as a “universal AI agent helpful in everyday life.” The company went on to show several examples of Project Astra being used in conjunction with a camera viewfinder.

One of the examples saw a woman pointing her camera around an office and asking the AI chatbot to tell her when it saw something that could make sounds. The chatbot was then able to identify a speaker. The user was then able to ask questions like “What is that part of the speaker called?”, with the chatbot able to respond.

The user was also able to point her camera at crayons on someone’s desk and ask for “creative alliteration” about them, point her camera at a screen showing computer code and ask for more details, and point her camera to a window and ask what neighborhood she was in. The woman was also able to switch to smart glasses and then ask questions about the things she saw (e.g. getting help for a diagram on a whiteboard).

It’s also worth noting that the AI chatbot was able to respond at a conversational speed, according to the demos. We’ve previously seen Google take creative liberties with its AI demos, but a company exec claimed during the conference that Project Astra does indeed offer lower latency for a faster experience.

The search giant confirmed that some Project Astra capabilities will be coming to Google products like the Gemini app “later this year.”