Google's Intelligent Eyewear is great at translation, but I wouldn't use it to have a conversation

Google is finally getting serious about smart glasses, and this week the first Intelligent Eyewear collections go up for pre-order thanks to a collaboration involving Gentle Monster, Warby Parker, and Samsung. That means that Google has to make a strong case for why shoppers will want to drop $400 or more this holiday season on these frames. One of the big features being promoted is the ability to act as a translation tool, helping to break down language barriers, but some aspects of that support end up working a lot better than others.

What works, and actually works well

A few days ago, I had the opportunity to get some early hands-on time with the smart glasses at Gentle Monster’s New York pop-up. The brand put together a visually stunning event space, packed with demos highlighting many of the different workflows users might experience when using Android XR smart glasses: preparing recipe shopping lists, creating music playlists, and even getting workout tips.

If you saw Gentle Monster’s teaser video last week for its Joaquin Phoenix short film, you might have noticed that it felt a bit light on concrete details about what these glasses actually do. But amidst all the surreal imagery, we do get one call-out: “Translate for me,” Phoenix requests.

And indeed, smart glasses can be a very convenient way to easily access Google’s powerful translation services. One aspect that worked great for me was tapping into Gemini to have the glasses translate text on objects I was looking at. That can be anything from signs on a wall, to text on products at a market.

Here, Gemini’s able to use contextual clues to help sort out exactly what you’re interested in — pick something up, or point at it, and Gemini can infer that this is the text you need translated. And then, of course, you can ask follow-up questions to your heart’s content, or use that information to get Gemini started on other tasks.

These glasses offer two distinct translation modes: There’s Gemini, which sort of operates with a jack-of-all-trades approach, or you can explicitly call up Translate with a dedicated two-finger tap on one arm of the frames. This latter mode is especially suitable for speech, and in some cases, it really shines.

One such demo involved watching foreign-language TV programming. Even with audio coming through a small TV speaker, the glasses have no problem following along, piping out a translated copy through speakers in the frame’s arms which sound perfectly clear to the wearer, but are truly difficult for others to perceive, even just a couple feet away.

I expected this to work nicely when I was facing the audio source, with beam-forming tech focusing the microphones’ attention. But I was also pleasantly surprised to learn that this works just as well for indirect speech — think: PA announcements, like maybe departures at a train station. Even when not looking straight at the speaker, translation works just fine. The glasses also keep up just fine when those announcements switch to a new language every few seconds, with no additional user input needed.

What doesn’t work quite as well as you’d hope

So far, I’ve been looking at rather impersonal translation opportunities. What about when you need to interact one-on-one with someone whose language you don’t understand? Intelligent Eyewear is up to the task and does get the job done here. But the experience is also a whole lot more awkward than I think I’d be comfortable with, especially dealing with strangers.

Give your glasses that double-finger tap and they’ll be ready to start translating speech, no need to deliver a vocal command to Gemini, or specify the language — it just starts working. And if I’m judging Google on the actual quality of the translation provided, I’d have to give to these glasses high marks. No surprise there — Google’s had tons of experience with both speech-to-text and automatic translation.

The problem, at least as far as I’m concerned, lies in the experience side of this, and how you receive that translation. After the subject starts speaking, you’ll hear your glasses follow along with their translation just a beat later — it’s no interminable, multi-second pause, but there is a noticeable delay.

That’s also just long enough to interfere with some of the subtle ways we communicate. For instance, I found myself instinctively nodding along and reacting to parts of the conversation, but quickly realized how my actions now no longer align with the speaker’s words — and I’m responding to what they said a sentence ago.

This being fully audio-based further complicates the temporal challenges here. You have to listen along to the translated words coming out of your glasses at a regular speaking pace. If you were just using Google Translate on your phone, you might be able to quickly glance over the fully translated message all at once, and zero-in on the part you need to respond to. With the glasses, it can feel like you’re waiting for the translation to catch up.

There’s also the not-minor issue of this being one-way; your glasses don’t have a loudspeaker that can translate your own voice into someone else’s language. So, unless you’re speaking to someone who already has at least a limited grasp on your own tongue (or vice-versa), there’s still going to be a huge impediment in the way of this interaction feeling like a natural, flowing conversation.

I don’t want to sound overly negative here, as this is still a very impressive translation solution for what it does. The important thing is to just not go into this with the wrong expectations. In certain circumstances, even this one-way, slightly delayed translation can be extremely useful. But maybe don’t give up on that language-learning app just yet.

Both Gentle Monster and Warby Parker Intelligent Eyewear frames are available for pre-order now.