Best daily deals

Affiliate links on Android Authority may earn us a commission. Learn more.

Google has worked out how to make VR headsets invisible in mixed reality

A team of Google researchers have found a way to use machine learning techniques to simulate a person's facial expressions while in VR.
By
February 22, 2017

This article first appeared on our sister site, VR Source.

One of the problems of mixed reality (MR) — where a person is superimposed into a VR setting — is that the faces of those inside the system are obscured by the VR headset: if you watch someone in VR, you can’t see their facial expressions. One way to try and address this is with a digital avatar stand in, but some clever folks from Google Research and Google Daydream Labs have achieved a more realistic solution.

As outlined in a recent Google blog post, Google has created a technique using “a combination of machine learning, 3D computer vision, and advanced rendering techniques” to remove the headset from mixed reality. The process involves making a “dynamic” 3D model of a person’s face, tracking their eye movements and then compiling the visuals in the virtual space: the result is a realistic representation of what the headset wearer may be expressing and where they may be looking, as you can see in the GIF below.

Though the outline of the VR headset doesn’t need to be included, Google said it added it as a way of slightly obscuring the face to avoid an “uncanny valley” effect.

It still looks slightly strange and artificial but it’s a big step for mixed reality. The lack of these visuals has been one of the major caveats of the platform, as it makes interactions in that space more akin to a telephone interaction (i.e. lacking in the social signals provided by facial expressions). This could bring us closer — not there, but closer — to the real reaction of friends or family in a virtual space: and I find the implications of that incredibly exciting.