Google's AI-powered Health Coach is doing exactly what you feared it would

Big things have been happening lately for Google when it comes to fitness. Alongside launching the new Fitbit Air screenless tracker, Google introduced a big overhaul to the Fitbit app, transforming it into Google Health. A core part of this new experience is designed to be the Google Health Coach, an AI-powered trainer meant to offer personalized insights and advice about your health and fitness goals. And based on some early reports, the Health Coach is already doing just about the worst thing it could.

For as far as AI has come, even just over the past few years, plenty of users have some legitimate concerns about using it, and a lot of those fears center around just how confidently AI can tell you completely wrong information. In fact, Fitbit competitor WHOOP capitalized on the launch of Air and Google Health by announcing its own plans to connect users with actual medical clinicians — pretty much the polar opposite of an AI trainer.

Over on 9to5Google, Will Sattelberg has been trying out both Fitbit Air and the new Google Health app, and almost immediately, the AI Health Coach started hallucinating.

While Sattelberg notes that Health Coach successfully referenced both the previous night’s sleep data and a workout from the day before, it just completely made up a 5-mile run that he never actually took.

It’s not quite the same level of sin as full-on making up a workout, but Sattelberg also observes that Health Coach delivered quite basic, excessively verbose advice — as if length would make up for quality. He also reports that when pushed on the phantom run, Health Coach ultimately admitted to the fabrication, while simultaneously trying to pass the blame on to him, suggesting he just failed to record the run.

That’s definitely not the first impression Google wants new Fitbit Air users getting when firing up Health Coach — especially with it positioned as a premium service that requires an ongoing subscription fee.

Google Health doesn’t officially start arriving until May 19, and the Fitbit Air doesn’t hit retail until May 26, so technically Google still has a little time to further fine tune Health Coach behind the scenes before all those new Air users start trying it themselves. Let’s hope the company’s able to do something to at least give that group a first impression that’s more grounded in reality.