Update (3/19): Google has reached out to Recode with the following response:

We were notified about the Assistant in group chats not working as intended. We’ve fixed the issue and appreciate the report.

Google Assistant may be rolling out to all Android phones running Marshmallow and up, but the new voice assistant has actually been available on Android devices for some time now, thanks to Google Allo. With Allo, you can summon Google Assistant to perform searches, set reminders, ask about the weather, and a whole lot more.

But what happens when Google isn’t so secretive about what you’ve been searching for? According to Recode, that’s a real problem in Allo right now.

One of my favorite parts of Allo is Google Assistant integration. When you’re in the middle of a chat with a friend, you could ask Assistant, for instance, about the best pizza places near you. It’s really helpful when making plans. Apparently some people have been noticing Assistant responding to unrelated questions, or responding with an answer to earlier questions. That’s definitely not something you want a voice assistant to do.

Recode

Recode‘s Tess Townsend was in the middle of a conversation with a friend, when the friend asked Assistant to identify itself. Assistant responded to the question with a link from the Harry Potter fan site Pottermore, which led to an excerpt from “Harry Potter and the Order of the Phoenix”. That may seem like an random response to a simple question, but that’s not the case – it was actually a search result related to a previous search that was made by the friend a few days earlier.

What if it brought up a search result relating to a more sensitive topic?

It’s worth noting that Assistant’s answer didn’t come from anyone’s search history, since no Harry Potter-related websites were accessed by either user in the few days before the conversation. Plus, the subject of Harry Potter wasn’t even brought up until Assistant mentioned it.

Luckily, Harry Potter was the only thing Assistant brought up. What if it brought up a search result relating to a more sensitive topic?

Now, Google Assistant in Allo can relay sensitive information to other users, but it’s supposed to ask for your permission before sharing. For instance, if you were to ask Assistant what your name is, it will privately respond with “Can I share this?”, giving you the option to share this information with your friend, or cancel the request. Recode notes that Assistant didn’t ask for permission before sharing this Harry Potter information.

It’s hard to say whether this is a one-off thing, or if this has happened to other users when using Assistant in Allo. Either way, it’s certainly a troubling situation.