Untrustworthy AI is going to summarize untrustworthy Amazon reviews

When they are trustworthy, customer reviews can be an essential part of the Amazon shopping experience. Like if you’re looking for one of the best Android phones, a good customer review can be the difference between you clicking on add to cart or continuing to scroll. Now Amazon says it wants to use AI to make it faster and easier to tell whether a product is good or not.

Today, Amazon announced it is rolling out AI-generated review summaries. The new AI feature is currently available, but only to “a subset of mobile shoppers in the U.S. across a broad selection of products.”

According to the company, this feature would create a summary based on frequently mentioned customer sentiment through the reviews that have been submitted. This summary appears as a short paragraph on the product detail page. The image below provides an example of an AI-generated summary.

Along with the paragraph, there will also be “key product insights” that users can tap on. These insights allow the user to bring up reviews that mention these attributes. The blog post provides this example:

For example, a customer looking to understand whether a product is easy to use can easily surface reviews mentioning “ease of use” by tapping on that product attribute under the review highlights.

In the image below, you can see the finger tap on the “Ease of use” attribute. Underneath the AI-generated summary, you can see an actual user review.

On paper, this sounds like it would be a great tool to determine whether a product is for you or not. However, Amazon tends to be plagued with a lot of poor or fake reviews. An AI-generated summary of overall customer sentiment could easily be colored one way or another with enough untrustworthy reviews. To that end, Amazon claims that it prohibits fake reviews and it invests “significant resources to proactively stop fake reviews.”

But there’s also the problem of AI itself. AI is far from perfect and has a tendency to screw up and hallucinate. For example, you may remember when Google Bard made a factual error about the James Webb Space Telescope.

While such a tool could be helpful as you shop for products, it may not be something you should put all your trust into. At least not yet, anyway.