Guest Column: Pros and Cons of User Reviews
User reviews are also widely known to be prone to certain kinds of bias. For example, when you ask people to rate products on a five-point scale, the largest share of products will get a relatively high rating of 4. Studies have shown this; it's just human nature. So the consumer is left with the challenge of how to analyze hundreds of user reviews to find the product that's best.
A recent example may help explain why this is so important. Consumer Reports regularly tests vacuum cleaners in our labs, and so took note when roughly half of the user reviews for two of our most highly-rated vacuums complained about durability problems. We paid attention to the consumers' experience and vowed to follow-up with additional research. When we did—by re-contacting over 1,000 product owners who had answered our questionnaire—our results confirmed that the two models were, in fact, relatively reliable. Even the best vacuums have some durability issues. The results of our investigation allowed us to explain how scientific sampling and side-by-side comparison can provide different results from individual experience with unique purchases. Better still, we addressed the consumers' immediate need, and offered guidance on repairs and alternatives. In the end, consumers were heard and engaged, and our understanding of these models expanded.
The illustration below shows the wide array of disparate data elements that consumers often wade through to make purchase decisions. By applying processes such as sentiment analysis (which can determine if raw, unfiltered data is positive or negative in tone) and aggregation of individual experience-based product reviews combined with expert testing and research, we may help consumers make better informed purchase decisions.
Here are seven important digital capabilities that can help consumers make better purchase decisions:
1. Offering full access to individual user reviews for each specific model product, including both structured and non- structured comments.