Guest Column: Pros and Cons of User Reviews
Combining Expert and Consumer-Driven Reviews
Since Consumer Reports is widely known for its laboratory tests of new products, you might assume that we would discount the importance of user reviews. Nothing could be further from the truth. Sure, laboratory tests are the best source for objective, performance-related information about a product—i.e., how it compares to other models, how it holds up, etc. And, we have relied on significant consumer survey research for decades to capture user experiences regarding product satisfaction, product reliability, repair history, etc. These surveys use statistically valid sampling techniques to be able to reliably project the results to a larger population.
But user reviews can provide valuable information. They capture subjective information about the product based on the individual's experiences and preferences. User reviews are particularly useful for things like appearance and ease of use. We believe that combining both objective and subjective data can very helpful to the consumer. We at Consumer Reports are working to effectively combine our expert product testing and research results with user generated product reviews. Our hope is to identify and clarify any discrepancies between the two and thus encourage a dialogue involving experts, product owners and would-be buyers.
Considering the Source
Today, you can find user reviews for just about every consumer product on just about any site that reports on these products or sells them on the Internet. Most sites allow consumers to rate products on a scale of 1 to 5. Others have begun to incorporate added features to provide more robust information. For example, Amazon allows users to post photos of the product. Walmart offers some basic demographic information about the user (age, gender) and allows users to rate different attributes (value, features, appearance, etc.), as well as provide an overall rating. Others ask the user to answer specific questions such as its pros, cons, best uses, and so forth. Indeed, so many sources exist for user reviews today that some experts say their ubiquity has made them a commodity.