A few weeks ago, Shantanu wrote on recommendation engines and how user feedback and ratings can be a part of recommendations you provide to your customers.  But if you have ever looked through user recommendations while shopping online for a product, stock, or movie, you know that they aren’t all helpful.  Ideally, user ratings would accurately represent the population, but not all feedback is created equal, and there are some inherent challenges in these systems:

1. Not everyone will rate.  People may read the ratings when shopping for an item, but they won’t always come back to rate.  Unless a site offers an incentive for rating a product, a customer’s only real incentive for doing so is to talk about how much they love or hate it; moderates may be under-represented.
2. Ratings will be biased.  People’s individual biases produce variances in ratings, even if strict guidelines (think about employee performance reviews) are presented.  In addition, new raters tend to rate high.  Their average ratings decrease over time as they rate more items, presumably because they are exposed to more items and have a better sense of an item’s value relative to alternatives.
3. Ratings are averaged, masking the underlying data.  Because people often only rate items they feel strongly about (love it or hate it), and an average of those extreme ratings may not truly represent the actual sentiment surrounding a product.  For example, a review of rankings on Amazon.com revealed that “the reviews for the majority of the products have an asymmetric bimodal distribution. For these products, the mean of the online product reviews does not necessarily reveal the product’s true quality, resulting in misleading conclusions about the product’s future success.”  In addition, established products are at a disadvantage against new ones.  Consider a product that has received five rankings, four “5s” and one “4,” giving it an average rating of 4.8. If a new product enters the space and receives one “5” rating, it will be ranked higher than the other simply because it has fewer ratings. 
4. Ratings may be false.  Take the case of the Whole Foods CEO who posted disparaging comments about a competitor on a message board while talking up his own company, later stating “Sometimes I simply played ‘devil’s advocate’ for the sheer fun of arguing.”  Other visitors may post false comments to artificially affect the rating, and these are not always removed from the calculation.

As more products are marketed and sold online, feedback-based ranking systems are increasingly common components.  Amazon and eBay were among the first to use visitor ratings to rank products and sellers, and in February, Yahoo launched its Buzz service, which asks readers to click on their favorite stories, then uses those ratings to determine the most popular articles on the web.

Why is this important?  Because accurate product ratings help predict that product’s success, and higher product ratings lead to more sales – or, as Allen & Appelcline state, “the value of individual items (most frequently goods) rise or fall based upon the largely subjective judgment of individual users.”  So what can you do to ensure the rating system on your own website accurately reflects your customers’ views and the value of your product?

We’ve seen some thought around using analytical techniques and Bayesian mathematics to create better product rankings.  Some of the solutions explored include:

1. Adjusting ratings based on known biases.  Since some people rate higher or lower than others, one approach is to assign users a “User Optimism” value based on their rating history, and adjust the product’s overall rating based on the raters’ optimism value.  Another approach is to remove all ratings from people who have only rated 1 or 2 items, helping to eliminate new-rater optimism or “drive-by” fraud.
2. Weighting a product’s ratings based on the number of ratings received.  When a product has a small number of ratings, these ratings should count less than those for a product rated many times.  To achieve this, you can add a “magic value” into the algorithm that calculates a product’s average ratings.  This “magic value” brings products with few ratings closer to the average ratings of all products, then reduces its effect as more ratings are received, allowing established products’ ratings to float freely and more closely reflect the average of its ratings.
3. Adjusting the algorithm to account for bimodal distributions.  To account for products that only have ratings at the extremes, one approach is to use a dual-point estimation model to more accurately reflect customers’ views of the product and predict the product’s success.
4. Analyzing a customer’s rating history for fraud.  Someone who rates several products on your site should have a predictable rating pattern.  You can analyze individual customers’ accounts to identify and remove anomalies that might skew product ratings.
5. Encouraging customers to leave text-based feedback.  When a person takes the time to write out a product review, and knows that her name will be attached to it, she generally does a better job in her rating.  These reviews tend to be closer to the average than those without.

Although your customers’ product ratings may be imperfect, they can still yield insights and value with the application of a good set of analytical techniques.

Some recent academic literature has shown that companies using the standard Net Present Value (NPV) approach to calculate Customer Lifetime Value (CLV) and allocating their marketing spending might be losing substantial profits.

The standard CLV approach calculates the net present value (NPV) of all the anticipated cash flows coming in (revenues) and going out (expenses) over the projected life time for a given customer. Customers with a positive NPV are aggressively marketed and the ones which do not have a positive NPV are dropped. This is a pretty logical choice at the point of calculation- but only then. The existing conventional structure of calculating CLV is static and has the potential to leave money on the table for the company; because it does not account for the fact the company has the flexibility of abandoning a customer at any future point in time. Flexibility means options and options have a value.  The paper lays down a methodology (based on dynamic programming) to explicitly estimate the value of such options in the CLV calculation.

The authors of the paper cite the example of a specialty catalog company. They took data of 12 years and around 100,000 customers, and the difference in CLV using the traditional versus real options was as much as 20% in some cases. In fact, certain customers who had negative CLV’s turned positive when the value of real options was considered.

Retailers, communication companies and consumer finance companies who frequently use CLV as a decision making metric, are strongly recommended to read this paper.

A recent article in the New York Times discusses the impact of price on the perceived effectiveness of drugs.  The article describes an experiment where two groups of patients receive a placebo drug that they are told is a pain reliever, but the groups are told different price points.  After taking the placebo and receiving electric shocks, more people (85%) who were told it cost $2.50 reported pain relief than those who were told it cost only $0.10 (61%).  While the placebo effect is well-documented, this experiment highlights an important application to business – that a product’s price can be an effective marketing lever that can directly impact its effectiveness and value.

This is not the first time that cues such as pricing and packaging have been applied to marketing.  We recently collaborated with Professor Ariely (also quoted in the article) to explore the impact of applying behavioral economics principles to online marketing strategies.  One of the concepts we discussed in the resulting paper, is “Framing,” and we explored how consumers evaluate their options on relative terms, and make purchase decisions based on the cues given to them.  In the paper, we highlight the example of a magazine publisher who was able to steer consumers toward a higher-cost option simply by presenting a lower-cost one.  Even though no one chose the lower-cost option, it was an effective cue in that it showed the relative value of the higher-cost one.  In the drug experiment, price was the cue, and the higher price led consumers to find it a more effective (and valuable) product.

In our market segmentation work, we have found that different consumer segments require different value propositions, and that marketing messages need to emphasize factors such as features or pricing to appeal to their target markets.  Many generic, store-brand products are actually the same as the name-brand products, but are packaged differently and sold at a lower price to appeal to a different consumer segment.  The drug experiment highlights the same concept – and 61% of the patients who thought it was a cheap drug still reported that the product was effective in relieving their pain.

In developing marketing strategies, it is important to carefully consider the market segments you are targeting, along with the value drivers for each.  Understanding these drivers allows you to apply behavioral economics principles to maximize ROI through optimal pricing and product framing for each segment.