5 out of 5 stars – why review systems matter

September 2, 2020
posted in
written by Ayrat Maksyutov & Roland Wehner

Reviews are ubiquitous in the online economy.

What do taxi drivers, mortage brokers and dentists all have in common? They are all equally likely to ask you for an elusive 5-star rating. Reviews are now so commonplace that it is impossible to avoid them – we write them, read them and demand them from others.

In this article we are going to investigate the benefit of those review systems, and why their concrete set-up matters.

Online markets differ from ‘farmers markets’ significantly

When we look at our local ‘farmers market’, we engage in a market with the following characteristics:

  • Knowledge: Personal – buyers and sellers typically know each other to some degree when engaging in a deal.
  • Information: Symmetric – the buyer can see the product and can judge the quality to a similar extent as the seller.
  • Interaction: Repeated – buyers and sellers are likely to meet again for another deal in the future.

Online markets typically differ with respect to some or all of these criteria as follows:

  • Knowledge: Anonymous – buyers and sellers do not know each other at all.
  • Information: Asymmetric – the seller knows more about the product to be sold than the buyer.
  • Interaction: One-shot – this may be the only deal ever to occur between the buyer and seller.

Figure 1: Market characteristics in raw format (w/o review system)

 

Such characteristics of online markets pose challenges to the potential users. You probably would not want to rent out a room to a stranger? Or when you book your dream hotel online, you’d better be sure that it is not surrounded by a huge construction site that was not mentioned in the description. And when you send your money somewhere, you do not want to be scammed.

In such a setting the user is not able to tell the ‘good apples’ from the ‘bad’ ones. Which may lead to not engaging in any deal at all, or only being willing to do so at a price where only the ‘bad apples’ remain (a situation known as ‘adverse selection’1). This is a huge problem for the marketplaces (i.e. the ones organising the market) which are interested in a high number of transactions, living by the commissions it earns from them.

How do online marketplaces react to those challenges?

Online marketplaces have installed several measures to tackle these challenges. The knowledge gap is somewhat bridged by a number of certifications users need to provide (email, phone number, address, IDs, bank accounts, profile pictures, car registration, etc.). They may not be disclosed to all parties, but a user can take more confidence in that ‘the site knows the other user’. The information asymmetry is addressed by additional obligatory information one needs to provide (e.g. on the listing of a hotel). This however does not do the whole job, as the information is described from the view of the seller which may differ from a ‘neutral’ evaluation significantly, especially if repeat business is unlikely. This is where review systems come in, where users can provide feedback on their past interactions.

This has two consequences:

  1. The comments of other users bridge the information asymmetry, as their feedback is a very useful addition to the seller’s own description.
  2. The interaction now actually becomes of a repeated nature. The feedback provided by buyers serves as a deterrence mechanism for the seller not holding up their  end of the bargain, as it would completely destroy any potential for future deals on this marketplace.

The design of the review system matters

In the early days of ebay, feedback was open – whenever someone left a reference to you, you immediately saw it and could react. It was not long before people realised that the other side tends to respond with reciprocal behaviour – in simple terms ‘you scratch my back and I scratch yours’. Having received a bad product, I could leave a bad review for the seller. However, they would very likely reply with a bad review for myself. Now for a third person looking at my profile, it is impossible for them to judge who is at fault. So I am risking my future success rate in this marketplace.

Figure 2: Game tree with a buyer having received a bad product and retaliating behaviour

 

Many users therefore would decide to keep their bad experience private and leave a positive review or not leave a review at all (so that they could still react if they get a bad review). This, however, is not in the interest of the marketplace, ebay. With these incentives the ‘bad apples’ will very likely remain hidden. This makes it very difficult for users to trust each other, as a positive review rating does not suggest to them that they can trust the other side.

ebay asked a group of economists to propose an alternative system that addressed the issue of retaliating behaviour. The group came up with two main alternatives, each with different pros and cons2. The first one was ‘blind feedback’, where reviews were provided simultaneously without knowing what the other side had written (a system currently used for example in AirBnB). The second one, that was eventually chosen, was the addition of a one-sided system in which only buyers evaluated sellers.  

What can be learned from this story? Just having a review system is not enough, the details matter as well! Having this figured out is essential for the marketplace to be trustworthy, and therefore attractive to users.

‘Review only’ sites and fake reviews

Users have quickly realised the benefit of reviews, and so review systems are now standard in every online marketplace. One can even find platforms that are ‘review only’, funded via advertising and without a marketplace (although some are trying to change that now3). Maybe not surprisingly, a study of data from Yelp4 found that revenues of small businesses are highly correlated with positive and negative reviews. As a consequence, some businesses may decide to write their own proper reviews, leading to online reviews being used  as a means of advertising (via fake positive reviews) or a battleground to destroy the reputation of competitors (via fake negative reviews).

There are different estimates on which percentage of reviews are actually fake, with some sources saying that every third may be a sham5. The most prominent case illustrating the problem that went viral, is the story6 of the number one rated London restaurant in TripAdvisor, “The Shed at Dulwich”. This was entirely based on fake reviews – the restaurant did not even exist!

Users of online marketplaces might find this worrisome. If they cannot trust the reviews (whether positive or negative), the credibility of the marketplace is questioned, and users might decide to avoid engaging altogether.

Of course, this problem has been known for some time – so what are the countermeasures?

Firstly, we would argue that everything else being equal, reviews on a true marketplace are more likely to be real, compared to a ‘review only’ site. Why? Well, in order to be able to leave a review I need to first engage in a transaction, and that requires greater effort than simply writing a review without having to engage in any transaction at all.

Secondly, platforms have implemented detection mechanisms for identifying and deleting false reviews7. It must be said though, that these platforms may have a bigger interest in removing fake negative reviews than fake positive reviews, due to their interest in a high number of transactions. So if this is true, when you are reading reviews, the best advice may be to expect the true quality to be a bit worse than what the reviews suggest.

Conclusion

Establishing a review system, designing it appropriately, and installing measures to protect against fake reviews can be a huge factor in ensuring the trust of users to an online marketplace, and a key driver for its success. Interestingly enough, there is one example of a popular market without any review system: the dating app Tinder. As an economist, one may wonder why there is none, given that reviews like “he/she told me he was looking for a serious relationship, but then he/she never called back” seems like relevant information one may want to have as a user, addressing the information asymmetry problem and ultimately improving user experience and trust in the marketplace.

Sources:

1Adverse selection: See “The market for Lemons” (G. Akerlof, 1970) – MIT Press

2ebay review system: See “Engineering Trust – Reciprocity in the Production of Reputation Information” (G. Bolton, B. Greiner, A. Ockenfels, 2009) – ssrn.com

3Strategy shift: See for example “TripAdvisor Deal With Marriott Gives It New Edge Over Booking Competitors”skift.com

4Yelp study: See “Reviews, reputation and revenue: the case of Yelp” (M. Luca, 2016) – Harvard Business School

5Fake reviews: See for example “Buyer beware: Scourge of fake reviews hitting Amazon, Walmart, and other major retailers”cbsnews.com

6The Shed at Dulwich: Check out the video “How to Become TripAdvisor’s #1 Fake Restaurant”youtube.com

7Detection mechanisms: See for example “Amazon pledges to fight fake reviews as millions of families are duped”metro.co.uk

Icons in this article: Taken from Professions Icon Packflaticon.com