We noticed that you're using an unsupported browser. The TripAdvisor website may not display properly.We support the following browsers:
Windows: Internet Explorer, Mozilla Firefox, Google Chrome. Mac: Safari.

Dodgy “1 review” reviewers - fact or fiction?

Helsinki, Finland
Level Contributor
73 posts
83 reviews
Save Topic
Dodgy “1 review” reviewers - fact or fiction?

I have recently been troubled by a case in which a hotel appeared to be “stuffing” its five-star reviews with anodyne four-line ‘thank you, thank you, everything was wonderful’ reports from people with one review to their name

.

More of that in a second.

First, a couple of product liability get-out clauses:

In the first place, I’m not suggesting, as some have earlier tried to suggest on these pages, that these “1 review” individuals are necessarily fakes, even though they do look and smell a bit funny.

Secondly, I’m not suggesting that “more is inevitably good” in terms of how many reviews a TA member can bring to the table. That is absurd. There are, I’m sure, plenty out there with a prolific review count who can barely string two words together, let alone give a sensible account of their stay or their meal.

Thirdly (sorry, having read what has happened to a few other thread-starters before me, I really DO have to get these “I’m not suggesting…“ things out of the way first), this particular case is not a matter of competitors putting up fake negative reviews to throw mud at other establishments, but a clear attempt to hoist the establishment up the all-important (well, to some people, at least) TA rankings of “#xx in Hotels in Toytown-on-Sea”.

It’s a hot-button discussion topic, clearly, and these threads usually generate flames and end up in acrimony and accusations that: “It's like trying to turn an oil-tanker/Trip Advisor doesn’t care/is only interested in the traffic and revenue/won’t go after the offenders” OR “You’re just being paranoid/people should read reviews intelligently/yeah, but EVERYBODY does it/99.95% of reviews are completely legit…”, etc, etc.

This made me reluctant to even voice my views here, but the numbers in question were rather compelling, and I very much doubt this particular hotel is a one-off example.

Here we go:

In order to check if my first hunch was exaggerated, a few days ago I looked more closely at the 120 most recent five-star reviews and the 70 most recent three-star reviews of the hotel in question.

This seemed a reasonable sample to take, roughly proportional to the total numbers of each through the hotel’s TA history, and covering pretty much the same time-frame.

Of the 120 ***** reviews, 108 were "1 Review". (and many also "Via Mobile"). This is 90%.

Incidentally, of the most recent fifty, 47 were in this category (94%), and of the most recent 100, 92 were single review posters (yep, 92% - maths is dead easy!).

In very stark contrast, of the 70 most recent *** reviews, just THREE were from single-review reviewers (roughly 4%).

ALL THIRTY of the 30 most recent three-star reviews were from multiple-review TA members.

So, 90+% noobs for the five-star reviews, and fewer than 5% for the 3-star reviews????

Given the numbers involved, I think it would be naive to suggest this imbalance is pure coincidence. I am, of course, open to your other explanations for the skewing, but I should warn you I will be hard to persuade that it is other than: “Hey, we’ll give you free this-or-that if you write us a nice TA review” (or some similar slightly dodgy receptionist gambit).

To return briefly to the “people should read reviews intelligently” counter-argument – YES, I quite agree, and I DO, and so do many others, but the point of such an exercise as described above is to gently shove the establishment up the ladder so it gets noticed by those users who think staying in hotel #8 is better than staying in hotel #88, and who cannot be bothered to find out WHY the two hotels are ranked where they are.

Just getting your hotel or restaurant “on the dance-floor" as far as organisations like TA is concerned may be as important as having reliable reviews. Begging for good reviews from noobs as they check out - or giving them a complimentary cocktail as a quid pro quo - is not exactly pernicious behaviour, but if you are a hotel or restaurant NOT doing this and you are watching the irresistible rise of one who does, you might well feel a bit peeved.

As to what TA can do about these sorts of things, that’s beyond me.

I don’t buy the “credit card details required” approach, as it plainly won’t work, and I’m also reluctant to have a weighting/handicapping system for “1 review” reviewers, although it might be worth noting that in places like eBay, vendors and buyers DO discriminate against people with minimal feedback. Amazon, too, allows users to sort reviews by their perceived worth: "5 of 6 people found the following review helpful".

Would it be quite impossible for TA to add a third means of sorting reviews: Rating, Date, and “Number of Reviews”? Or even a fourth, flagging up "Helpful Reviews"?

Many of us already subliminally use this technique, and I hardly imagine it would do any harm if TA made it that much easier for its users. Since it is possible to reward what seems like a good review with a "helpful" tick, it seems rather curious to me that there is no rapid mechanism for searching for such reviews...

Chester, United...
Level Contributor
61,453 posts
72 reviews
Save Reply
81. Re: Dodgy “1 review” reviewers - fact or fiction?

"I'm sorry, but this is so patently absurd I have to respond"

You may think that it is absurd but I speak form experience

"The two hotels I have looked at (and I have ONLY looked at two - so far...)"

Well I follow several different destinations and have looked at 100# properties over the years and as I say I speak of my personal observations and comments made by many regular experienced contributors

Helsinki, Finland
Level Contributor
73 posts
83 reviews
Save Reply
82. Re: Dodgy “1 review” reviewers - fact or fiction?

Good post, KVE1005. Wraps quite a lot of it up.

TA seems to want it both ways, though... They ARE trumpeting their popularity index as some kind of Asterix magic potion, and they are trumpeting their Certificates of Excellence and Traveller's Choice Awards, and hence one has to infer that they think "ranking" is important, even if we probably know ourselves it's only one aspect of a more comprehensive analysis of finding good places to stay or eat.

Whether you intended it or not, in your remark about "turning over multiple billions in revenue, and crazy amounts of members, I highly doubt they will change their methodology", I sensed also a reluctance on TA's part to rock the boat by suddenly causing some large and powerful establishments to slide down the rankings because the PI algorithms got tuned for the better.

That might scare the horses.

One thing... no, they shouldn't reveal the workings of the algorithms or whatever automagical things they use.

Think about it: as soon as they became public knowledge, every hotelier and restaurateur would instantly know how to milk the system.

At least now they aren't TOTALLY sure how it can be gerrymandered.

Helsinki, Finland
Level Contributor
73 posts
83 reviews
Save Reply
83. Re: Dodgy “1 review” reviewers - fact or fiction?

Alright, flip-flop_man, how do YOU explain the clear anomaly in these two establishments you have before you?

THAT was the absurdity we are dealing with and that I was addressing. It was not a personal attack, I assure you.

Has the system suddenly failed here? Have the two hoteliers - clever scheming Moriartys that they are, found the TA weak spot, the soft underbelly, and are they the only hoteliers in the world able to do this?

I don't think so.

The reviews speak for themselves. Enough anodyne "excellents", gathered at check out, will quite clearly - the evidence is indisputable - push an establishment into a place that it manifestly does not deserve to be in, at least if we are to pay any mind - as TA does - to the "quality" of reviews.

I'm at a loss here. To my eye, the ranking system (for what it's worth, which may not be much), is being royally creamed. If one can do it, anyone can.

How come nobody sees it?

Pittsburgh...
Level Contributor
6,421 posts
78 reviews
Save Reply
84. Re: Dodgy “1 review” reviewers - fact or fiction?

Nobody "sees it" because, as several of us have pointed out, it seems to work FOR THE MOST PART. It may not be perfect, but generally speaking, TA's ratings make sense from a customer satisfaction standpoint. Perhaps there are tweaks that can be made, but I don't support the "solutions" I've seen so far.

I know how I use TA, and as such, membership fees don't work for me. I like to contribute and want my reviews to mean something, so setting up the "Welcome to TripAdvisor where your first review doesn't count" scenario doesn't work for me either. Same goes for the already failed credit card verification.

To answer the inevitable, "Well, what do YOU propose we do about this horrific problem?" I'm not making a proposal at this point. I'm not seeing the looming disaster on the horizon, so I'm not ready to commence the "shock and awe" offensive. When my (here I go with it) personal experience with TA dictates otherwise, I'm sure I'll change my mind.

Melbourne, Australia
Destination Expert
for Sydney, Bargain Travel, Food and Travel
Level Contributor
26,669 posts
65 reviews
Save Reply
85. Re: Dodgy “1 review” reviewers - fact or fiction?

I agree EC.

Although there maybe a few looming disasters on the horizon for those who depend entirely on the PI, and not bother reading review content.

Pittsburgh...
Level Contributor
6,421 posts
78 reviews
Save Reply
86. Re: Dodgy “1 review” reviewers - fact or fiction?

I'd say that goes both for businesses and travelers. I don't believe for a moment that this one travel site will tank your business if you drop a place or two. At some point, this site (and review sites in general) will have been around long enough that people will start to understand that the rank is meaningless.

As far as the ratings and rankings are concerned, I still believe that everything will eventually come out in the wash. A crap hole can't just fake its way to the top. Even if it could, it couldn't stay there. Eventually, people are going to figure it out and they'll lose business. Likewise, the consistently good properties will retain positive reviews and stay near the top end of the rankings. The truth doesn't hide forever.

Wales, United...
Destination Expert
for Bargain Travel, Cruises, Swansea, Cardiff, Carmarthenshire, Neath, Port Talbot
Level Contributor
64,688 posts
451 reviews
Save Reply
87. Re: Dodgy “1 review” reviewers - fact or fiction?

I don't care what they do to the algorithm. I take no notice of the PI when I'm looking for somewhere to stay or eat, and I will write the same reviews no matter what effect it will have on the rankings, which would be very little in most cases anyway, I suspect.

No review site can hope to achieve perfection, and I agree that, for the most part, this one works well enough, at least for my purposes, as it is.

The only thing I would urge TA to do is to alter it's descriptions of what the popularity index is, and remove all reference to quality,. Though, in fairness, my own interpretation of this is that TA means that if someone has a low quality experience they are not going to give an establishment 3 or 5 stars, I don't think the intention is to infer that we are all rating quality as a hotel inspector would, but I could be wrong. They just need to clarify it a little better than they currently do.

88. Re: Dodgy “1 review” reviewers - fact or fiction?

-:- Message from TripAdvisor staff -:-

This post was determined to be inappropriate by the TripAdvisor community and has been removed.

To review the TripAdvisor Forums Posting Guidelines, please follow this link: http://www.tripadvisor.com/pages/forums_posting_guidelines.html

Our staff may also remove posts that do not follow our posting guidelines, and we reserve the right to remove any post for any reason. Thanks for being a part of the TripAdvisor travel community!

Removed on: 14 August 2014, 13:43
Pittsburgh...
Level Contributor
6,421 posts
78 reviews
Save Reply
89. Re: Dodgy “1 review” reviewers - fact or fiction?

With respect, we all "get it." We just have a different opinion than you. You'll probably notice that many of us are regular contributors on TA forums. Might I ask that you use your own logic (more reviews=more reliable) and recognize that opposition to your idea isn't born of ignorance, but (dare I say) personal experience on TA.

I've supported possibly TWEAKING the PI. I just don't support nullifying first reviews, as again, I've read many first reviews that have been more informative than some 100th reviews. I get your idea on the helpful vote thing, but just remember that people (myself included) often forget to click that button because we're reading many reviews on many properties. I look at my own reviews where I've gotten 65 helpfuls on 58 reviews. That's a good average, I think, but that's not the point. I've got one with 9 helpful votes, but a good chunk of them have none. In my opinion, some of my better reviews are ones that haven't gotten a single vote. I don't think those reviews should be stuffed to the bottom or rendered meaningless in the PI just because of the very unpredictable "helpful" button.

I would support TA taking a closer look at the reviews you've questioned. They do seem to be an anomaly. However, I strongly contend (based once again on my experience) that these are in fact anomalies and not the norm. I'd rather TA spend the time and resources addressing that than blowing up something that has generally worked to this point.

Helsinki, Finland
Level Contributor
73 posts
83 reviews
Save Reply
90. Re: Dodgy “1 review” reviewers - fact or fiction?

E-C:

As I've said, I'm done here, and I have passed it on to Newton, to see what - if anything - they make of it.

I would point out, nevertheless, that I have not been advocating "nullifying first reviews". They can be perfectly reliable, and - as I have also noted - ALL of us wrote a first review once. If you've looked at the examples I've shown you, however, this is regrettably not the case here. And if I didn't have a life, a job, and grandkids to play with, I assure you I'd get down to some heavy open-data data-mining and I'm confident I would find the same patterns as in Jerusalem repeated all over the place, even in Pittsburgh. :) :)

I think the "anomaly" is bigger than many would like to believe.

Just as I agree that a first review can contain more pertinent and objective information than a hundred by someone wearing a gaily-coloured Top Contributor beanie, I'd venture to suggest cheekily that mere volume of forum postings should not necessarily be taken as providing oracular skills.

I was not - and I have repeatedly stated this throughout this thread - suggesting the "1 review" entries be removed from the pack, but merely arguing that since the Trip Advisor Popularity Index demonstrably contains a methodology-hole large enough to drive a London double-decker bus through it if you are a scruples-challenged (and very very shrewd) hotelier, then perhaps "review quality" might be given more serious attention.

I feel fairly secure that that one valuable review by a first-timer WILL get noticed and will get some helpful ticks. No, it might not get noticed as much as it SHOULD, but it will certainly get more TLC than the examples I have been complaining about. Consequently under a system where "some kind of invisible** weighting for usefulness" was involved, the idea of mass producing bland "excellent" reviews at check-out would not be so attractive.

** Invisible in the sense that TA would be stark staring mad to actually admit they've done it. The same chancers who are playing the system now and (in accordance with TA's own suggestions, please note) soliciting short positive reviews from happy-faced customers would make damned sure that these same short positive reviews would get bucketloads of useful votes.

One MIGHT argue that adding this "quality" filter (allegedly already there, but this thread is evidence it does not work) is only going to kick the can down the road a bit, because hoteliers will wise up (see above). Nevertheless, it would at least mean they have more work to do to fiddle the ranking numbers, and possibly more ways of getting caught red-handed doing it.