Amazon can’t end fake reviews, but its new system might drown them out

https://www.vox.com/recode/2020/2/14/21121209/amazon-fake-reviews-one-tap-star-ratings-seller-feedback

by pabo • 6 years ago

114 140 6 years ago

AlexandrB• 6 years ago

In my experience, making reviews easier to give causes the review quality and usefulness to go down. This happened when Netflix went from 5 stars to a simple like/dislike. I’m not sure why Amazon didn’t just block non-verified-purchaser reviews, increasing spammer costs significantly.

I suspect that Amazon reviews are going to be even less useful now. Especially given things like this:

Amazon does not provide many specifics about how a product’s overall star rating is calculated, other than stating that it is not a simple average but instead uses “machine-learned models” that take into account factors such as how recent the rating or review is and whether it was a verified purchase or not.

awinder• 6 years ago

I'm pretty sure that a ton of the review fraud on amazon comes through verified purchases. You refund/pay people to buy the product on amazon and leave a review in the best case, in the worst case you operate accounts buying your own stuff, and then flow the inventory back and basically pay an amazon tax for leaving good reviews.

IMO this is pretty solvable by looking at an account's purchase history too, but I don't think it's just as simple as blocking non-verified-purchase reviews.

outlace• 6 years ago

As long as there is some, even small, cost to posting a review it makes review fraud much less scalable compared to zero-cost reviewing.

figgyc• 6 years ago

Perhaps they could limit refunded products to 3 star or below? Really there isn't a good reason to positively review something and take it back, if it was your fault eg if it didn't fit, then you could just not review it.

Nasrudith• 6 years ago

I suspect solving it is hard to impossible. You can heuristic anything as suspicious at a glance but moderation at scale fails. You personally can decide "this review is bullshit" and block without issue but start flagging in a false positive or even true positive by a manipulator looking to start shit could cause considerable backlash as a futile attempt at pleasing everyone is made. Transparent and consistent rules invite gaming and opaque ones invite accusations of malfeasance and the uncertainty promotes bad feelings and bad behavior.

Worse still is that even taking a stance to not take a stance because you know it cannot be done will bring backlash as there are many who demand you take their stance, even allowing easy distribution of self chosen block and filter lists are not enough. There are many who demand the appliance experience.

freeAgent• 6 years ago

Yes, I've seen articles where people who do this say there are a bunch of Facebook groups for exactly this purpose. Manufacturers/sellers ask people to buy their goods on the group and write good reviews. Once they get the proof of the purchase/review, they Paypal (or whatever) money to the "reviewer." I remember an article I read where some woman's house was overflowing with junk she didn't want because she was writing so many of these fake reviews. I would think Amazon could apply their awesome machine learning to figure out which accounts are pumping out these suspect reviews and perhaps de-prioritizing and maybe even rate limiting and delaying publishing of their reviews. I'd think that would do a lot to alleviate the problem.

sct202• 6 years ago

There's also low scale review fraud like my sister bought a pet cam and gave it a 3 star review, and the seller started to offer escalating offers for her to remove or increase the rating from full refund, another camera, refund + free camera, and refund + money for the inconvenience.

groby_b• 6 years ago

And while you're dreaming that purchase history solves that, the scam world has long moved on.

Currently, companies pay people to buy items. They can keep those items, they just need to leave a good review. There are intermediaries who handle lots of sellers, so people buy a mixed bag of random garbage in exchange for the occasional review.

Yes, you can probably test for statistical anomalies, but I'm willing to bet that's quickly countered too - just have people buy occasional legit items so their profile is "statistically normal".

As far as I can tell, Amazon tries to fight that by keeping their ML model secret so scammers don't learn too quickly, but essentially, they're currently finding out what the Internet learned about SEOs manipulating search results.

giancarlostoro2• 6 years ago

I rather look at the content of the reviews than their ratings, there's been times where someone left what looks like an honest review and it's 4 / 5 but they only write positive things. I don't know what the solution is, but I definitely think more thorough reviews would help.

If you return a bad product, try and get a replacement to see if it was just bad luck, but also please do write a detailed review.

If you get a good product and it barely has reviews please review it.

CWuestefeld• 6 years ago

someone left what looks like an honest review and it's 4 / 5 but they only write positive things

That may well be me. I don't think that "meets requirements" should merit a 5. When I do reviews, I view a bare "meets requirements" as 3 or maybe 4, depending on the type of product. I want to leave some headroom to be able to point out products that really do excel.

Another ambiguity is whether the rating is on an absolute scale, or normalized for value (it's not a perfect product, but it's super-cheap).

throwaway55554• 6 years ago

I’m not sure why Amazon didn’t just block non-verified-purchaser reviews, increasing spammer costs significantly.

I'm not sure this would help much. I've been asked by vendors to leave a review and if it's 5 stars, I will get something in return. I think for some people this would be a "well, nobody is getting hurt, right?" decision and they'd just do it.

TwoHeadedBeast2• 6 years ago

Amazon should provide an incentive for reporting this sort of thing.

thiagomgd• 6 years ago

I think that on Netflix's case they are not going for reviews. They want to know what you like, to suggest something you'll probably like too. Doesn't matter if it's 4 or 5 stars.

Seenso• 6 years ago

I’m not sure why Amazon didn’t just block non-verified-purchaser reviews, increasing spammer costs significantly.

I don't think it'd do much good. There are already large groups that subsidize verified purchases to get 5 star reviews. I read an article about it, confirmed some investigation on Facebook of on my own.

Basically someone runs a Facebook group where sellers advertise free product, promising Paypal reimbursement of the purchase price in exchange for reviews. There are at least (or were, I haven't checked recently) hundreds of Facebook groups across many languages with thousands of members each doing these activities. The reviewers are randos who like free stuff, and I don't think anyone could detect them if they only casually participate in the review scams.

agumonkey• 6 years ago

making reviews easier to give causes the review quality and usefulness to go down

Allow me to generalize

making X easier causes the quality and usefulness to go down

ebg13• 6 years ago

“For brands, this means the black-hat review clubs and sellers will have less impact, as fake reviews as a percentage of legit reviews should decrease.”

Well, at least they're thinking happy thoughts.

luckylion3• 6 years ago

Amazon is competing against the fake reviews with their own Vine club.

I've checked dozens of their verified & official "testers", and their five-star-rate averaged somewhere north of 98% which makes their reviews just as useless as the cheap auto-translated fakes to me.

FakespotCom3• 6 years ago

And from our internal research at Fakespot, we concur. The positivity bias from influenced reviews such as the Vine review program and multitudes others is rampant.

Our ML based algorithms will penalize any biased reviews and show case them to our users.

MichaelApproved• 6 years ago

The new one-tap feature asks customers to select from one to five stars for a product. It’s only available to customers who have actually purchased the item from Amazon — “verified” buyers. That barrier alone creates one hurdle that will make the new rating system harder to game, since Amazon does allow written reviews from non-verified buyers.

I understand that Amazon allows non-verified buyers to leave a review, so they can have more product reviews but those non-verified reviews are the source of so much garbage.

Sure, let people post non-verified reviews, so you look like you have a lot of reviews but at least let me filter them out in my search.

Filter: 4+ stars from only verified reviews.

Hell, even 3+ stars from verified reviews would be helpful.

I know sellers can fake verified reviews by paying the consumer back for the product but that's a massively higher barrier for the manufacturer. The difference is a few cents per review VS a few dollars per review.

I've seen products with thousands of fake non-verified reviews. I'm pretty sure they couldn't afford to do the same with verified reviews.

beerandt• 6 years ago

Except negative non-verified reviews tend to be helpful, especially for sellers that constantly relist the same product under different listings.

I've bought a few products that are bad to the level of fraudulent, only to see the product relisted so that my review wouldn't be verified.

Off course this could be abused by leaving fake negative reviews for competitors.

Whack mole. Repeat.

heartbeats3• 6 years ago

What about making them stick, somehow?

If I leave a negative review for a product that later gets delisted, Amazon could flag the account on basis of "deleted product with poor reviews". It could then try and see if there are any substantially similar items listed a week on. If so, ban them.

Perhaps even easier, make verified negative reviews stick to the seller's account. If you delist a product with negative reviews, those just transfer over to the seller, so you see a 5-star product sold by a 1-star seller.

tobtoh2• 6 years ago

The (edit: average) Amazon star rating has been such a poor indication of the quality of a product for a very long time - at best, it's a weak indicator of which products you might check out first.

For me personally, the most valuable bit of feedback are the negative (edit: 1 and 2-star written) reviews - they are pretty much the only review content I trust. I'm looking for patterns of issues that multiple reviewers raise about the product.

The positive reviews have so little value when anyone can post a review. So many shallow positive reviews from unverified 'buyers'.

casion3• 6 years ago

I'm not sure I agree with this. I always look for and at the 1 start reviews, and I find that to be more useful than reviews I find anywhere else.

4/5 star reviews are useless, but 2/1 star reviews reveal a great deal of useful information about a product.

A bunch of 1 star reviews of "UPS damaged my product" indicates a product that's as advertised and isn't astroturfed (much?).

No 1 star reviews, or 1 star reviews indicating misleading advertising tend to indicate astroturing.

1 star reviews indicating product failures, support failures, DOA etc... indicate to not purchase the product at all.

It's not a pure indicator, but I've found it to be more reliable than other review systems.

Shivetya• 6 years ago

plus with some of the low star reviews you can compare what they stated was wrong with your own expectations of what could go wrong with the product.

the only two considerations for me are the number of reviews and the quality of the low star reviews for the same. the dates of reviews is very useful as well, if a product doesn't have many recent good reviews it can offset the number of reviews in my view

obelos• 6 years ago

A useful heuristic that I also use, but it's good to be aware that fake reviews go in the other direction, too, of sabotaging competitors' product reviews.

0xffff2• 6 years ago

Am I crazy, or does the entirety of your comment except for the first sentence agree with the person you replied to?

tobtoh2• 6 years ago

I think we are in agreement - I didn't word my comment well.

When I was referring to the star rating, I meant the overall average rating of the product (ie what Amazon is trying to promote now over star rating plus written reviews).

When I mentioned the negative reviews, I was referring to the 1 and 2-star written reviews. I use the negative written reviews exactly how you described - and it's what I find valuable.

snarf21• 6 years ago

The downside of this approach is for sellers to use this as a weapon. Pay people to buy a competitor's product and give it a nasty 1 star review. It becomes an arms race.

It is also super annoying when there are reviews like "came in three days not two. 1 STAR!!!" that are not product related. This doesn't even get into the commingling of product which is the real problem.

throwaway27862• 6 years ago

The smart sellers realize this as well. Often the most "useful" review is also fake. Let's just create a 2 or 3 star review and list out the pros and (fake) cons and get people to upvote it.

Source: I'm an Amazon seller.

olah_12• 6 years ago

When you review books on amazon, are you reviewing the content or the edition that you purchased (paper, typesetting, cover, etc)?

I wish there were two separate ratings.

martin_a3• 6 years ago

I think this a constant problem with these type of rating systems.

We have also implemented one of the leading feedback-survey-customer happiness-system-things in one of our shops. The amount of negative ratings, because the shipping service provider fucked something up, is immense.

There is no way for us to really get those removed (integrity of the platform - good thing in theory) and commenthing something like "it wasn't our fault" doesn't really help either as the rating will influence the total rating one way or another.

Maybe that's cost of business, not sure.

JohnFen8• 6 years ago

The other problem with reviews is that they seem to be scaled according to price. A garbage product that sells cheap often gets reviewed as highly as genuinely great product that is more expensive.

tumblen3• 6 years ago

Yep - and a lot of reviews are even people complaining about shipping which really is irrelevant (unless it's a packaging complaint about the product itself).

jasode• 6 years ago

But the new rating system isn’t fool-proof, since some fake-review schemers have a way to get around the “verified purchase” requirement. One popular method is to recruit buyers in private Facebook groups with a promise to refund them for their purchase via PayPal after the shoppers show proof of writing a five-star review.

I heard similar gaming techniques are used by makers of diet supplements to build up 5-star reviews. They notify customers that they will send another free bottle of supplements of they leave a 5-star review. This means that the following amazon search results example showing showing overwhelming 5-star ratings are very likely fabricated and can't be trusted:

https://www.amazon.com/s?k=lion%27s+mane+supplement

This is how you get "snake oil" products like noncompliant USB cables or defective Apple chargers into the "5-star" club that amazon buyers unwittingly trust.

seunosewa• 6 years ago

Amazon should strongly incentivize their customers to report sellers who make such offers to them. This should make the game more dangerous for sellers seeking fake reviews.

martin_a3• 6 years ago

If I'm somewhat "vulnerable" to "free stuff", this will not work. Couldn't care less about Amazons problems with fake reviews if I get something for free (that I kind of like).

mrec• 6 years ago

Doesn't that just shift the game to joe jobs against the competition? There's no reason the offerers need to be closely/obviously tied to the seller. If I'm flogging Foo brand widgets and you're flogging Bar brand widgets, I offer $$$ in exchange for 5-star Bar reviews, report it and bingo, you get a kicking from Amazon.

alok-g• 6 years ago

Any such incentive by itself be prone to abuse.

the_duke• 6 years ago

Amazon reviews are about as trustworthy as the selling points of a slimy salesman.

I've bought products with a lot of 5-star ratings which were utter garbage.

Even if you only consider "Verified Purchase" reviews, every one of them requires careful vetting for authenticity.

There are companies providing quality fake reviews by native speakers as a service, where care is taken to not be overly optimistic and provide a few negatives to sound more believable.

Even negative reviews might be fake, often mentioning a competing product that is supposed to be better.

There are plenty of valid reviews out there, but the need to constantly question the authenticity makes them almost worthless to me.

I've mostly gone back to buying from medium sized local retailers. They might have a much smaller selection and are a bit more expensive, but make up for it with a decent quality control.

coleca• 6 years ago

It would be a shame if the reviews went away though. There are tons of garbage reviews out there, but I still find some gems out there.

Recently, I was in the market for a paint sprayer and one reviewer wrote this crazy detailed review of the sprayer he purchased, what type of paint he used to paint his cabinets, why this paint worked vs. the other ones he tried, and included numerous pictures of the process and results. It was super helpful not only in making my selection, but ultimately completing my project.

I also have found a bunch of instances where people will post tips in the reviews about how to fix or modify a product to make it perform better (or sometimes unfortunately just perform to its advertised specs). In one case I was able to repair my own refrigerator with a part purchased on Amazon and the instructions found in a review. While I'd rather have purchased it locally, that would have meant a service call and spending half the cost of the unit to get it repaired vs. a hundred or so dollars for the new circuit board from Amazon.

olah_12• 6 years ago

The most useful reviews will always be ones from within your own social network. Because if you have a relationship with the person, they won’t endorse something unless it reflects well on themselves.

This is why I’m bullish on protocols like SSB[1] and Iris[2].

[1] https://handbook.scuttlebutt.nz/

[2] https://hackernoon.com/what-is-wrong-with-the-internet-and-h...

krisroadruck• 6 years ago

I wonder what the average account age and purchase total is of genuine amazon accounts. Seems like setting some sort of age and purchase threshold to reviews could make the fake review game a lot more difficult.

heartbeats3• 6 years ago

One approach might be to shadowban fake accounts' reviews. They still show up as text, but they don't get counted in the average. If Amazon at T = 0 know which reviews are legitimate and which aren't, someone who consistently gives five-star reviews to shit products could have their ratings lowered in importance.

You could go all the way with this: look at my reviews, and then only show me reviews made by people with similar tastes.

julianlam3• 6 years ago

A good idea. I'd wager that a high proportion of users who join these "rating review" schemes are accounts that rate 90+% of their items as 5 stars.

If you check my Amazon account, I don't even rate most of my purchases.

It's a rather low sensitivity metric but one that has some value.

ahpearce• 6 years ago

I did a quick Google to see if anyone was trying to solve this independently, and this looks pretty neat: https://thereviewindex.com/ ... Anyone have experience with it?

brlewis2• 6 years ago

See reviewmeta thread https://news.ycombinator.com/item?id=22327272

izzydata• 6 years ago

At what point does the poor experience of Amazon outweigh its convenience and cause people to start shopping in physical stores again?

Or would people shift to other online retailers?

DarkTree• 6 years ago

I think this parallels really well with the question of how much privacy people are willing to give up for the convenience of digital services. In both cases, it seems like we are increasingly willing to give up privacy and trust in the name of convenience, but I do wonder if there is a breaking point. If there is a breaking point, is it even a system that can be reversed?

tumblen3• 6 years ago

Amazon has been more expensive lately and most other stores have free-shipping now too. I've definitely found myself being much more open to shopping around and purchasing elsewhere than I was 4-5 years ago.

davio• 6 years ago

Amazon is probably in excess of 99% perfect service for me. It feels like a step back in time when I order from other retailers.

kop316• 6 years ago

I haven't ordered from amazon in over a year. I to go to physical stores and/or order from a reputable vendor online.

Actually it's nice. If I truly want it, I'll actually go and get it, or spend the time to find a reputable vendor. The added friction ensures I don't buy random things that I thought I wanted but don't.

GaryNumanVevo• 6 years ago

For anything outdoor related, climbing, hiking, cycling. I'll always go to REI and talk with a sales person there. REI seems to hire people who have actually used the equipment they sell, I don't mind paying a small markup for the experience.

JohnFen8• 6 years ago

Amazon reached that point for me a number of years ago. Since Amazon reviews are largely worthless, I don't use them to decide whether or not to buy a product. I do read them, though, because they often contain useful information about specific issues and usage tips.

Amazon is fourth in my list of where I buy things from. First is a local physical store. If I can't find what I need there, then I seek out manufacturers websites, or websites of authorized retailers. If I can't find what I need there, then I check out eBay. If that doesn't work, then it's Amazon.

This changed over the last year or so, though -- before, I would go to Amazon before eBay.

isufud• 6 years ago

Meh, I prefer having some real reviews in an ocean of fake ones than having none at all.

Magnets• 6 years ago

I've not ordered from Amazon for 3+ years now. You can buy the same products at the same price elsewhere or generally always from ebay.

The only real advantage Amazon have is prime (if you pay for it) and simple returns, but you have to deal with a drone if anything goes wrong aside from that. Most other retailers have actual people that can make decisions answering queries

cik• 6 years ago

Doesn't everyone just use reviewmeta (https://reviewmeta.com/) now? They're pretty solid at what they do, the results appear to be rather accurate - and at the very least it's a decent filter.

ceejayoz• 6 years ago

I use Fakespot.

I think it's only a matter of time before these sites get coopted/bribed by vendors, in the same way a lot of adblockers have been.

FakespotCom3• 6 years ago

Hi, Saoud from Fakespot here. My co-founders and I all got scammed and duped by fake reviews and Fakespot was built to bring back trust and transparency to the e-commerce experience (and others).

We have a strict policy to remain neutral in our analyses and we do not in any way, shape or form accept bribes or copt partnerships.

anonsivalley6522• 6 years ago

I use it too. It's almost always the no-brand import products that add fake reviews/solicit phony glowing reviews. Most big brands have too much to lose to add fake reviews because it would be PR-damaging news.

Even so, you have to consider real reviews to be only partially-reliable because a lot of people use their products incorrectly and give excessively glowing (or negative) reviews while ignoring reality because they may feel socially-obligated to not say something negative. And then there are hundreds of millions of specialized SKUs without any reviews at all.

ilamont• 6 years ago

Early adopters use such sites. A significant portion of Amazon users may not even be aware there's a problem with fake reviews.

tumblen3• 6 years ago

Do any of these sites have an extension that can rank individual reviews for trust-worthiness based on the content + other reviews by that user?

It's always a bit of a schlep to have to remember to use another website to check review credibility.

arafa• 6 years ago

The review situation is bad enough on Amazon now that I won't buy any medium to high value product without using ReviewMeta. I find their service to be invaluable. Reading reviews and using ReviewMeta still isn't enough, but it's reasonable and much better than not using them.

JohnFen8• 6 years ago

I'd never heard of reviewmeta until I read your comment. I'll check it out. Thanks!

subroutine• 6 years ago

To me a bigger problem than fake reviews is the rampant bait-and-switch. Way too often I will see a product with a 4 or 5 star rating, and find the reviews are talking about something completely different.

That, or there will be 10 different products nested under the same identifier. Not just different colors, but different products. So I have no idea which one the person it praising or calling crap. Maybe instead of Avatars next to the reviewer's name, Amazon should just attach an expandable screencap of the product page at the time the "verified purchase" was made.

topicseed• 6 years ago

Sadly, Amazon and Google reviews are mostly ignored when I am looking for a product, or service. It's so evidently clearly gamed. And Amazon's results pages with 5 of the very same product just with a different sticker/brand is also silly.

Reminds me of SEO's early days with keyword stuffing.

JohnFen8• 6 years ago

This may help a bit with fake reviews, but it amplifies the other substantial problem -- authentic, but untrustworthy reviews. I see two type of those all the time:

1) Reviews by people who have not used the product. The tipoff for this is when the customer says something like "I just got this, it looks great! I can't wait to use it." If someone hasn't even used it, then they can't possibly give a useful review of any sort.

2) Reviews in product listings that contain multiple products, or products that have changed since the original listing. The tipoff for this is comments describing a product that is clearly not the one being listed, or is one of several different products in the same listing.

If there is an increase in reviews that are just stars without comment, it becomes impossible to root these out at all. I don't see how this would lead to more reliable reviews.

tartoran• 6 years ago

Perhaps limiting the number of reviews a user can make could mitigate this somehow?

stebann• 6 years ago

What about the trade offs between privacy and the possibility of being part of a community? I mean, you have some way to identify core characteristics of the subjects whom make the reviews, and if you can't do that, how can you trust them?

throwaway27862• 6 years ago

So here's some perspective from an Amazon seller doing 7 figures+ annually.

- On average, only 1-3% of customers review products.

- Each review is worth a lot of money, often times multiples of the product itself, and especially if you're just starting out.

- Each category in Amazon has it's own Average rating, for example, electronics typically have lower ratings because more things can go wrong and there are more usability issues vs something like kitchenware, where less things fail outright.

- If you play in a category with a certain failure rate, it is absolutely essential that you do everything you can to mitigate bad reviews as enough of them will sink your business, even if you have a great product.

- It takes 8+ 5 star reviews to counteract a 1 star review if you want to maintain a 4.5 star average which is the bar for a good product. This is extremely hard to do without manipulation.

- People who complain about fake reviews are only seeing half the problem, the other half is that legit businesses who do it the fair way can't compete. How do you launch a great product on Amazon with 0 reviews? Hope that 500 people buy it to maybe get 5 reviews? Alternatively you spend thousands on product ads hoping that enough people buy... or just succumb to the dark side and pay for reviews which is WAY cheaper.

- If you hate Amazon reviews, do your part and start reviewing the good products on Amazon. It is worth more to the seller than you think!

borkt• 6 years ago

Where do people find paid reviewers for amazon? Seems like something they could target to close pretty easily

wyxuan• 6 years ago

Facebook groups

petrocrat• 6 years ago

Upwork, fiverr, freelancer also have jobs posted which are for fake reviews, even though the job description is cleverly disguised as a a blog post or some other innocent thing.

beauzero• 6 years ago

The exception to the rule is books.

vinniejames• 6 years ago

Yes, the exception is books are even worse:

https://thehustle.co/underground-world-of-kindle-ebooks

https://thehustle.co/part-2-confessions-from-the-scammy-unde...

https://thehustle.co/part-3-confessions-from-the-scammy-unde...

lifeisstillgood2• 6 years ago

It seems that Amazon stores (or some franchise) could act as a good proxy for launching / improving these products - imagine running a store, and lining up products to be "test driven" in store, with all the drivers licenses and check ups one might want. I can happily imagine trying out a fairly wide range of electronics - others may prefer shoes on Thursday or Fly traps on Friday.

TBH, retail is supposed to becoming "experience" based so this might be a runner. If someone tells me Jeff's mobile number I will persuade him.

ABeeSea• 6 years ago

They sort of do that with their launchpad program.

https://www.amazon.com/launchpad/startups/benefits

amelius• 6 years ago

If you hate Amazon reviews, do your part and start reviewing the good products on Amazon. It is worth more to the seller than you think!

And who will own my reviews? I'd rather put my reviews on a more open review system.

brlewis2• 6 years ago

I don't think there's anything stopping you from putting the same review in multiple systems.

lovemenot• 6 years ago

Is there a system in widespread use which allows a user to accumulate reviews they have made across multiple platforms?

I may be able to trust a reviewer on product A if their opinions on B,C and D were like my own.

cbd7f73• 6 years ago

Reviews are worth a lot to sellers.

So much so that if you leave a 1-star review, there's a good chance you'll eventually get an email asking you to remove the 1-star review in exchange for a refund, a giftcard worth the price of the product, etc.

It encourages bad behavior though; You can basically gamble on what sellers will send you that email and thus give you the product for free.

JohnFen8• 6 years ago

there's a good chance you'll eventually get an email asking you to remove the 1-star review in exchange

Wow. I've never given an Amazon product a 1 star review, but if I did, and I got such an email, I would absolutely update my review to warn everyone that this is happening, and that the real average review for the product is likely to be lower than is shown.

thewebcount• 6 years ago

I once bought an O-ring for my blender. It's literally a 1cm-wide circle of rubber with a diameter of about 2 inches. It has 1 job - keep liquid from leaking out the bottom of the blender.

The day after I received it, the company sent me an email asking me to please please please rate my new product!

Fine! I gave it 3 stars and wrote, "It's an O-ring. It does exactly what I expected it to and nothing more."

The next day I get an email saying, "We see you gave our product a 3-star review. What can we do to improve our product? What didn't you like about our product?"

WTF? It's a goddamn O-ring! There's nothing to review beyond "it works" or "it doesn't." What the heck do you want me to say about it? No O-ring is ever going to be 5 stars. Sorry! That's just the nature of the product.

At this point, it's just harassment. Stop begging for my approval, and especially when I give you my opinion, please don't question me about why. I explained it in the review.

Wowfunhappy4• 6 years ago

Here's my problem with leaving reviews:

To me, a five star review means that a product went above and beyond my expectations in some extraordinary way. I have bought products that fit that description, but not many.

Everything else, I'd give either 4 stars or a still-very-satisfactory 3 stars.

The problem is, I know these 4-and-3 star reviews actually hurt sellers, which isn't my intention at all. So I just don't leave feedback.

This is also why I don't rate Uber drivers.

TwoHeadedBeast2• 6 years ago

I totally agree. For five stars it would have to be exceptionally good. Something I'm very satisfied I might give four stars, which ironically drags the score down.

I recently purchased an alarm clock. When I put the clock in my bedroom, the light from the display was so bright it kept me awake. I went back to the item on Amazon and read all the reviews. One person mentioned the same issue yet still gave it four stars!

JohnFen8• 6 years ago

This is also why I don't rate Uber drivers.

It was here on HN a number of months ago that I learned that in the Uber world, 3 stars does not, in fact, mean "acceptable". That knowledge altered my view of all reviews, including Amazon's, in two ways:

1) Like you, it means that I'm no longer willing to give reviews/ratings. If there is no consensus on what the different numbers of stars mean, then I can't be at all comfortable that my rating will indicate to others what I intended to indicate.

2) It means that I no longer put any weight whatsoever on ratings I see from others, for the exact same reason: I can't know that what I think the rating means is at all what the rater intended it to mean.

That 3 and 4 star ratings hurt Amazon sellers underlines this problem.

crankylinuxuser• 6 years ago

Thats why I turn my rating into a 1(lowest) or 5(highest).

I binary-ize any score that isn't already a yes/no, because thats what every score turns into.

trjordan• 6 years ago

Do you buy products with 3-star reviews?

the84722• 6 years ago

The score inflation has been plaguing many areas. this helpful chart is many years old already. It's probably a consequence of goodhart's law.

https://imgur.com/A1umWh2

Gustomaximus• 6 years ago

Knowing how the system works and how 5 stars are expected, the way I look at Uber rating is all drivers start with 5 stars. A good enough journey gets that. But stars will be taken off if things like the car is dirty or they drive dangerously etc. It starts at 5 starts, not has to earn them.

hbosch• 6 years ago

I get why this is, but’s it’s really never been this way. Even on early eBay, if a transaction simply occurred without errors it was pure etiquette to rate them a full 5 with a comment along the lines of “A++++ SELLER! SHIPPED ON TIME, GOT EXACTLY WHAT WAS LISTED!”

Really, same thing goes for Amazon and Uber, etc. the etiquette is to start at a full rating by default and deduct based on what goes wrong. You can’t “earn” extra stars... if I order a box of batteries, and they arrive on time and the box is full, that’s a perfect transaction. 5/5, no problem. You can’t realistically expect extra batteries, or for the batteries to perfect above their rating. They’re batteries.

pavelrub• 6 years ago

Not sure I understand your problem. What prevents you from simply changing your interpretation of review scores so that it will better fit the expectations of others, instead of avoiding leaving helpful reviews altogether?

clairity• 6 years ago

i believe that's the reasoning for why many systems have devolved to boolean choices rather than multi-tier (distortions in voting cause mispresentations of the customer population's sentiments).

but boolean choices are really 3-tier systems where it's assumed that only the extremely statisfied or dissatisfied customer will vote (up or down), and the lukewarm/indifferent customer is assumed not to vote. however, that assumption very likely misrepresents the sentiments of the (majority) non-voters and thus the population as a whole.

you might address this by moving to a 3-tier system: (1) unsatisfied/bad, (2) acceptable/fine, (3) exceeded expectations/great to more accurately differentiate the non-voting/indifferent customers, but non-voters would have no incentive to suddenly voice their opinions and make the system more accurate.

you might be able to counteract that impulse by incentivizing customers to vote on every product/service delivery event (like earning points for future discounts) to lower response bias. you could also do a separate study to see how the voter/non-voter population differ, and adjust the boolean ratings accordingly.

in any case, rating systems are tricky.

Johnny555• 6 years ago

or just succumb to the dark side and pay for reviews which is WAY cheaper.

I thought this was what the Amazon Vine program was for -- a legitimate way for retailers to provide free products to reviewers in return for a review, and those reviews are clearly labeled as such.

Seems to work pretty well - I've bought some products that only had Vine reviews, and I've agreed with the reviewer.

dylz• 6 years ago

Crap companies don't want Vine reviews, they want "real" fake reviews without the "I received this for free" marker.

Legitimate ones go for Vine that would have probably kinda earned it anyway.

scarface74• 6 years ago

Isn’t it more of business problem that someone is hoping to stand out and become successful by selling on Amazon or any of the big platforms without creating an “unfair advantage”?

I don’t mean by gaming the system or doing something unethical, I mean in the classic business sense of creating a differentiator or in MBA speak thinking about Porter’s Five Forces.

user00012-ab3• 6 years ago

I ended up just not buying stuff from amazon anymore, because I was just getting literal trash in the mail that I would have to deal with.

miguelmota• 6 years ago

What kind of stuff were you buying?

user00012-ab3• 6 years ago

The final straw was a NEW mechanical keyboard. I received a keyboard covered in perfume, that was clearly (and badly) repackaged.

wombat-man• 6 years ago

Yeah I cancelled my prime. Nowadays a lot of retailers will ship for free, sometimes 1 or 2 day shipping without an annual membership fee. You just have to spend a certain amount, but more and more items on amazon are 'add on' items anyway.

photon_off2• 6 years ago

Amazon has been my wheelhouse, fulltime, for the past 7 months. I'm building a site that looks at most listings per category and determines "true" scores for each one. The site also lets your filter and sort, instantly, on a variety of attributes, like "price", "unit price", "shipping time", "recent price drop %", "used price discount", "popularity", "brand quality", etc. Spending 7 months collecting and analyzing data of several Amazon categories has been exhausting, but quite revealing.

As you can imagine, dealing with low quality products with fake reviews is a challenge -- but it turns out it's not too hard to handle, even with my dataset which is far more limited than Amazon's. Without looking at any reviews or any metadata of reviews (author, count, chronology, etc), one could filter out "impostor" products with 95%+ accuracy.

Here's a neat trick: Next time you're unsure if a product has fake reviews, click on the brand of the product and see what else they sell. If you're looking at binoculars, and that same brand also sells dog food bowls, then maybe you should reconsider.

I've concluded that Amazon really doesn't care about fake reviews -- they will show users whatever listing has the maximum Expected Value (conversion rate * revenue), per your context (search term, category, or both). Even if a product has obvious fake reviews, if there are enough other people buying it it will float to the top, and Amazon is fine with that.

the84722• 6 years ago

unit price

This is quite an important feature. Amazon already shows this information (sometimes at least) and yet they don't let you sort by it.

photon_off2• 6 years ago

The main problem they have, I imagine, is sparse data. There are only certain fields (depending on category) which they force sellers to populate, eg: "name", "brand", etc. Item weight (which is distinct from _package_ weight), and "number of units" do not seem required, and so not many items have that information filled.

So, with sparse data, they have three choices:

1) Allow filter/sort by "unit price" and do not show the X% of listings that are missing this data -- many of which the user may actually be aware of and/or interested in.

2) Don't allow the option at all, and just rely on the fact your customer will do comparisons manually.

3) Try to derive the number of units from text cues in the product name, features, and description, then do #1.

It seems they chose #2. I'm going for #3.

anonsivalley6522• 6 years ago

I think, independent of Amazon, another way to combat fake reviews is to have a modern, social-media-enabled, subscriber/patron-supported "Consumer Reports" review service that:

- doesn't accept free products or endorse any particular brands, and buys products like an individual could

- tests products thoroughly, both short-term and longer-term, including what happens within the warranty period

- reviews big-ticket items frequently enough to stay current

- offers purchasing gotchas and decision tree for selecting a product

- does the math on Total Cost of Ownership, including calculators so someone can plug in their local values (utility costs, taxes, rebates) to make the best decision

- has concise maintenance/operating advice

- starts small and looks for good products within a category

- also seeks out lesser-known, independent manufacturers who stand behind their reputations and products that people might not find on their own

- teardown and grade for repairability like iFixit

- doesn't spew blog articles that are merely copies of press releases

- compare products to those to the past to see the trends in quality, repairability, features and relative cost

The problem is that going into a store, platform or IRL, without trustworthy information leads to more arbitrary decisions. And then when there is a more reliable system for deciding what to buy, it's not that hard to have a browser extension that tags items on various platforms with recommendation badges.

There are various sites that try to present some reviews in particular genres, but the depth and breadth is usually relatively shallow and incomplete. Then there are the zillions of "review" sites that don't actually even test the products and just put manufacturer specs into a grid, and you have no idea if they have undisclosed sponsors, kickbacks or other deals.

Granted, it would be a labor-intensive proposition, so I think the best model would be a worker-owned co-op.

joshvm• 6 years ago

In the UK you have described Which. They've been around for years and getting a "best buy" rating is generally seen as a big deal (certainly it will get you sales).

Their website does have a lot of blogspam type content now, and whether you trust their "experts" is up to you. But for generic household goods, they're probably less biased than the average comparison site.

https://www.which.co.uk/

vsskanth• 6 years ago

Slightly OT, but I noticed I've been banned from posting reviews or asking questions on Amazon. No idea why. I don't review usually and my last review was in 2017! I only noticed it a few days back because I wanted to ask a question on a product.

Does anybody know how to get out of their blacklist ? I tried emailing but get no response.

Amazon can’t end fake reviews, but its new system might drown them out

Related Stories