Most common words unique to 1-star and 5-star App Store reviews

http://www.marco.org/1111087530

by replicatorblog • 16 years ago

143 54 16 years ago

jdietrich2• 16 years ago

The words that leap out at me from the five-star list are "never" and "always". If ever a piece of data underscored the value of consistency, it's that.

lotharbot• 16 years ago

The words that leap out at me from the one-star list are "actually" and "says", words indicating that real behavior differed from expected/advertised behavior. The app says X but actually does Y, and therefore sucks. This strongly underscores the value of consistency between what your customers think they're getting and what you actually deliver.

megablast2• 16 years ago

If only this was based on any kind of logic. The problem with the app store users is that they often just look at the picture, and do not read the description. There are lots and lots of cases where people are asking for something that is specifically mentioned not available in the app store description.

abp• 16 years ago

Hm, think of:

... was never made before ...

... does always what it should ...

Something like that?

kes• 16 years ago

Maybe something like: ...I never have to think about... or ...never makes me...

necrecious• 16 years ago

haha, it matches my app's 5 star and 1 star reviews surprisingly well.

Most apps settle into the 3 star range for that reason.

chrisbaglieri2• 16 years ago

It would be fun to expose this data (perhaps with a few additional dimensions?) and hack up an extension/greasemonkey script to rip out reviews that fit a certain criteria.

riffer• 16 years ago

Would you pay money for this?

chrisbaglieri2• 16 years ago

Me personally, no, but it raises an interesting point. I could see a market in "scoring" reviews using models based on data sets similar to the one examined here.

rbritton4• 16 years ago

In my experience the #1 cause of "crashes" is being killed by the memory watchdog. An app I publish deals with photos, and I ran into such a problem with people trying to load in massive photos that I had to enforce an artificial pixel-size cap to stop it.

The iPad has a memory fragmentation problem. The longer the unit is up, the less you can guarantee that a request for a large amount of memory won't be met with a watchdog kill. What works once when it's from a fresh boot may not work a few days or a few weeks later.

jrockway• 16 years ago

Why do apps get killed for requesting memory? Can't malloc return NULL and the app act accordingly ("your picture is too big, so i am not going to edit it").

Now, it seems most people never handle malloc returning NULL, and "you can't do this today" is a bad user experience. But crashing is what happens in the first case anyway, and being auto-killed is a bad experience too.

rbritton4• 16 years ago

I don't know. But it'll usually give you the memory, send the level 2 memory warning (level 3 is kill), and then kill you shortly after. The memory warning is supposed to be a warning to release cached data, so it's sort of hard to release the data you're working on right at that point in time gracefully.

Zev• 16 years ago

Apps don't get killed for requesting memory. They get killed when they're running in the background and other apps request memory.

Can't malloc return NULL and the app act accordingly ("your picture is too big, so i am not going to edit it").

Thats exactly what happens. You're free to run:

  void *foo = malloc(INT_MAX);

foo will be NULL. The rest of the app will continue to run without any problems.

// edit: Here's what happens when you do run that (the last line is a NSLog() to show the address of foo):

  Malloc(4972,0x3e088868) malloc: *** mmap(size=2147483648) failed (error code=12)
  *** error: can't allocate region
  *** set a breakpoint in malloc_error_break to debug
  2010-09-12 19:44:59.154 Malloc[4972:307] malloc: 0x0

// edit 2: Here's the code: http://thisismyinter.net/misc/hn/Malloc.zip

cmelbye• 16 years ago

We deal with large photos as well, and it's usually just a matter of making sure the only thing in memory is what's currently showing on the screen, and optimizing which size of photos we're using so that we can use as little memory as possible. It's a pain though. We also run into problems with orientation changes, because the animation for the orientation change gets very, very choppy if you're using too much memory.

rbritton4• 16 years ago

Your experience echoes mine. There are only two situations where I use the full-resolution image, and I had to go to great lengths to make sure NOTHING else was loaded. The rest of the time it uses pre-rendered representations at various smaller sizes.

DrJokepu• 16 years ago

Ok I don't know a lot about iOS development or ARM development in general, but there's something I don't understand. I had the impression that the ARM architecture uses a memory paging system not unlike the one used by x86. This would imply that memory fragmentation shouldn't be an issue, unless we're talking about in-process logical memory address space fragmentation, which would be largely the developer's fault anyway. Am I missing something here?

rbritton4• 16 years ago

I haven't done any real investigation on the whole thing, but the general feeling I get about the whole thing is that over time, system-related processes may hold on to more and more memory. I don't know what specifically is doing it or why it's doing it, but the end result is that trying to load a 2048x2048 pixel image will work one time and the next time your app will get killed for it.

iOS devices have NO virtual memory at the OS level. Once it's out of physical RAM, that's when it really begins to clamp down on app memory usage. There's only 256 MB of RAM on the iPad, so loading that one image needs at least 40 MB of that. If a bunch of the Apple-originated processes are resident in memory, and it seems like it hangs on to a bunch of them even after the app is closed, then it may or may not let you use that much memory.

mikeklaas• 16 years ago

It's interesting how the negative list seems to be heavily-populated by unigrams from a phrase something like "doesn't actually do anything". The app store must have a significant misrepresentation-of-functionality problem.

jdietrich2• 16 years ago

It may well be a usability problem - the app does what it says on the tin, but the user can't figure out how. In my experience the usual meaning of "It won't do x" is in fact "I know it does x, but I can't figure out how to make it". Rightly so in my opinion - I think it's part of the great cultural divide in computing, where developers see software as a toolkit but users expect appliances.

rdrimmie• 16 years ago

Marco also notes that the app has to contend with the user's expectation of what it should do. Which might not have any basis in reality. If an app doesn't do what a user expects, regardless of where that expectation originates, it earns the 'useless' designation.

dirkstoop• 16 years ago

Funny to find "open" in the 1-star list. (not saying that that actually means anything)

slevcom• 16 years ago

Probably along the lines of "...as soon as I open the app it crashes..."

Most likely due to the memory manager issues mentioned above. I've seen some quality apps get massive numbers of one star reviews for this type of problem.

The star rating system really irks me for this reason. Something that could be fixed to perfect condition in less than 24hours permanently carries the baggage beyond its relevance.

streamline2• 16 years ago

GPL apps arent allowed on iphone you know that right?

sahillavingia2• 16 years ago

What I am going to be doing immediately: composing mock 5- and 1-star reviews based on these words, and running through my apps' future updates to make sure they are much more more readily described by the former.

wallflower• 16 years ago

It would be useful to separate the common words into two buckets (paid and free). Freeloaders reviewing free apps can be merciless...

megablast2• 16 years ago

I think it is the opposite. There is nothing like an user scorned for wasting their 99 cents.

patio11• 16 years ago

I strongly think that he is likely to be empirically accurate with regards to the free apps.

Relatedly, you should always have a price tag on anything you are selling so that you do not compromise people's perception of its value. (This is not necessarily going to make you the most popular person in your techy circle of friends, but then again since they won't actually buy your stuff their opinion is not the world's most relevant.)

If you give it away for free, give it away by special arrangement. (Which will make people feel like they're in the in-crowd with backstage passes, rather than entitled louts.)

sahillavingia2• 16 years ago

While this does happen sometimes, the more expensive the app the higher the app rating (in general).

Free apps, people will download haphazardly and then delete, leading to many 1-star reviews.

With more expensive apps, people are much more likely to appreciate the app as they do the research before spending their money.

Source: taking a look at the App Store plus my own apps.

togasystems• 16 years ago

Another great thing to measure would be the common words in the app descriptions. I would love to find out why people still download 1- star rated apps.

benatkin4• 16 years ago

As I guessed, the words used by those writing 1-star reviews suggest bitterness and idiocy on the part of the reviewer. Most nice people grade on a curve.

The words used in 5-star reviews aren't bad, though, except for the word "perfect". It's often misused when describing software.

sundarurfriend• 16 years ago

> Most nice people grade on a curve.

I don't understand this, could you explain?

benatkin4• 16 years ago

What I mean by grading on a curve is that, rather than give as many 1-star reviews as 5-star reviews, nice people almost always give between 3 and 5 stars. Even if they find an app to be useless to them, they often give the benefit of the doubt. Perhaps it will improve in a future version, or be more useful to someone else.

JoeAltmaier2• 16 years ago

But we're talking about the best and worst here- so presumably at the ends of the curve, and deserving of superlatives.

benatkin4• 16 years ago

Yes, but I think there's a different type of person that regularly leaves one-star and two-star reviews. I think there are a lot of people who don't want to trash something, even if it sucks, and don't care about statistical integrity.

vl• 16 years ago

It would be interesting to see such list for the app descriptions (may be gauged by sales, not by the app score). It might be that some harmless-looking words trigger positive or negative reaction. Freaconomics has interesting chapter on words used in house advertisement - some word like "wonderful" send negative message.

m3mb3r• 16 years ago

Someone should do this for the most upvoted posts/commented on Hacker News. I'm too lazy for that :)

Surely, we will find content with a lot less use of subjective terms like "awesome" but I'm curious by how much. That's one way to get a sense how objectively critical the posts/comments mostly are.

dpcan2• 16 years ago

The word "useless" has been popping up in my free apps' reviews too. I don't know how to interpret it.

I'm wondering if this is a new word among young people now. Kind-of the new "sucks" maybe?

po• 16 years ago

"useless" is more extreme than "sucks". I think it is dismissive in the same way as describing something as "FAIL". People are being exposed to more options and are being asked to make more and more judgements. I think making a gray analysis like "It's good at this but bad at that" is hard to keep track of with the quantity of options. It is far easier to simply lump things into black and white buckets of good and useless.

dpcan2• 16 years ago

But ultimately what's "useless" is the comment itself. It does nothing to better anything. It leaves the developer in a void of wonderment. What's useless? Is it lacking a feature? Is it not working at all? What void were you looking to fill?

I also get an overwhelming amount of 5 star ratings with well thought out reviews. So when one of these "useless" comments pops in, it throws me back to high school days, when everything "sucked" - which is why I wonder if the word is the new "sucks".

Thinking back, saying "sucks" so often was quite useless.

streamline2• 16 years ago

really really this analysis is so lame, if you take any product review and applied this analysis it would give similar results.

This is equivalent of all those spamfographics

inboulder• 16 years ago

The biggest problem with App Store 'reviews' is that iOS asks for a rating on uninstall, this leads to many '1 star' ratings, with no review at all, leaving the developer in the dark as to what the user's issues are.

mcobrien• 16 years ago

This was a problem, but it was fixed with the release of iOS 4.0. Now users have to go to the app store to rate an app.

JoeAltmaier2• 16 years ago

Surprisingly appropriate works to describe the best and worst people, as well!

jcl• 16 years ago

Interesting that "Apple" appears in the one-star list. In what sort of contexts does this appear? Are people actually blaming Apple for the crappy apps?

scdlbx• 16 years ago

I would guess it is along the lines of "I am surprised Apple allowed this app in the AppStore"

Most common words unique to 1-star and 5-star App Store reviews

Related Stories