Data mining Reddit posts reveals how to ask for a favor and get it

http://www.technologyreview.com/view/527496/dating-mining-reddit-posts-reveals-how-to-ask-for-a-favor-and-get-it/

by Libertatea • 12 years ago

131 47 12 years ago

wingerlang3• 12 years ago

Off topic now, but is the title confusing? The word "Dating", I can't make sense of why it is there and/or what it means.

dewey• 12 years ago

Typo, should be "Data Mining" not really that confusing in this context though.

wingerlang3• 12 years ago

Aha. In hindsight, that does seem logical.

stewbrew• 12 years ago

I'd rather expect it to be the work of auto-correction or spell checking.

etfb• 12 years ago

Ha! I didn't even notice the error - read it as "Data Mining" two or three times.

userbinator• 12 years ago

I read it as "Dating -- Mining Reddit Posts...", and expected something a little different but still related to data-mining.

wingerlang3• 12 years ago

That was my initial reading too, hence the confusion.

philh• 12 years ago

It turns out that their algorithm makes a successful prediction about 70 per cent of the time. That’s far from perfect but much better than random guessing which is right only half the time.

From the graph, it looks like only about 27% of requests are fulfilled in the best case (jobs). In which case I can do better than 70% just by constantly predicting "no".

(I assume that this is just bad reporting. I haven't read the reference.)

Edit: skimmed the referenced article. Average success rate is 24.6%. The 70% they give (well, 67.2%) is "the probability that a classifier will rank a randomly chosen positive instance over a randomly chosen negative one".

im3w1l• 12 years ago

They got a ROC AUC score of 0.67. This means that for a randomly chosen denied request and a randomly chosen accepted request, they will give a higher score to the accepted request 67% of times.

Irishsteve2• 12 years ago

Below 0.8 is pretty much random

ronaldx2• 12 years ago

This makes the article correct (although makes no allowance for sampling error, nor for overfitting the available data).

Even so, I still dislike the use of this figure, as ROC AUC overstates a poor score: it hints at the model being correct 67% of the time.

In fact, the model is only knowledgeably correct 33% of the time, and just guessing the rest (taking its binary-choice 'score' up to 67%).

CurtMonash• 12 years ago

It's been a long time since a date of mine was satisfied with pizza.

nwenzel• 12 years ago

Althoff and co used a standard machine learning algorithm to comb through all the possible correlations.

What exactly is a "standard machine learning algorithm"?

I'm sure that probably means that they used something from scikit-learn but "comb[ing] through correlations" isn't as simple as clicking the Go button.

The rest of the article does start to get into labeling, holding out a test set, and some of the data cleanup (the real combing working).

I guess I was just hoping for more detail of how it worked and not that it worked. I get that this wasn't meant to be a PhD thesis on supervised machine learning, but the mechanics of data analysis are really interesting as a process of discovery.

Curious to know what others think. How did the balance of how vs that work for you?

jebus989• 12 years ago

It's a logistic regression model, a basic statistical technique which wouldn't have even come under "machine learning" a few years ago. Later they use some kind of LASSO regression to penalise the inclusion of redundant features.

"Combing through the correlations", it seems, literally means calculating the (Pearson) correlation between two variables (success vs. an input feature) and adding some interpretation, as they do on p6 of the arXiv paper. For test/training data, it looks like they just used a 30/70% split rather than k-fold cross-validation and holdout, but I'm sure it makes no difference either way and in this case (as often) is trivial to design and implement. Presumably their AUROC could be increased just by dropping in an SVM or a Random Forest in place of the logistic regression.

From what I've skimmed of the paper you're over-estimating the complexity of the study.

md2be• 12 years ago

Great Comment: Having studied graduate statistics within the stats department and a data mining within the CS dept, It amazing how well the CS crowd has rebranded statistics into something you can talk about at a bar without people's eyes glazing over.

colanderman2• 12 years ago

I'd expect better from a publication titled "MIT Technology Review" than to start a Y-axis oh-so-close to – but not quite at – zero.

Makes "craving" seem to perform much worse than it actually does at a status of 0.

pvdm• 12 years ago

"MIT Technology Review" has more similarity to BusinessWeek than the actual university.

scottfr• 12 years ago

Random pet-peeve. I dislike how the graph in the article has markers at each decile.

This is model-generated data so they could put an infinite of markers at arbitrary locations. The use of markers implies to the viewer that is a data-generated figure, which it is not.

ronaldx2• 12 years ago

I was burned by this, thanks.

pervycreeper2• 12 years ago

We find that Reddit users with higher status overall (higher karma) [...] are significantly more likely to receive help

There could be a common cause to these two factors (a certain way of writing, for instance), that could explain the correlation. I can't imagine someone basing a judgement on a stat reported on someone's user page.

sliverstorm• 12 years ago

I expect the higher score makes you seem reputable. Less chance you're trying to game the system with multiple accounts.

maxerickson2• 12 years ago

GP means that things like being an effective communicator and knowing how to play to the crowd will factor into karma and into asking a question that gets answered.

That doesn't mean score-viewing doesn't happen, but it isn't likely to be the only mechanism at play.

rjvir• 12 years ago

Do you think someone with an account of say 10 karma points and 2 comments on a month old account is equally likely to receive help, ceteris paribus?

pervycreeper2• 12 years ago

I'm sure it could be a factor (nothing's preventing someone from doing that, after all) but my original point was essentially as characterized in this post below https://news.ycombinator.com/item?id=7793607

The tone of the article, I think, was suggestive of a causal relation, which surely doesn't (necessarily or plausibly) hold.

001sky• 12 years ago

even if it is "threshold" its simply a binary relation and not an explanatory variable at the margin. so, yes...you might be able to weed out some data...but of what is left its unliklely to differeniate, except by proxy (as alluded to in the other comments) for something else.

tedks• 12 years ago

Reddit Enhancement Suite, a ubiquitous browser extension, shows the link karma, comment karma, and account age of any username that you hover over.

So, it's totally possible that, like others have said, people deciding which request to fulfill take karma/account age into consideration to avoid gaming the system with multiple accounts.

erikb• 12 years ago

beside the other reasons, a simple alternative might also be that users with high reputation usually are known by the regulars. If you recognize someone’s name for posting a lot of interesting stuff on your favourite website you probably help him out before other guys.

B-Con• 12 years ago

That seems fairly self explaining. By the definition of having more karma, they're better acquainted with the site and have successful experience appealing to the audience.

bwooceli• 12 years ago

A more interesting analysis would be to identify the correlative features that describe the giver of pizza vs the asker.

danbruc3• 12 years ago

Internet access but no food - strange world.

pbhjpbhj• 12 years ago

Are you suggesting that these people aren't being truthful about their need, or, are you saying it's amazing that internet access is so [relatively] cheap? Or... ?

Internet access costs less per month than a takeaway pizza in my country. Libraries and other community centres provide free access to computers with broadband internet.

One of the surprises for me when, some years ago, I found myself in a developing nation (with unstable power supplies and non-potable tap water) was that every street corner in the town I was staying in had a (hand painted) advert for an internet cafe. It wasn't cheap compared to food prices - but since then globally food prices seem to have gone up and internet prices gone down considerably.

danbruc3• 12 years ago

I just wanted to express how bizarre this situation is but not imply anything about (possible) reasons. My first thought was like just terminate the internet contract, sell the phone or computer or whatever you are using and buy you 50 kg of rice. That would be even healthier than pizza!

But I decided not to phrase it this way because this is obviously a very simplistic view and the people on the internet will tell you that in full detail even if you are aware of it. They might have free access to internet. It might be a temporary situation and not be economical to sell and later buy back the computer. Corn is much cheaper than rice. You have to cook rice and that requires energy. Rice alone is not healthy. Maybe they just do this to get in contact with others not because of having no money for food. They might need the computer for work. They just got robbed late at night, no money left but they still have a phone.

coralreef• 12 years ago

Hmm, these people could be making the more rational choice. Depending on what they do for money, internet access could provide a higher return than food.

danbruc3• 12 years ago

Higher return than staying alive? (Ignoring the fact that they can possibly dual use the internet to get free stuff, especially food, making it really a rational choice.)

pcrh• 12 years ago

Further, a single pizza is unlikely to make much difference to one's nutritional status. I see these sorts of things as closer to emotional support than life-saving interventions.

lotsofmangos• 12 years ago

This is why the coming AI overlords will not need to kill anyone. They will just be really convincing instead.

jlrubin• 12 years ago

Here's a project I put out in January, 65% accuracy, that does the same thing. These guys actually contacted me back then, surprised to see no mention.

And it has a live demo!

http://randomacts.media.mit.edu

Data mining Reddit posts reveals how to ask for a favor and get it

Related Stories