Netflix Progress Prize Awarded

http://biz.yahoo.com/prnews/081210/aqw507.html

by linhir • 18 years ago

55 53 18 years ago

biz.yahoo.com

FiReaNG3L2• 18 years ago

As per the prize rule, they are under the obligation to describe and publish their mode (as they did last time). I really wish someone would release an open source recommender package, coded in C, with PHP bindings, based on the latest / greatest algorithms, or any algorithm (just plain SVD if anything).

Interesting links (from the winning team(s) websites about their latest research) : http://public.research.att.com/~volinsky/netflix/kdd08koren.... http://www.commendo.at/index.php?lang=0&_0=2&_1=3

icefox2• 18 years ago

Well I am not going to do everything for you, but here is something to start with:

http://benjamin-meyer.blogspot.com/2006/10/netflix-prize-con...

It is a little framework I put together to play with the netflix prize framework. I implemented SVD as one of the examples.

On a side note I did think about the idea of putting together a small company that would sell recommendation systems to companies for their website. The value would not be in the algorithm, but in the ability to integrate with their software.

FiReaNG3L2• 18 years ago

Very nice, I saw many Netflix Prize derived frameworks but I somehow missed this one.

Did you test RMSE of your SVD on Netflix Dataset?

jmtame9• 18 years ago

2 years in and they still haven't given out the prize money. How's that crowd sourcing working out for you NetFlix?

boredguy8• 18 years ago

A 9% improvement represents a substantial increase in quality of recommendations. If I remember correctly, a 10% improvement meant the difference between a complete star rating, on average. (There was a fairly substantial write-up about what the 10% meant last year.) I'd say for the people-hours put in to the project, they've gotten more than their money's worth.

jmtame9• 18 years ago

Oh I bet they have. Have you calculated the number of hours the 35,000 teams put in relative to the amount of pay they received?

I bet they've gotten more than their money's worth.

zack2• 18 years ago

It's working great. In fact, I'd say it's fucking awesome. The recommendation algorithm has improved greatly, and contributions to the science of statistics have been made because of this corporation.

Your comment strikes me as obnoxious. You try to gibe NetFlix, but the truth of the matter is that there is a market. The considerable prize sum is enough to interest a competent individual.

jmtame9• 18 years ago

Sorry, you'll have to excuse that I'm not immediately a huge fan of corporations who give small amounts of money for significant amounts of research that ultimately go back to benefit the corporation.

Apparently, they're going to give all of the code out, which is good to hear. I'm proud of the machine learning community for that. As for Netflix, it's difficult for me to believe this is really about research and not about increasing their profitability.

I could be way off. Maybe $1 million over the course of 2+ years is a heck of a lot of money for a team of developers (selected from 35,000 teams), which will be used to generate more profit for NetFlix. Who knows. I'm glad that so far over 2 years, $100k was given out. That's good to see.

jderick3• 18 years ago

You have to remember that many of these researchers are funded by government money (eg, university researchers) or shareholders (eg, corporate labs) that may not be seeing any return on their investment. In some sense, Netflix is stealing cycles that may have been put to better use elsewhere. This could be seen somewhat akin to bribing a public official, only in a 'steal one penny from a million officials' kind of way.

siong19872• 18 years ago

Hey, it is not easy to develop a better recommendation engine. They have problems like the "Napoleon Dynamic" which is still remained unsolved right now.

Computer can only predict human behavior which is unchanging and static. But, this is always not the case.

jmtame9• 18 years ago

I didn't say it was easy, I'm not sure where you're drawing that assumption from.

I did essentially say that over 2 years, I'm glad that the collective research efforts of 35,000 teams, one of the teams has seen $100k from a corporation who will ultimately profit off their research.

schtog• 18 years ago

You are wrong that it can only predict static and unchanging behaviour. It might just be that there isn't enough data to draw any relevant conclusions about Napoleon dynamite.

sysop073• 18 years ago

Terrible! We so badly want to give away a million dollars, and yet all these people will do is give us a massive improvement on recommendations without taking our money! We are pissed

- Netflix

gambling8nt• 18 years ago

Successfully giving out the money--especially if there is (for example) the romantic, swooping in ending that staunch stated that he wanted (in his post above)--will generate a lot more positive PR for Netflix than the remaining 0.44% improvement will help them at this point. Regardless of whether you think of this as a genuine research prize, or simply a corporate ploy, you have to admit that Netflix is probably genuinely unhappy that no one has managed to complete the competition yet.

aswanson7• 18 years ago

It may be simply mathematically impossible to get to 10 percent given the observable variables. Maybe netflix knows this and just gets synthesized research on the cheap.

pg2• 18 years ago

I'd guess the machine learning guys by this point understand the data set at least as well as Netflix does.

aswanson7• 18 years ago

Wouldn't argue with that, but they may have run up against a bound, multiplied it by a constant, and said to themselves "highly unlikely that anyone can beat this, so lets put it out there and see if someone out there can beat it."

So far no one has and they have gotten a ton of research on the cheap, like the current team's for 50 k USD. Smart.

jderick3• 18 years ago

The machine learning guys have only a subset of the full dataset to work with. They have to send their algorithms to Netflix for testing on the full dataset. So it may be possible Netflix knows something they don't. It does seem quite strange how close and yet how far that 10% mark has been.

Eliezer• 18 years ago

They're over 9% already. Sheesh.

aswanson7• 18 years ago

http://en.wikipedia.org/wiki/Asymptote

light3• 18 years ago

Have you seen how long they've been at over 9%? ;p

Haskell4• 18 years ago

If the goal had been 5%, you would say: - It may be simply mathematically impossible to get to 5 percent...

And they are already at 9.56%.

aswanson7• 18 years ago

No I wouldn't have, because I could only make a statement like that after seeing performance information. Since at least 40 teams beat that performance the first year, I would have said they set the bar too low.

wheels• 18 years ago

They are getting it on the cheap. They've got many teams of top-notch folks working on building an algorithm tailored for their data-set and problem definition. For what they're getting out of this, $1m is a stupidly low price. If they've incorporated the stuff that's come out so far, they've already boosted their sales by several million based on the results of the contest.

staunch• 18 years ago

I keep hoping someone will swoop down with a crazy solution and snatch the prize from the people trying to solve it conventionally. So much more romantic than this slog!

tocomment2• 18 years ago

cool thought. any ideas what the solution might look like?

inerte2• 18 years ago

Each movie rating will be represented by a star in the sky, the constelations are the rating clusters, and on 09/20/2009 they'll align to achieve the 10% mark.

Hey, it works for astronomy!

tocomment2• 18 years ago

I'd love to win just so I could settle arguments by saying "anyone who has won a million dollar data mining competition raise your hand"

madmanslitany• 18 years ago

I started working on this about a month and a half ago. My RMSE is pathetic thus far(just started tweaking a kNN with Pearson correlation as a distance metric), but I don't really care. It's a great way to brush up on a lot of CompSci concepts at once you may be rusty on. Besides obviously machine learning, when you're trying to process 2 gb of training files on your personal computer, O(n) time/space complexity REALLY starts to matter, as does choice of implementation language...using mostly Python with C extensions for the heavy math right now...

Netflix Progress Prize Awarded

Related Stories