Crowdsourced Data mining, forecasting and bioinformatics via competitions

http://www.kaggle.com/

by pitdesi • 15 years ago

41 16 15 years ago

kaggle.com

stfu2• 15 years ago

Very interesting project! Can anyone recommend some good data mining for dummies tutorials/books/etc?

moserware3• 15 years ago

I recommend the video at http://blog.kaggle.com/2011/03/23/getting-in-shape-for-the-s...

I gave more recommendations at http://stackoverflow.com/questions/598726/overwhelmed-by-mac...

(Disclaimer: I work at Kaggle)

EDIT: Fixed second link

stfu2• 15 years ago

Sweet, thanks! I would very much appreciate if you guys could somehow motivate the great data mining talent on your site to produce tutorials. I would be even willing to pay there for some training course because traditional tutorial providers (Lynda etc) have apparently not yet discovered this as a potential subject.

_delirium2• 15 years ago

From a stats-heavy perspective, Hastie/Tibshirani/Friedman's Elements of Statistical Learning is pretty good, and now free: http://www-stat.stanford.edu/~tibs/ElemStatLearn/

ap22213• 15 years ago

How does Intellectual Property work in competitions like these? Are the entrants allowed to use proprietary methods? Do they give up IP by participating? Do the hosts of competitions gain any rights to IP or its usage?

It's not immediately clear by skimming the legal terms of service. I couldn't find a FAQ.

moserware3• 15 years ago

See section 9 of our terms: http://www.kaggle.com/pages/terms

In general, the competition host gets a license to the winning algorithm in exchange for a winner accepting the prize money.

We also have private competitions that do not appear to you unless you're invited. In these competitions hosts can offer appearance fees just for participating as well as access to more restricted data.

mwexler2• 15 years ago

This stuff is great. The rewards/prizes for the best model are so minimal compared to what it usually costs to build a great model via a consulting contract or hiring a high quality miner.

Similar to Mechanical Turk, we've managed to create a completely different value structure for some amazing work by smart folks... mostly by making it a competition. Great exposure for winners, sure, but these prizes are pretty minimal.

http://www.kaggle.com/c/GiveMeSomeCredit, for example, has a total basket of US$5K (only US$3K for first place) for a model predicting credit scores (in this case, likelihood to default or have financial distress). Folks I talk to who do this type of work professionally tend to charge far more than that to create these models.

For the company sharing this data, of course, big win: They get a cheap, potentially fantastic new model, and the creator gets some good exposure and some cash. But if these take off, they can really change the economics of how this work is created.

Vandy_Travis• 15 years ago

You are right -- there are some neat aspects to this model.

However, it also tends to devalue the work invested by the analysts. They are doing the work for essentially a lottery ticket -- winner take all. That's the reason that it can be so cost effective to the company running the competition -- they don't absorb the costs of the failures (or less optimal approaches).

Those costs have to be absorbed by someone. In this model, they are eaten by the analysts, who (generally) don't have enough resources to cover those costs. Due to that, I don't think this model is sustainable or will really catch on.

OTOH, in some (ideal) academic endeavors, having multiple groups compete for more funding or for a prize has certainly benefitted the sciences. In that case, however, the competition was more friendly than zero sum. Also, I believe the different sides tend to share information more than hoard it, yielding lessons learned to the entire group from one team's failures.

mwexler2• 15 years ago

That's fair. I originally had the word "devalue" in the comment, but I pulled it out. Why? No one is forcing the analyst to participate, but instead, they absorb the costs to get exposure or growth (or fun). Whether that's a fair tradeoff is up to each person, but given the number of folks participating in Netflix competitions, these, and others out there, a group of smart, analytic types feel the tradeoff is worth it.

jph00• 15 years ago

My experience is the opposite. I entered a number of competitions last year. For a data scientist, the opportunity to access clean data sets and interesting real-world problems, as well as to see how your techniques compare to others', is really compelling. (At least it was for me!)

I won a couple of competitions, and discovered that the competitive environment pushed me to create new algorithms and develop new ideas that I otherwise would not have. (Normally I would have thought my initial answers were "good enough" - but in the extremely competitive environment of Kaggle that is never true!)

And of course there were many more competitions I did not win. Honestly, I learnt much more from these - because by the end of the comp I knew quite a bit about the problem domain, and had tried a few ideas out, so when I then read the winning papers it gave me heaps of new ideas and insights that I could use in future projects and competitions.

I became so interested in the company that I invested in it, and then started helping out here and there, and finally joined full-time and this week have moved to SF (from Melbourne, Australia) and am now Kaggle's President and Chief Scientist.

tryitnow• 15 years ago

It depends on what you mean by "sustainable." Is it going to create a sustainable business model that will displace the whole analyst labor/consulting market? No. A significant portion? Probably not. Will it create opportunities for firms needing analysis to get it done cheaply and for non-traditional or inexperienced analyst to show their stuff? For sure.

Might it lead to totally new ways of approaching problems that would never have been discovered otherwise? Most likely.

So it's not going to replace analyst in a niche that is already hot, but it will be a valuable component of that ecosystem.

I wish them the best of luck and I'm already thinking of ways my company could use this.

zeratul• 15 years ago

Would "kaggle" be able to handle patient data? Would "kaggle" sign data use agreements with hospitals that are interested in a shared task? There is a growing number of medical data mining competitions, e.g.:

https://www.i2b2.org/NLP/Coreference/PreviousChallenges.php

But the data mining challenge delivery systems in medicine are scattered. Mostly because of inability to create a secure and centralized web service.

Crowdsourced Data mining, forecasting and bioinformatics via competitions

Related Stories