Microsoft Research wins image recognition competition

http://venturebeat.com/2015/12/10/microsoft-beats-google-intel-tencent-and-qualcomm-in-image-recognition-competition/

by espeed • 11 years ago

166 41 11 years ago

venturebeat.com

dplarson• 11 years ago

It's mentioned in the article, but here's a direct link to the paper on arXiv: http://arxiv.org/abs/1512.03385

And direct link to the PDF: http://arxiv.org/pdf/1512.03385v1.pdf

wyc• 11 years ago

"Baidu does not show up in this year’s rankings. The company made more submissions than were permitted and ultimately apologized and fired the team leader who directed juniors to make the unacceptable submissions."

escap2• 11 years ago

see http://dswalter.github.io/blog/machine-learnings-first-cheat... for some background on this.

cooper12• 11 years ago

Pretty good read, provides all the context needed and raises some important points about benchmarks. Thanks for sharing.

escap2• 11 years ago

now discussed at https://news.ycombinator.com/item?id=10730020

sherjilozair• 11 years ago

Might be worth noting that it is very likely that Google, Facebook and Baidu did not even participate in this ImageNet. It is widely believed in the vision community that this dataset is at the end of its run, i.e. any subsequent improvements in performance are not due to scientific breakthroughs but through hyperparameter optimization and ensembling.

anirudhgarg• 11 years ago

The article and the title specifically mentions Google.

krasin• 11 years ago

Google participated with ReCeption / Inception v3 model, see the results: http://image-net.org/challenges/LSVRC/2015/results

In classification task, MSRA and ReCeption show very similar performance: 0.03567 vs 0.03581 top(5) error. The gap is much more drastic on the localization task: 0.090178 vs 0.195792

The residual learning presented by Microsoft Research seems to be a breakthrough, based on the early evidence. But yes, ImageNet needs to be updated to stay relevant.

Estragon• 11 years ago

Surely that's not true until someone achieves human-level performance with their entry.

nl2• 11 years ago

Google entered with ReCeption and finished just after the Microsoft team in the "Ordered by classification error" section (which is the usual "headline" number).

Baidu was banned. I don't think Facebook has ever done ImageNet?

But I agree that most of the interesting work will happen outside ImageNet now that human performance has been comprehensibly surpassed.

The only exception is non-Deep Learning based systems, where some people remain convinced that alternative approaches can match DL systems and have other advantages.

mac01021• 11 years ago

I bet there are breakthroughs to be had if they would compete on training time and computing resources, rather than just accuracy.

mikeskim2• 11 years ago

Is there a private one shot hold out in this particular competition?

smhx• 11 years ago

yes. more like 5-shot hold-out, and the held-out set is pretty big as well, so more legit.

mikeskim2• 11 years ago

so you mean there are 5 entries you can select to be scored in the end, and there is one private leaderboard no one can see but the admins? do you know the size of the private leaderboard test set?

robgibbons• 11 years ago

I'd like to have seen how IBM would have fared with Watson.

tmerr2• 11 years ago

Isn't Watson a question answering system? I don't think it's specialized for image recognition.

hiddencost• 11 years ago

https://www.ibm.com/smarterplanet/us/en/ibmwatson/developerc...

Watson is a marketing term.

IBM provides a variety of machine learning services and calls them all Watson.

skadamat• 11 years ago

Watson doesn't have much unique computer vision capabilities unfortunately. They did some good work in NLP a decade ago but not added much since

douche• 11 years ago

Have they not really added much to it since the Jeopardy stunt? We get some periodic rumblings about integrating it with a customer-service tool, but it is starting to sound a little Duke Nukem Forever

nl2• 11 years ago

AlchemyVision[1] from IBM Watson's Developer Cloud/BlueMix[2] is part of the Watson stack, and is pretty good. One of the few public APIs that lets you train your own categories.

[1] http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercl...

[2] Pretty sure I need some (c) and TMs in there somewhere

geomark• 11 years ago

I'm disappointed to hear this after watching the video of Andrew Ng's talk. He was very proud of Baidu's accomplishment in this competition.

free2rhyme2142• 11 years ago

What products does Microsoft use this technology in?

I've seen what Google's doing in Google Photos but I've noticed it's not accurate all the time.

eru• 11 years ago

Sometimes even with political repercussions.

jacquesm• 11 years ago

What products does Microsoft use this technology in?

http://venturebeat.com/2015/11/11/microsoft-launches-project...

free2rhyme2142• 11 years ago

Developer API's. It would be nice to see a consumer use case in Windows.

cmarschner• 11 years ago

Can't speak for this particular team, but a submission for a competition is usually done on the last minute, so it's highly unlikely that _this_ algorithm is part of a product like Oxford API already. But a predecessor might, and it might get integrated over time. A research prototype usually requires a lot of duct tape to work. Productization usually requires a different mindset (i.e. an engineering rather than a research team) who care more about things like test coverage, distribution, (time) performance etc. So by definition, MSR is ahead of the curve, product teams are on the curve.

Artemis2• 11 years ago

It could be used in Project Adam: https://youtu.be/zOPIvC0MlA4

dharma1• 11 years ago

Has anyone implemented this yet for Tensorflow or Caffe? Also, any more details on the MSCOCO segmentation part?

Microsoft Research wins image recognition competition

Related Stories