DMTK: Distributed Machine Learning Toolkit from Microsoft

http://www.dmtk.io/

by mrry • 11 years ago

138 20 11 years ago

arbre• 11 years ago

This is very exciting to see these distributed deep learning frameworks open sourced by top companies. What I do not understand is why amazon AWS and other top cloud service are not integrating these frameworks in their services. Training a distributed neural net should be as simple as defining the model and specifying the resources.

vegabook3• 11 years ago

agreed - though I'm somewhat saddened to see that this library doesn't seem to be getting the same positive reception (in terms of scoring / comments) as Tensorflow did yesterday. Am I right and if so, is there a technical reason for that?

graffitici• 11 years ago

Not sure if this was ready for public release.. The chess demo released this morning seems more polished than this library.. Also, probably not the best decision to post it a day after the TensorFlow release..

math_and_stuff2• 11 years ago

Less fancy landing page? No fancy names in the title? Marketing works.

cbgb• 11 years ago

One technical reason may be that C++ (I only looked at LightLDA) has a higher learning curve than Python, which the Tensorflow docs stress.

Though Tensorflow stayed at #1 for a while, it only garnered around 200 comments, which is very high for a ML topic on HN (in my anecdotal experience, ML topics are highly rated but under-commented.) I imagine the audience for this library to be very small compared to Tensorflow, which likely included more ML/Google FOSS enthusiasts than day-to-day practitioners. Looks like this library is firmly targeted toward the latter.

blazespin3• 11 years ago

NO GPU, not for production use.

x0x0• 11 years ago

distributed ml tends not to work well on aws. Many (most?) distributed ml algorithms are essentially limited by network speed. In a p2p topology, many are effectively limited by the slowest peer-to-peer link of the (nodes choose 2) links. aws networking is mediocre at best; even the rack-local instances are relatively slow.

cred: helped build a distributed in-memory ml toolkit

math_and_stuff2• 11 years ago

Wow, more surprising than this being open source from Microsoft, it is based on MPI.

vegabook3• 11 years ago

and looks like Linux first class citizen.

jhartmann• 11 years ago

This is very interesting. I was very surprised that Tensorflow did not use a central parameter server, and this looks like a good foundational parameter server to build something like DistBelief on top of from Microsoft. I wonder if they have a Deep Learning module that they will open source eventually. Microsoft has a few tricks they used in some of their recent papers and it would be interesting to see them in a production quality system.

amaks• 11 years ago

Here Jeff Dean explains why there is no central parameter server in Tensorflow: https://youtu.be/90-S1M7Ny_o?t=28m59s

blazespin3• 11 years ago

Where's the GPU support?

brazzledazzle• 11 years ago

Sitting in your future pull request.

azinman2• 11 years ago

That they're providing a distributed LDA implementation with and O(1) Gibbs sampler is a big deal. I haven't played with it yet but the numbers they are reporting relative to cluster size is orders of magnitude improvement.

fitzwatermellow• 11 years ago

I was doing some research for a small demo. As a proof of concept, implementing a small ConvNet using WebGL shaders. But in the course of my search I stumbled on this very broad and interesting patent granted to Microsoft Corp.

Processing machine learning techniques using a graphics processing unit

http://www.google.com/patents/US7548892

Pretty amazing when you consider they wrote the application a decade ago! There is no arguing we are experiencing a golden age and an embarrassment of riches provided by ML toolkits and GPU cloud capabilities. But further inspection of patents in the space brought up this gem, in which Emotient is attempting to patent the crowdsourcing of training data...

Collection of machine learning training data for expression recognition

https://www.google.com/patents/US20150186712

DMTK: Distributed Machine Learning Toolkit from Microsoft

Related Stories