XNOR.ai frees AI from the prison of the supercomputer

https://techcrunch.com/2017/01/19/xnor-ai-frees-ai-from-the-prison-of-the-supercomputer/

by dwynings • 10 years ago

84 25 10 years ago

curuinor• 10 years ago

Rumelhart was the first to note this, I think, although of course he didn't have mobile phones. Implementation looks spiffy, but not spiffier than Han et al's work

saycheese• 10 years ago

Thought that Google's TensorFlow already does this and has since shortly after it was released - what am I missing?

Example: https://www.tensorflow.org/mobile/

hidden-markov2• 10 years ago

TensorFlow supports weight quantization, but XNOR seems to be rewriting the trained model graph into a one that only uses binary boolean ops, and probably optimizing the graph along the way.

shpx2• 10 years ago

I think this is their paper http://www.arxiv-sanity.com/1603.05279 and this is the (torch) code https://github.com/allenai/XNOR-Net

Cursory look suggests they quantize both the weights and functions during classification on the smart phone and during the forwards and backwards pass during training. But you still have to use a continuous version of your logic gate to get an error gradient during training, so I'm guessing they still do the training on "supercomputers" (a desktop with a gpu really).

Houshalter• 10 years ago

That uses lower precision arithmetic. In this case they are reducing the precision to 1 bit, apparently.

emcq2• 10 years ago

XNOR-Net is not currently part of TensorFlow, although there is a feature request to support it. [0]

The main difference is that XNOR-Net uses 1 bit quantized weights and XNOR+popcount to approximate the dot product of convolution, which can be implemented very efficiently by using 64-bit arithmetic instructions on a CPU to operate over 64 1-bit components in parallel. [1]

TensorFlow's optimization is also using discretized weights, but AFAIK targeted at 8-bit quantized weights. [2]

[0] https://github.com/tensorflow/tensorflow/issues/1592

[1] https://arxiv.org/pdf/1603.05279v4.pdf

[2] https://petewarden.com/2015/05/23/why-are-eight-bits-enough-...

arjo129• 10 years ago

TF is not the nicest thing to work with. It is still very computationally expensive to use on embedded hardware platforms.

krosaen• 10 years ago

Isn't this article conflating feed forward applications of trained conv nets with the training of conv nets? The latter is what is so expensive, yet it seems like the demo of object recognition is the former, and not a breakthrough to run on a phone AFAIK.

gtani• 10 years ago

recent discussion of running tensorflow/grad. descent on DSP chips: https://news.ycombinator.com/item?id=13379847

This is a good writeup on mobile/low power devices, lower precision arithmetic, compression, sparsity, hogwild type "naive" assumptions etc https://arxiv.org/abs/1612.07625

qeternity2• 10 years ago

Backprop is expensive. Not sure what the revolutionary aspect is here. Smells a bit like the Theranos of AI.

hidden-markov2• 10 years ago

I am pretty sure this is about optimizing the graphs of trained models. That is, training is not freed from "the prison of supercomputers".

qeternity2• 10 years ago

Exactly my point. Feed forward can run on much lower specs just fine. So their whole "prison of supercomputers" analogy is for the wrong side of the equation.

emcq2• 10 years ago

This is for optimizing inference. For consumer devices such as smart cameras, speech recognition on your phone, or robotics that's the part that matters.

They have real speed improvements by using 1-bit quantitzed weights and then running operations using 32-bit or 64-bit bitfields which happen to use very few instructions/cycles on modern CPUS: https://github.com/allenai/XNOR-Net

swatkat• 10 years ago

Uncanny Vision[1] have similar computer vision and deep learning libraries for ARM chipsets.

http://www.uncannyvision.com/

Dzugaru• 10 years ago

Inference (forward pass) in convnets is really cheap even in floats, there are some awesome hardware like NVidia Jetson and most deep learning framework (i.e. Caffe) support it right out of the box. So we were doing realtime inference in embedded for years now, what am I missing?

general_ai2• 10 years ago

You're missing the fact that Jetson is $435, it draws a lot of power, and requires cooling.

llukas• 10 years ago

So where do I buy those xonr.ai, how much does it cost and when do they ship?

adultSwim• 10 years ago

Disgusting appropriation of language to punch up a business headline

Hydraulix989• 10 years ago

It's tech journalism, what did you expect?

franciscop2• 10 years ago

The license specifies: The following license governs your NON-COMMERCIAL use of the Software. Commercial use is strictly prohibited.

That's a goodbye for anyone wanting to build anything interesting in the future with it. Such a pity. I couldn't find anywhere any kind of "commercial license".

fifnir• 10 years ago

Why is it that only commercial applications are interesting ?

franciscop2• 10 years ago

TL;DR, it is NOT that only commercial is interesting; it is that not commercial allowed is really restrictive.

I do a lot of Open Source, so I know the troubles associated with GPL and the like. This is a lot more extreme than that.

If you were to make a tool based on this a lot of your potential userbase dissapears. Like most of the people.

Why? For using it, while I mostly use Open Source, learning/mastering a tool only to discard it and then learn/master a different tool that does the same for a commercial application is a HUGE waste of time.

If there was a way (2 licenses) to use this as commercial software then it'd be a different story.

PS: I'm the author of legally and I made it for this reason: https://npmjs.com/package/legally

shpx2• 10 years ago

Sending data to a server isn't that "comically clumsy", light speed is very fast. https://superuser.com/questions/419070/transatlantic-ping-fa...

Unless you're talking about privacy, there I agree, I would much rather my phone do voice recognition locally than send it to google servers (where it'll be stored [0] so they can train better models later). That's a bit of a catch-22 though, if people don't need to send you their data to use your model then you won't get data for training your model.

[0] https://myactivity.google.com/myactivity?restrict=vaa

vavav123• 9 years ago

There are lots of starputs like XNOR.AI for example: http://aipoly.com/ http://scorch.ai/ they have dealing with devices also

XNOR.ai frees AI from the prison of the supercomputer

Related Stories