cbcase

Baidu Silicon Valley AI Lab | Sunnyvale, CA | Full-time | On-site | http://usa.baidu.com/careers/

The Silicon Valley AI Lab is Baidu's US-based research group, started a bit more than two years ago by Andrew Ng and Adam Coates. The mission of SVAIL is to build hard AI technologies that let us impact hundreds of millions of users.

We work on deep learning for speech and language; systems research to drive scalability of deep learning models; and new product development to bring research success to end users.

We are hiring for lots of roles in all three of these areas. The above link has the full list, but I'd like to draw particular attention to our need for software engineers (the "Software Engineer - AI Product" role). There is a huge opportunity to be an early member of a newly-formed team responsible for building the next generation of AI-enabled products. No prior experience in machine learning or AI necessary -- if you are a strong engineer, we feel confident we can teach the needed ML.

Apply at the link above, or email eloise@baidu.com if you have questions (or ask right here). Thanks!

Thanks for the kind words, Brandon! Been a busy couple of months :)

Both Kaldi[1] and CMU Sphinx[2] are high-quality open source speech systems. I know for a fact that Kaldi includes support for DNN acoustic models (I'm less familiar with Sphinx).

[1] http://kaldi.sourceforge.net/ [2] http://cmusphinx.sourceforge.net/

Mostly this, though it's not so black-and-white. The paper discusses results from a DNN-HMM system (Maas et al., using Kaldi) trained on 2k hours, and it does provide a small generalization improvement over 300 hours.

Much of the excitement about deep learning -- which we see as well in DeepSpeech -- is that these models continue to improve as we provide more training data. It's not obvious a priori that results will keep getting better after thousands of hours of speech. We're exited to keep advancing that frontier.

For a single utterance, it's fast enough that we can produce results in real time. Of course, building a production system for millions of users might require just a bit more engineering work...

Hi Jerome, those are great results! We got an email this morning from someone else on the Watson team pointing out that we didn't include the latest IBM number -- we'll be sure to update the results in the next version of the paper (three cheers for arXiv).

Of course, we openly say in the paper that we don't have the best result on easy subset of Hub5'00 (we had it as 11.5%). We're more interested in advancing the state of the art on challenging, noisy, varied speech. Of course we'll be working to push the SWB number down too :)

As in many things, it's a combination of both. For example:

- We wanted no more than one recurrent layer, as it's a big bottleneck to parallelization.

- The recurrent layer should go "higher" in the network, as it's more effective at propagating long-range context when using the network's learned feature representation than using raw input values.

Other decisions are guided by a combination of trial+error and intuition. We started on much smaller datasets which can give you a feel for the bias/variance tradeoff as a function of the number of layers, the layer sizes, and other hyperparameters.

I should add that I had the opportunity to work on this project and am happy to answer questions.

Thought it best to post the arXiv link, but there's some press coverage as well:

- https://gigaom.com/2014/12/18/baidu-claims-deep-learning-bre... - http://www.forbes.com/sites/roberthof/2014/12/18/baidu-annou...

Glad to hear it! I had a lot of fun coming back to campus. Good luck to you too!

carl @ sift

The first two classes (106A and 106B) are very techinical, though I'd hesitate to let anyone who hadn't taken classes beyond them to work on a piece of software I had control over.

I think a good metric for classes is the final assignment, since it captures "how far" the class goes. For reference, the final assignments are:

106A --- It varies, but has recently been a text-based "Adventure"-style game (in Java) that requires tracking the map, player state, various objects and their capabilities, etc. I think there might also be a small graphical component.

106B/X --- Again, it varies, but the best assignment (in my view) is a BASIC interpreter that implements both a REPL and stored programs. All C++, it's about building a big list of abstract expressions of different types (assignment, etc) that can be executed by walking that list (taking advantage of dynamic dispatch) and tracking global program state. It's a nice intersection of data structures and (very simple) recursive descent parsing.

107 --- A heap allocator to implement malloc(), realloc(), and free(), written on top of mmap(). A fantastic assignment.

So yeah, they're not trivial classes.

It's not exactly paradise. Because classes are only ten weeks total (slightly less in spring), you can't exactly spend the first week or two shopping around without doing non-trivial work for every single class you're shopping --- an approach which doesn't scale, to say the least. There's still a sprint the first few days. But still, it's nice.

I cannot understand why it is that so many obviously very intelligent people decide that we need another computer vision-based startup. Because the unfortunate truth is that computer vision (right now) doesn't work.

Let me qualify that. From the academic / research point of view, there have been a collection of real successes in computer vision in, say, the last ten years. But my sense is that what counts as a research success is a long way from what counts as a practical business success.

For example, the best generic object detector at the moment is probably Felzenszwalb's using deformable parts-based models[1]. And it's just not that good. On the latest PASCAL object detection challenge, you'll see that its mean precision is only ~30%.

Scott Brown, the interviewee, sets Vicarious apart by highlighting the fact that their system will be neurobiologically inspired. But the idea of learning hierarchical systems that mimic the brain's visual processing system is hardly new, and the jury is still out on whether these systems can do better than the "hand-coded" systems like Felzenszwalb's. As a random example, see [2].

Like.com showed you can build a business that uses computer vision in some way. But as Brown snarks, they "use a big bag of different heuristics to figure out the image." For the time being, that seems to be the only way to get computer vision to work in practice.

That all said, I wish them luck.

[1] http://people.cs.uchicago.edu/~pff/latent/

[2] http://www.cs.stanford.edu/people/ang//papers/nips07-sparsed...

Reusing exam/written homework problems is unquestionably lax, I'd agree, but I do not think that professors should be writing new programming assignments each year.

I worked as an undergraduate teaching assistant for intro programming classes at my school, and the amount of debugging (as that's what it is) that goes into designing a good assignment is extraordinary. We had a small collection of world-class teaching professors, and each time a new assignment was introduced, there were always problems. Some starter code didn't work for students A-E. Or a requirement was so ambiguous that solutions varied in a material way.

Like everything else, writing a really excellent assignment is an iterative process. After two or three offerings, you finally have something your users (ie, students) really value: an assignment that teaches well.

And to be honest, the presence of cheating on programming assignments never made any sense to me. It's so simple to detect plagiarism: you have the source right there (and a database of all previous submissions). I really do think that CS stands out among the classes for plagiarism because of how much easier it is to spot rather than an unusual incident rate.