atrilla

From MSc to PhD in 2 year's time... in Spain this is the time you are expected to figure out what to research about. No doubt why Spain needs a bailout. So much to learn.

A very practical and useful collection of hints. I particularly support the point about tailoring the set of informative features. In the end, there is no free lunch for the "core" of the machine learning method (cf., http://ti.arc.nasa.gov/m/profile/dhw/papers/78.pdf).

Now deep learning seems to debunk the more-or-less usefulness of the feature extraction step.

And the closest to solving this we seem to be at present is fuelled by deep learning, which is basically a big neural network with an absurdly vast amount of neurons (i.e., parameters for learning, like the brain). We can observe how this brute-force technique works, but unfortunately no-one can explain why (it's a black box model). The same story has been on for decades.

I would relate it to being a discriminative model, which is tailored to solving a specific task, in contract to generative models, which try to model and explain the world. Perhaps the brain is not meant to understand how the world works but how to do take advantage of it.

I would still add another challenge for analogue: power harvesting. That's an environment where the amount of energy you can collect is so low that you can't simply afford to run a uC. However, you still need to manage that negligible amount of energy, because in the long run, it builds up, and then awesome stuff happens (switching a uC on, for example).

I bought the 2nd edition 15 years ago, I loved it, and I now cherish that useful book in my bookcase.

A wonderful book, indeed. It "teaches" a useful business management lesson: when two strong egos clash, the company drowns.

I also admire Carmack's way to tackle new (sometimes unknown) problems: read the literature, learn, do. And never failed following this.

I loved the parts when Romero swam across the lake to work all night with the rest of the crew, or when they invited in a stripper with pizza but Carmack wouldn't set the keyboard aside. So determined.

Precisely. In fact, I chose the term "maker" for my blog: http://ai-maker.com/ and the gist of it is teaching, discussing, critiquing and creating stuff related to Artificial Intelligence (in this particular case). To me, none of the aforementioned topics is incompatible.

I like seeing (soft) AI as the buttress that allows us to see farther, like Newton standing on the shoulders of giants. I guess the sudden presence of AI is due to the sudden amount of available data (and thus, a potential source of useful information).

Awesome! It's wonderful to see these kind of side projects for deliberate practice.

I also expect to build some of these at http://ai-maker.com/

Thanks for sharing it and congratulations!

I agree. In fact, this is a recurrent piece of criticism that also appeared on an MIT Tech review, which I commented here:

https://news.ycombinator.com/item?id=8866253

I'd the thrilled to hear your story with AI and OReilly. It must be very encouraging to have this project with such an important editor.

Sorry for that. I just found it appropriate for each of the different sub-discussions (and I just removed one of them). When threads get massive with text, I sometimes miss interesting stuff and I didn't want this to happen to other readers.

I'm starting this side project and recently HN is on fire wrt AI. I'm seriously willing to put a lot of time in this, and the more people I can help the more I can learn.

Sorry again if that bothered you or any of the other watchers here.

It must be hard to resist a big payslip, we're all humans, heads of family, we want to provide the best for our beloved ones.

I see a veneer of disdain to Norvig's attitude in your words. I don't know much about this criticism, but to me he has done a lot of good to us researchers with his AIMA book. In fact, this is the core that vertebrates my blog: http://ai-maker.com/. In the end, Goertzel (guru in AGI) works again for a financial prediction firm, doesn't he?

There's a practical side to this stuff that doesn't involve ads.

Just have a look at the AIMA book (http://aima.cs.berkeley.edu/), it's full of applications not related to advertising (both of its authors, Russell and Norvig, have signed the letter). I'm going to implement many of them in my blog (http://ai-maker.com/), so if you want to have a lot of fun with AI, join me in this learning quest.

I'm glad to read such positive attitude towards AI... I recently discussed in another thread if AI was indeed pushing people into the unemployment lines (an opinion from an MIT Tech Review). Link as follows:

https://news.ycombinator.com/item?id=8866253

Parent: https://news.ycombinator.com/item?id=8863279

As I said, I am so firmly convinced that AI has so much good to do that I just created a blog (http://ai-maker.com/) solely dedicated to AI and its applications, and I'm going to dedicate my spare time for the following years to grow this side project into something awesome, because that's where AI is leading us.

This aggregation/collaboration of weak agents... isn't it boosting? It's certainly a recommended approach to building robust systems, see Pedro Domingos' paper (tip number 10):

http://homes.cs.washington.edu/~pedrod/papers/cacm12.pdf

But I had the impression that the letter had a more transcendental feel than just focusing on a particular tuning technique.

Getting better at something is always good. Like The Joker said at the return of the dark knight: if you're good at something never do it for free. So there's always an avenue to make money.

As it is discussed elsewhere here, challenging yourself technically to build a portfolio may also lead you to making money, e.g., from consulting. I'm offering this service for free at http://ai-maker.com/ as long as problems are openly discussed in the blog. For private consulting, a budget.

That's precisely the kind of approach I'm seeking with my new blog on AI and data analysis: http://ai-maker.com/

Totally agree. He who expends all his time working has no time left to improve. I think I got this from the pragmatic bookshelf...

Related to the impact of AI to society, especially wrt the increase of unemployment. I was reading the comments of this MIT Tech Review (http://www.technologyreview.com/news/533686/2014-in-computin...) and I came across this opinion, which gave me the shivers. Two points:

1) AI taking manual workforce-based jobs. I can't help seeing how beneficial the industrialisation of processes has been for humanity. Instead of relying on inaccurate human judgement for manufacturing jobs, we let machines produce perfectly similar assets much better than we can do. This has increased the reliability of the outputs, in addition to lowering the prices of the products, which has made them affordable to many more people. Jobs get more specialised, so like the tools human beings have developed throughout history. Once more, survival entails adaptation. And this is again a matter of supply and demand. In Spain, where the economic crisis is still hitting the markets and unemployment, having a proper specialised education no longer guarantees landing a job (and it's not because evil robots are doing the tasks of leaving scientists).

2) AI taking over engineers, lawyers, etc. AI is difficult per se. Nobody comes up with a human replica made of metal by chance. Things take their time, and improvements are gradual. That's a matter of fact. At present, AI (plus Machine Learning, Pattern Recognition...) delivers a set of tools that allow us to see father, from the shoulders of giants. We had never been able to digest the amount of data we are capable of doing nowadays. Isn't this progress? We haven't yet created a creative machine and I don't see it coming any time soon.

I am so firmly convinced that AI has so much good to do that I just created a blog (http://ai-maker.com/) solely dedicated to AI and its applications, and I'm going to dedicate my spare time for the following years to grow this side project into something awesome, because that's where AI is leading us.

I think I disagree with your comment regarding programmers: Linus is admired for such strong opinions, close to impoliteness, for example.

More than a jujutsu technique for dealing with such incisive questions, it is clear that being the second contender to enter the marker saves you the trouble of having to create such market (and it is sometimes reasonable to play it this safe). It's just a different approach to making money. To me, a VC asking this is a sign of weakness on his behalf (due to ignorance?).

Simply put: be curious. As far as I know, the first hackers built train models (http://tmrc.mit.edu/).

Research, make stuff, learn.

I see lots of common points with the "lean" method here (at the expense of technical debt), but can't forget Knuth's advice that premature optimization is the root of all evil.

To me, the utmost important aspect to take into account is the "key" of the business. I had a similar story with my PhD research (AI+NLP): I wanted to focus it on adding value to commercial products. I worked on the core of the implementation, I built it with Java, with a configurable pipeline, plus an online app with servlets. I did attain getting noticed by some companies, but at the moment of closing future lines with them, my adviser told me this was not what he expected from me, and I was forced to get back to the non-useful-goal-centred research of academia. I am still glad I did what I did, regardless of how useless it was eventually, but my knowledge grew, and I learned how important it is to know your business model before you do anything at all. I learned how unbalanced is the academic market, always relying on public funds to survive instead of worrying about building useful appealing stuff. I realized where I wanted to be, so I dropped out and joined the private industry where I now feel very fulfilled.

My present approach goes from small to big, little by little, getting as much feedback as I can so I can fix mistakes asap and prevent them from getting bigger and more difficult to manage. My agreement with your words.

Don't you think that companies should support such personal deeds? This is very common in the electronics community, where expensive instruments are needed and workers find it difficult to buy them themselves. Eventually, motivated and eager-to-keep-learning employees are the best assets that a company can have.

I agree with you. It's a matter of taste and purpose, and this is dynamic, so you'll probably want to see them all in time.

My motivation has always been focused on deliberate practice (i.e., a fancy way to approach personal and professional development). I have been through the app-based MVP and the personal mind-dump. I recently started a new blog, which intends to be a mixture of the two: personal interest and passion for Artificial Intelligence, and an approach to consulting:

http://ai-maker.com/

Programmers tend to say: ABC, which stands for Always Be Coding. I apply it to anything I like and that I intend to get better at.

A very interesting requirement. Thanks for noting it.

I'm setting up a blog on Artificial Intelligence (which inherently includes Machine Learning) focused on the contents of the AIMA book (by Stuart Russell and Peter Norvig):

http://ai-maker.com/

How would you like it to be? Code will be developed in Matlab, so the math depth is rather convenient using this tool.

Hi, thanks for your feedback. I've been posting technical Machine Learning issues (my interests) for a while now (a few years), not quite regularly I must say, but no success. I assume I must be doing something wrong here, because others do succeed with similar topics. Moreover, as my professional career evolves so do my interests (business topics), and that's what led me to stick out a little. I'm very eager to learn about the hot topics of a challenging community like HN, and that's what pushed me to ask the question directly.

Hi, thanks for your feedback. The classifier behind the scenes has already been trained with a supervised learning algorithm, so it can be customised/adapted to any user input. This is a business case I explore with each customer in order to properly fit the tool to the particularities of their problem (e.g., the specific salient features that represent their data).

Yep, Pattern Classification by Duda, Hart and Stork:

http://www.amazon.com/Pattern-Classification-2nd-Richard-Dud...

It is very pragmatic, including algorithms for many machine learning and artificial intelligence topics (from fitting functions for classification or regression purposes to search processes). The authors have a strong industrial background (in addition to the academic).

Great work! But I missed the "sentiment analysis" flavour that used to be so popular some years ago with the NLP bunch... In this sense, I did something similar:

http://dtminredis.housing.salle.url.edu:8080/EmoLib/

and

http://nlptools.atrilla.net/web/omsa.php

Drop me a line if I can be of any help!