Deep Learning Is Eating Software

https://petewarden.com/2017/11/13/deep-learning-is-eating-software/

by gk1 • 9 years ago

45 16 9 years ago

sytelus2• 9 years ago

Deep learning is certainly eating conferences, funding and PhDs. And that wasn't a bad thing until everyone got focused on generating another random architecture that yields another 2% improvement on their favorite dataset so their paper gets through.

niyazpk• 9 years ago

> another 2% improvement

2% is HUGE at this point, at least on the datasets that I am familiar with - ImageNet, MS-Coco, PascalVOC etc. And at this point, any modifications or strategies that gets you the 2% improvement is noteworthy, and I know that people in my team are looking forwards to techniques that will give us these improvements.

skierscott• 9 years ago

2% is HUGE at this point

Hell, on MNIST 0.14% is huge. Geff Hinton created an entire new architecture to get 0.25% error, which is far better than the baseline 0.39% error [1].

To be fair, he did reduce the error by 35%.

[1] https://arxiv.org/pdf/1710.09829.pdf

sytelus2• 9 years ago

Exactly my point. The "huge" improvement is perceived as important although has no real impact on advancing the field and solving core real issues.

dontreact• 9 years ago

Agreed. In medical applications a tiny "incremental" advance can mean the difference between worse than human and superhuman performance, which could mean revolutionary changes to clinical care.

carterehsmith2• 9 years ago

Sure. Can we see an example of Deep Learning used to create a regular CRUD app?

fny2• 9 years ago

I wouldn't be surprised to see deep learning optimize a GraphQL backend by indexing and caching queries as needed.

carterehsmith2• 9 years ago

Indexing and caching is optimization. Who did the real job, that was being optimized?

If my account balance is $5, and I buy that yummy Cheese Swirls for $2, I expect the balance to be $3 afterward. Deep Learning will not help, or frankly it can only make this simple calculation wrong. So... no, deep learning is not eating software. BTW deep learning IS software, so this is all just headline-grabbing.

scj• 9 years ago

While I agree that Deep Learning would be terrible at building CRUD apps (I'd be happy if it was good at performing analysis on human written code), one question I have is "If AI is performing more tasks, can we reduce CRUD app requirements?"

For example, if an AI is performing the task, it could interface with a back-end API rather than a front-end GUI (cutting CRUD development time). Also, we may not need to track as many things, like the time someone begins and ends their workday.

petra• 9 years ago

There are various low-code tools that greatly reduce the complexity of writing CRUD apps.They're good on their own, but Some of their possible tasks may be helped by deep learning: automatic UI generation. information management and search in complex systems. knowledge reuse(translating and recommending knowledge/code). Building semantic systems to help with some of the verification.

And from another angle: deep learning favors big business. so we'll see more consolidation. so less CRUD will need to be written.

thisisit• 9 years ago

Some of my work is related to traditional BI tools and not one day goes with me hearing someone, specially the management, talk about applying machine learning in some way.

Now while I can see some aspects which can be improved by usage of ML many don't tend to understand the implication of such systems. A good example came last year. On the BI system there existed 200+ reports with an average user having access to maybe 20 reports based on their department - finance, sales etc. There was a big push from the management to "improve" user experience with ML. Build a recommendation system they said. There was no cost benefit analysis done for using a complex ML based recommendation system on 20 reports - It sounded cool and in line with the hype.

Then there is the question of user involvement. While I believe there is surely a case to made for stuff which involves software engineers, like the ones mentioned in the post - search ranking, data center energy usage etc. But things which require non-software engineers is a bit doubtful. This is because as the post puts it - "This doesn’t require the same technical skills as traditional programming, but it does need a deep knowledge of the problem domain. " And engineers cannot be expected to have deep knowledge of every function. If they get too drunk on the ML kool aid, and some of them do, the end result will be mess.

halfnibble• 9 years ago

After reading this, is machine learning just the coded version of what you do in Calculus II? Analyzing a scatter plot and trying different equations to get the correlation coefficient value close to "1" and predict the next value? Because that would seem to me to have limited usefulness.

joepanda• 9 years ago

What you describe sounds like regression, which is a powerful technique with varied applications, but is not the same as deep learning. Aspects from calculus do play an important role in ML notably in the gradient descent algorithm used to adjust weights. If you’re interested in learning more I recommend the 3blue1brown series on YouTube

ianl_person• 9 years ago

I would say deep learning is a type of regression. The architecture of the network dictates the form of the equation. We're betting that equations of that form: 1) can fit the data 2) can generalize well to data that it wasn't fit against and 3) fitting/training is tractable. That allows making predictions about complicated data like pictures, words, sounds, etc. We may know the picture and need to predict a label for it.

Derbasti• 9 years ago

Not every problem can be solved by matrix multiplication and gradient descent. Particularly, many problems require exact solutions, and not just good approximations (think aerospace or science or finance). And speaking of science particularly, many solutions require insight, which ML often can't provide.

Deep Learning Is Eating Software

Related Stories