Deep API Learning

188 35 10 years ago

You may hate the tech industry for recklessly destroying other people's jobs, but just know that we're working hard to build a next generation AI that will destroy our jobs too.

xiphias• 10 years ago

At least I hope that AI will think of Bitcoin as a good investment

thomasahle• 10 years ago

Good thing is, that once they fully destroy our job, we will have achieved the singularity, and all jobs will be destroyed. Ergo, our jobs will be the last to be destroyed ;-)

amelius• 10 years ago

Most likely, big corporations will reap the benefits of AI for quite a while before the AI will escape and take over the world. Those CEOs and perhaps a handful of supporting personnel will get paid long after programmers have lost their jobs.

Turns out that there wasn't so much room on the shoulders of giants after all.

jtolmar• 10 years ago

That's only really true if you're an AI researcher. You could automate away enterprise bucket-brigade code and web-based CRUD apps and destroy an awful lot of programming jobs with no singularity.

raverbashing• 10 years ago

Oh I do want an AI to decipher an API for me, it's usually annoying to discover how it works

daef• 10 years ago

.oO( we're good at writing code, but bad when it comes to reasoning about code, so why not write code to solve that problem... )

BillinghamJ• 10 years ago

If we struggle to reason about code we've written, perhaps we weren't so good at writing code?

vog• 10 years ago

Not quite. It also means that our languages are inadequate for the task.

As a rough analogy, consider how ancient philosophers tried to reason about natural language explainations (as a vehicle to reason about the world). This led to the development of formal languages, especially in mathematics, but also e.g. in laws (both have lots of clearly defined terms in them that try to make up for the ambiguities of natural language).

This is exactly what I like about languages like Haskell, where you can reason realtively easily about code (although it's far from perfect). Or OCaml, where in addition you can reason about the performance (although not perfectly, due to garbage collection etc.). Or Rust, where in addition the compiler helps you to reason clearly about memory usage and aliasing.

This is all far from perfect, but my point is that improving languages (and actually _using_ these good languages!) is as important as writing good code in the first place.

kilotaras• 10 years ago

Here's a direct link to demo[1]. Looks like it received a hug of death.

[1] http://bda-codehow.cloudapp.net:88/

simongray2• 10 years ago

This is a pretty cool research area, but isn't this implementation basically just a Javadoc search engine? A basic keyword search of the javadoc descriptions would return similar results in most cases, wouldn't it?

nl2• 10 years ago

Section 5.2 compares it to search techniques. It's a bit more than twice as good as the next best method.

simongray2• 10 years ago

Ok. I definitely see the value then.

daveguy2• 10 years ago

The technique is a bit more complex than simple text search. It uses actual click-through data to determine quality of the search result as it applies to the query about an API. The SWIM (synthesizing what I mean) paper can be found here:

https://arxiv.org/abs/1511.08497

Also of note from section 5.2: SWIM uses Bing clickthrough data to build the model.

Using a better (or simply more used) search engine like google search would likely improve the SWIM results.

EDIT: The method they use to compare the methods is BLEU which stands for Bi-Lingual Evaluation Understudy and was developed for automated machine translation evaluation. Apparently CS authors no longer bother with expanding acronyms the first time they are used. Paper is here:

http://www.aclweb.org/anthology/P02-1040.pdf

EDIT2: Also, for the BLEU comparison they compare the computer generated API sequence to a human-written API sequence. However, they give no details on who or how the human-written sequence is developed. Are the researchers coming up with their own API sequence? Are they using mechanical turk? Interns? There could be significant bias depending on how these human-written sequences are generated.

petra• 10 years ago

Does any of the advanced machine learning and information retrieval gets implemented in tools useful for searching API's and source code ? or is Google keeping all the cool tech to itself ?

npalli• 10 years ago

The results show that our approach generates largely accurate API sequences

OK then, humans still needed. Best case, it seems AI will take up what one would consider interesting work (algorithms, thinking) and humans will end up doing grunt work -- testing out bugs, cleaning up data, formatting data, explaining to other humans etc.

Realistically though, this just creates two API's (instead of one) for humans to master, the original API and the 99% accurate machine API and knowing where the gap/bugs are.

reality_hacker• 10 years ago

humans will end up doing grunt work

labeling training examples for deep learning

skoocda• 10 years ago

this just creates two API's (instead of one) for humans to master

Not really, it's one mega-mecha-meta-API instead of hundreds of disparate small ones.

So you'd need to be able to find API bugs, I agree with that, but overall you'd probably need less knowledge. Especially if a large number of humans use this DeepAPI system, the bugs can be fixed relatively quickly.

vessenes2• 10 years ago

I just tried this, and it is surprisingly not bad functioning as a sort of oracle / suggester. I keep a notes.txt file with thoughts questions, ideas, etc. in most projects, and I would like this hooked up to it in vim, with a given line opening up the results window on the right as I moved to it.

I tried "draw a 3d cube", apparently it doesn't have any 3d java libraries baked in, but it did give me a bunch of 2d APIs, and then "plot a math function", giving me some trig functions directly and some plotting functions.

That would probably have saved me 80% of my time looking stuff up, especially in such a large search space

karmacondon• 10 years ago

This is a good idea for a product. I guess it's kind of like a super grep

vessenes2• 10 years ago

Yes. It could be an API that stackoverflow licenses and puts on their question page.

Deep API Learning

Related Stories