Computer program fixes old code faster than expert engineers

http://newsoffice.mit.edu/2015/computer-program-fixes-old-code-faster-than-expert-engineers-0609

by Libertatea • 11 years ago

132 62 11 years ago

newsoffice.mit.edu

CmonDev• 11 years ago

Title correction: "Computer program optimizes the speed of image filters".

IceyEC• 11 years ago

And general programs like MS Word...

chucksmash• 11 years ago

Where do you see that? I saw mention of using it on one other program - an image viewer, Microsoft Windows IrfanView.

bentcorner• 11 years ago

I believe MS Word (or some components in office) already has some sort of PGO applied to it:

http://llvm.1065342.n5.nabble.com/Capabilities-of-Clang-s-PG...

notacoward2• 11 years ago

Well, the technique could be applied in other areas, but optimizing is still not the same as fixing. The headline implies a solution to a much larger and more difficult problem than the researchers were actually trying to address, so it's effectively clickbait. I for one would not have clicked through if the headline had been accurate.

mafribe5• 11 years ago

    the technique could be applied in other areas

The optimisations used in the paper are tightly coupled to a certain approach to image-processing.

nickpsecurity2• 11 years ago

Exactly...

r4um• 11 years ago

Relevant paper http://dl.acm.org/citation.cfm?id=2737974&CFID=526962266&CFT...

aaronkrolik3• 11 years ago

Thanks! and the project source: http://projects.csail.mit.edu/helium/

akkartik• 11 years ago

Oddly enough, there's another group at MIT working on rewriting binaries that was recently discussed on HN: https://news.ycombinator.com/item?id=9804036

jakozaur• 11 years ago

Black swan event in 2020: AI replaces 80% of software engineers.

</fiction></joke>

prof_hobart• 11 years ago

I know you're joking, but the reality is that software has been replacing huge chunks of developers' work for years, whether through optimising compilers, common libraries, CI/CD tools, web APIs, IDEs or whatever.

I've been involved in development for over quarter of a century. Back in the late 80s, we had a dev team dedicated for years building some mapping software. Their entire product could probably be put together in a couple of hours using Google Maps and a bit of JavaScript.

All this automation has happened, yet I see no slowdown in the amount of development that still happens. They can just concentrate on more bespoke, more value-adding work these days.

pjc50• 11 years ago

more bespoke, more value-adding work these days

A software version of the Jevons paradox: making software cheaper increases the demand for software.

Fede_V• 11 years ago

Yeah, the paper in itself is interesting, not sure why they had to give it that completely misleading title.

plaes• 11 years ago

Did they lose original source code?

cognivore• 11 years ago

No, I don't think they did. I think they're just taking their old awful unmaintainable source, adding more to the hot heap of slag, and then using some clever compiler optimizations to make it run better. No love for the actual programmers. You get to keep working on the nightmare code (3 months to make changes...).

Now, color me impressed if the thing output the same language as the original source, all spruced up.

As it is, they'll just have a code base that will eventually transform into the anti-christ cause management will always be like, "Hey, don't ever fix the code, just run that see-saw whatever thing on it afterwards."

mafribe5• 11 years ago

I think what the paper does is a variant of profile-guided optimisation (PGO) [1], an old approach to optimising compilers, where the binary is annotated and run to produce information about runs that is then feed back to the compiler so as to allow better optimisation the next time the compiler is run. The most well-know off-shoot of PGO are tracing JIT compilers. The first (well-known) tracing JIT compiler was Dynamo [2], and indeed, the system under discussion here uses DynamoRIO [3], a descendant of Dynamo to instrument a compiled binary. The instrumented binary is then executed multiple times to generate execution traces that allow the optimiser to find hot code, which is then analyses to adapt the code to the new architecture, e.g. change buffer sizes.

[1] https://en.wikipedia.org/wiki/Profile-guided_optimization

[2] http://www.cs.virginia.edu/kim/courses/cs771/papers/bala00dy...

[3] http://www.dynamorio.org/

aaronkrolik3• 11 years ago

Interesting. So what would you say is novel/noteworthy about this research?

mafribe5• 11 years ago

I have not had time to study this paper in detail, so please take what I say here with a grain of salt.

Dynamic instrumentation is powerful, I think Google is using DynamoRIO heavily. Optimising compilers for DSLs is a trendy topic because

1. We are beginning to understand well how canonically to embed DSLs in general purpose languages, and have to build compilers for languages.

2. DSLs typically work on much more restricted domains than general purpose languages, so optimisation is much easier, and more powerful.

The Pochoir compiler for stencil computations is an example of this. There are many open questions about compilation of DSLs other than optimisation, for example automatic generation of operational and axiomatic semantics of DSLs, and associated tooling. Really interesting research field.

[1] https://people.csail.mit.edu/yuantang/

vmarsy• 11 years ago

In typical PGO you have access to the source code, the compiler instruments it for you automatically, you run the program with what You consider good input. During that time the instrumentation records a lot of interesting stuff. Then the compiler recompiles your program with that extra knowledge, making much better decisions[1]. For instance the compiler might realize: "Oh, this function is called much more often than I thought, I will now inline it."

Here (Link to the paper:[2]) this is different: The authors do NOT have access to the source code, they only have stripped binaries. From these unreadable binary instructions, they are able to identify interesting patterns: they extract the algorithm from the assembly. For this to work, they seem to need run time information. This means, as in conventional PGO, they need to have a wide variety of valuable input. It apparently cannot be done at compile time: "Current state-of-the-art techniques are not capable of extracting the simple algorithms from these highly optimized program."

Once the algorithm is figured out, the Helium framework is generating domain specific code in "Halide". Halide compilers knows how to optimize these stencil computation better than old hand written code, which give them these impressive improvements.

[1]https://msdn.microsoft.com/en-us/library/e7k32f4k.aspx

[2] http://groups.csail.mit.edu/commit/papers/2015/mendis-pldi15...

omarforgotpwd• 11 years ago

The headline made me think: Imagine the irony if, after years of various human jobs being replaced by software, someone one day completes the cycle by inventing an AI that can develop applications for humans. Far off, but funny to think about.

sz4kerto• 11 years ago

Software development is mostly not about syntax and programming languages, but domain knowledge and understanding humans. If a non-programmer could really describe what he wants, then you could write a compiler for that text.

But it's not possible yet, as the AI would need to have a human-like intelligence, and that's very far away.

raverbashing2• 11 years ago

It's closer than you think

You are right about humans knowing about domain knowledge, but you don't need human level AI to have computers writing software

Maybe future programmers will be more like spec writers than actual coders

codeshaman3• 11 years ago

I think what we're looking at in about a decade are interactive natural language development, in which one 'architect' will replace an entire team of programmers, designers, testers, etc.

"When I tap on this thing " - put the finger on the control "Go ahead and get the latest articles from HackerNews"

The System contacts "Hacker News" and queries for "latest articles" API. The systems negotiate an API key, account, etc and is good to go. The result is a list of articles with title and number of comments.

"Now use a nice table. " System formats output. "Show me other styles" Lists table styles. "This one. Now if the number of comments is more than 100, show them in red" And so on.

In the end, the System can generate source code in a multitude of languages (why?), some kind of pseudo code and of course an interactive, editable video of the "programming" session which others can watch and fix if necessary.

So basically, one person can design the whole "application" in a couple of hours.

We're not that far from that. And as we get closer, it will become better and better and it will be a lot easier to extend and modify this "System".

Imagine developing the System using the System.

Programming as we know it will remain a thing of the distant past as is the case with all things that evolve over time.

seanmcdirmid2• 11 years ago

We could have AI assisted tools; e.g. very intelligent code completion systems that are trained on your's and everyone else's code. Sometimes programmers don't know exactly what they want, but when given a few plausible choices based on some initial code, can generally pick out what they really wanted.

Up from enhanced code completion, AI can become effective in coding even before they are human-level intelligent simply by creating a conversational feedback loop with a human; e.g. more to the right, more to the left, redder, yes! One could argue that the human is still programming, but they might not need any specialized programming training to use such a system. In that case, we really aren't as far away as many think.

wsxcde• 11 years ago

Some amazing progress is being made in the field of program synthesis. In particular, there have been a lot of successes in generating programs from input/output examples, provided the domain is sufficiently restrictive and/or the programmer provides a "sketch" or partial program to start off the synthesizer with. One good paper to start looking at is: Oracle guided program synthesis, Jha and others ICSE'10. This algorithm has made its way into Microsoft Excel: http://research.microsoft.com/en-us/news/features/flashfill-....

You might argue this isn't "true" program synthesis, but in some cases it can outdo human programs, because the synthesizer can be tied into a verifier to produce only "correct" programs, where correctness is defined by a few simple and declarative properties. One example of this sort of thing is from Udupa and colleagues in PLDI'13: http://dl.acm.org/citation.cfm?doid=2462156.2462174. They're able to generate correct cache coherence protocols from a partial specification and input/output examples. This is a big improvement over the current state of the art, where a lot of very smart people think really hard and come up with a coherence protocol, convince themselves it's correct, and then some other very smart people spend a few months playing with automated verifiers trying to prove the protocol correct, and finally someone actually implements the protocol in hardware, and you might still run into bugs at this stage.

bottled_poe• 11 years ago

First they came for the factory workers, and I did not speak out - Because I was not a factory worker.

Then they came for the vehicle drivers, and I did not speak out - Because I was not a vehicle driver.

Then they can for the accountants, and I did not speak out - Because I was not a accountants.

Then they came for me - and there was no one left to speak for me.

keithb-• 11 years ago

Let's not conflate this with the Holocaust even as a joke. Automation like this is a step forward even if it only inspires a dreary debate about economic theory.

aaronkrolik3• 11 years ago

Question for clarification: the title uses the word "fix", implying that this new program repairs code that was broken. But in reality, it optimizes repetitive algorithms. Is that the correct takeaway?

jaawn3• 11 years ago

I believe you are correct.

kenbellows• 11 years ago

I think it "fixes" code in that it makes it work on newer hardware, but the title is still misleading; it suggests that the software is performing bugfixes and correcting code that has failing unit tests, which is totally unrelated.

burnte• 11 years ago

Fix in the sense that it takes a thing that used to work well and no longer does (due to changes in the underlying computing platform) and programmatically changes the methods used to be more efficient on modern platforms, while producing the same output. Fix in the sense of "improve" rather than "make not broken".

aruggirello• 11 years ago

Title is misleading; can a computer program step in and fix it, please?

jnaglick• 11 years ago

When people on HN don't bother to read the article, clickbait titles flood the frontpage.

brudgers• 11 years ago

The article is interesting because it is about the potential disruption of an industry and is based on work done at an institution with a reputation for research that disrupts technical industries, e.g. Chomsky's formal grammars.

cscurmudgeon• 11 years ago

The MIT PR machine has perfected the press cycle eclipsing good research.

http://www.phdcomics.com/comics.php?n=1174

thrownaway2424• 11 years ago

The amount of meaningless verbiage in this press release is indeed disturbing. What does it mean for the program to become "less effective"? A program is either effective or isn't. And a "billion dollar problem?" Adobe doesn't even spend a billion dollars on R&D entirely.

birdman3131• 11 years ago

Less effective meaning that the program does its job slower. From the way I read this it sounds like the original programmers used optimizations that worked well on older processors but not newer ones. For instance you might optimize one way for a pentium 4 but differently for an i7. It is not that the code for the P4 will not work on the i7 but the code is optimized for an older processor. Now this can be mitigated by the fact that the i7 is much improved over the P4 but you can get greater improvements by actually modifying the program source code as done by the software here.

As for the billion dollar problem I would guess it (Bit rot) is that or more over the entire software industry.

paulsutter• 11 years ago

I suspect that a valuable application of deep learning will be learning how legacy UIs operate by watching real users, and implement an easier to learn / nice UI / mobile interface on top of the legacy app. Probably with a human UI designer stitching it all together. It might even learn a voice interface by observing call center operators.

Could be easier, less risky, and more urgent than learning to recode the whole system.

lostmsu• 11 years ago

Clickbait title

foofoo55• 11 years ago

For the love of the human race and all of us still using Windows, I really hope a faster version of Irfanview gets released as a result of this.

Florin_Andrei• 11 years ago

Computer program generates machine code faster than expert engineers. It's called a compiler.

mzs• 11 years ago

Here's the paper in PDF: http://groups.csail.mit.edu/commit/papers/2015/mendis-pldi15...

Computer program fixes old code faster than expert engineers

Related Stories