Intel Fortran now 20% faster than C, fastest language on Shootout

http://shootout.alioth.debian.org/u64q/which-programming-languages-are-fastest.php

by markokocic • 15 years ago

72 49 15 years ago

shootout.alioth.debian.org

markokocic• 15 years ago

You have to hit F5 to reload page, since otherwise the old page will be displayed from browser cache. Probably something wrong with page cache expiration settings.

antirez• 15 years ago

With Fortran this is expected, since for language semantics the compiler can perform more assumptions compared to C, resulting into more optimizations.

gcp7• 15 years ago

For the shootout, all the source code is visible to the compiler (though at least GCC needs to be told so with a flag). So I don't think there's much more assumptions that can be made.

wolf550e• 15 years ago

AFAIK GCC doesn't add "restrict" to pointers by itself. Fortran semantics mean that pointers are "restrict" by default.

hnwh• 15 years ago

Fortran's always been faster than C

masklinn• 15 years ago

Most of Fortran's gain is on mandelbrot, but it sucks on binary-trees (10x slower than C). It also tends to use 2x more memory (with a 23x outlier on fannkuch): http://shootout.alioth.debian.org/u64q/benchmark.php?test=al...

gcp7• 15 years ago

From a simple look, the Fortran code seems to have an additional optimization that allows it to avoid a lot of checking for overflows in the innermost loop.

larsberg• 15 years ago

binary-trees is rough to beat C on. It involves allocating and traversing some pretty gigantic data structures, and the C code is optimized by using region-style allocation to avoid all of the individual calls to free (or garbage collections in a GC'd language). From the CPU usage, it also looks like they also haven't parallelized the Fortan version yet, though even at ideal speedup it'll still lose.

I've been doing some tuning work on our parallel dialect of ML, and many of the C programs are fairly well-optimized, though mandelbrot still has some room for improvement.

chj• 15 years ago

These reports are worse than useless. Any person with good understanding of computing knows that implementation plays a huge part in it. Why people wasting their time comparing speed of different programming languages while they should focus on different implementations. I guess the latter one is difficult so people all try to avoid it, and the first one is just so easy to do and everybody is doing it.

wheaties• 15 years ago

That fact stands out immensely. The Scala code and java code should be equal as the first can be written exactly like the second. Don't even see Clojure. Hate to see what times they get there.

verroq2• 15 years ago

Except Scala has no 'break' (that isn't a hack using exceptions), this makes imitating imperative style difficult and some syntaxical sugar should no doubt ensure that Scala will always be slower than Java.

BindGuice• 15 years ago

Clojure is there but where is clang?

igouy10• 15 years ago

What would be interesting about Scala programs written exactly like Java programs?

Hint: If you want to check for some text on a web page, use page search.

verroq2• 15 years ago

Sorry, but your Ruby code will never the same speed as C code, even if it is the same implementation. We'd all like to believe all programming languages are equal, but that's not the case.

dalke• 15 years ago

Which of course the web site freely acknowledges. At the bottom of that page states the real question is "Which of these programming language implementations have the fastest benchmark programs?"

igouy10• 15 years ago

And at the top of that page ;-)

biasedstudy• 15 years ago

Intel Fortran vs gcc ???

How about Intel Fortran vs. Intel C ? How about gcc C vs. gcc Fortran ?

Hyper-optimized compiler for Intel architecture beats compiler designed to be portable to many, many target architectures? Not very shocking.

mrich• 15 years ago

This. ICC is known to beat gcc usually.

jbooth2• 15 years ago

Yeah, I'd suspect that better use of SSE instructions is the big win for number crunching.

stephencanon3• 15 years ago

This is a really terribly done study, with a misleading title to boot.

First, even if we take the conclusion as given, "Fortran" isn't faster than "C". The median of one set of programs, compiled with one Fortran compiler with one set of options and run on one processor, is faster than the median of another set of programs compiled with one C compiler with one set of one and run on that processor. This tells us approximately nothing.

If you only expand your consideration to include the whole distribution of programs they tested, GCC actually looks better than iFort (eg. there is a single program which ran 2x faster with iFort, but multiple programs ran more than 2x slower). Median is a spectacularly poor choice of metric, especially with so few samples in the distribution.

But also: Intel also makes a C compiler. Why wasn't that used? How were these benchmark programs chosen? Etc.

This study says close to nothing about "Fortran" and "C", or about any of the other languages involved.

mrgoldenbrown• 15 years ago

This is a really terrible criticism of the linked page, with a misleading first statement to boot. Where do you see the word study? The title at the top of the page says "Computer Language Benchmarks Game". How is calling it a benchmark game misleading? That's what it is, and it doesn't pretend to be anything else - it is a sorted list of benchmarks run on one particular configuration.

stephencanon3• 15 years ago

They conducted a series of experiments, and reported their results. If that's not a "study", I don't know what is. The fact that the authors acknowledge that it's a "game" doesn't change that. What is misleading is the title on the link: "Intel Fortran now 20% faster than C".

gcp7• 15 years ago

This study says close to nothing about "Fortran" and "C", or about any of the other languages involved.

Sorry, but I disagree here. It's one of the best resources out there for getting numbers on the relative performance of languages, despite the pitfalls. The implementations of popular languages are well tuned (sometimes extremely so), the used compilers and settings are quite sane, and the benchmark programs are fairly well distributed in functionality.

The median is a reasonable choice because it's less sensitive to outliers and was probably chosen for that reason. "Spectacularly poor metric" seems to be an opinion, not a fact.

I admit it would have been better to compare Intel C with Intel Fortran. Why wasn't this done? Perhaps some of the submitted programs don't compile correctly with Intel C, or perhaps Intel C doesn't actually perform well on them. Who knows.

But to say that this is a "terribly done study" because of that is doing it injustice.

chalst5• 15 years ago

Median is a bad choice, because it completely ignores just how bad the worst performance holes are for a compiler. Discarding outliers is bad methodology in this case.

stephencanon3• 15 years ago

The median really is a spectacularly bad choice when there are so few data points, and especially when thinking about performance. If A and B get 3x faster, but C gets 3x slower, then your overall workload gets slower, even though the "median performance improvement" is 3x. Obviously the effect is magnified with only 3 data points, but the number of points used in the examples on that page is not so much greater as to eliminate this effect.

Adding to this is that significant slowdowns, the "outliers" that you're keen to disregard, tend to dwarf performance improvements that the median might show (because of the multiplicative nature of these values: if A/fortran is 10x faster than A/C, and B/fortran is 10x slower than B/C, then A+B/fortran is 5x slower than A+B/C).

What's really right is to show the complete distribution of speedups and slowdowns (as one of the detail pages on that site does). If you look at the actual distribution for the comparison of C and Fortran in question, C actually comes out looking much better than Fortran does (which shows precisely how misleading using the median can be).

If you must to boil it down to a single number, there are several choices better than the median. I would be happier with a mean, despite its imperfections.

tomandersen• 15 years ago

Fortran was designed to munch numbers. As most programs are written by people who will not spend much time getting that last 3x of performance out, a language like Fortran makes sense for the number crunchers. Its easier to screw up in C. These kinds of tests are not all that great, as the code is optimized. Its un-optimized code that matters, since that powers the world.

seltzered_• 15 years ago

FWIW, This Developer's Life has an interesting interview with a fortran programmer this month: http://thisdeveloperslife.com/post/2-0-7-dinosaurs

onedognight• 15 years ago

Is there some reason that gfortran from the GCC project is not on the list of compilers for this language shootout? Instead only the branded "Intel Fortran" compiler shows up. Hmm.

Intel Fortran now 20% faster than C, fastest language on Shootout

Related Stories