Threadripper 3960X Compiles Linux Kernel in Under 30 Seconds

https://www.phoronix.com/scan.php?page=article&item=amd-linux-3960x-3970x&num=9

by zzeder • 7 years ago

190 54 7 years ago

phoronix.com

pella• 7 years ago

discussion: https://news.ycombinator.com/item?id=21628149

Linux # Threadripper : https://news.ycombinator.com/item?id=21628482

philliphaydon2• 7 years ago

I clicked to see discussion on the topic of the thread but you’re linking to threadripper discussion. Disappointed in lack of linux kernel compile time discussion.

ghostpepper• 7 years ago

Not trying to be rude, but genuinely curious: what else is there to discuss?

Is this a new record, or just a record for the price point? What was the previous record? Will this have a large effect on how the kernel is developed?

For what it's worth, I understand how hard it is to focus on a project when compile times get more than about thirty seconds.

linsomniac• 7 years ago

It is cool to see this level of parallelism coming to commodity hardware. 5-ish years ago a friend of mine had access to an experimental box one of the server vendors had made. Never went to production AFAIK. My memory is that it had 128 cores, 256 threads. One of the things we did on it, of course, was compile the kernel. I don't remember the exact numbers, but recall it being in the single digits number of seconds. This was done with just a ton of Xeon sockets.

hrgiger• 7 years ago

i think this the script they use: https://openbenchmarking.org/innhold/9a7355bdb73d85c9e044d02...

on vanilla head branch myn "time make -s -j32" took "0m56.226s" with 1950x

corysama• 7 years ago

We need to move on from the Linux Kernel and start using “compile Unreal Engine” as a CPU benchmark.

https://gpuopen.com/threadripper-for-gamedev-ue4/

dragontamer2• 7 years ago

That's probably a good point: the Linux Kernel is written using the C language. But C++ is a far more complicated language to parse and compile.

Unreal Engine, Firefox, Chrome... large C++ programs easily take an hour to compile on normal hardware.

With that being said: compiling for an hour is a huge hassle for reviewers. I think reviewers prefer benchmarks that complete in under a minute, rather than over an hour.

michaellarabel• 7 years ago

That's why I also use the LLVM compilation test (C++) and other workloads besides just the Linux kernel... But it seems the Linux kernel results are always what generates the most interest.

catalogia• 7 years ago

With that being said: compiling for an hour is a huge hassle for reviewers.

Can't they just set it to run then go to lunch? Seems like a non-issue.

KirinDave6• 7 years ago

I don't really understand why this is newsworthy, so maybe someone could help me understand. It's a big, power hungry desktop CPU with incremental performance gains over the last generation.

Why is this important? Is there some kind of architectural breakthrough these CPUs are using? Are these CPUs recovering performance lost to Spectre/Meltdown mitigation?

barkingcat• 7 years ago

This is a big power hungry desktop CPU with incremental performance gains with the x86/x64 instruction set from someone not Intel and forcing intel to compete on pricing and performance.

Just that alone is newsworthy.

KirinDave6• 7 years ago

I think that's it: it's sort of an intel vs AMD interest piece. It's newsworthy in that it's a market signal, not a technology signal. The desktop CPU market segment is seeing a lot of transformation as secondary purpose built processors and SBCs become more and more of the market, so this is just sort of a fun puff piece.

Thanks for helping me answer the question.

Karliss• 7 years ago

3970x almost halves the LLVM compilation time compared to 2990wx using the same core count. I would call that more than incremental performance gain.

KirinDave6• 7 years ago

Why? Isn't that within a modest error margin for the general arc of these? I'm looking at this graph over time and it seems like a linear projection wouldn't be broken by this.

"The next generation of processors is not quite twice as fast as the current generation" just seems like a pretty normal statement to me.

zamadatix• 7 years ago

Just to note AMD CPUs were not affected by Meltdown in the first place.

abridgett• 7 years ago

Reminds me when they built a (much older, smaller) kernel on a pSeries (POWER architecture) in under 5 seconds: http://es.tldp.org/Presentaciones/200211hispalinux/blanchard...

Gosh, 18 years ago, now I feel really old :D

l33tman2• 7 years ago

I was compiling the kernel for my boxes for the most part of the 90's and 00's and the funny thing is a sub-60-second compile time was always possible with the "latest gear". Like a 100 MHz pentium. Of course the number of modules you want/need to compile has steadily increased but still funny to see someone impressed with a 30 second compile time 25 years later and with a 100x+ increase in CPU power :)

xenospn• 7 years ago

Everybody knows the REAL test is a FreeBSD 'make world' anyway :)

basementcat• 7 years ago

I used to think 'make world' on BSD was some big deal until I started to synthesize HDL code to bitfiles or GDS.

jandrese2• 7 years ago

Compiling Firefox or Wireshark or Libreoffice all work the machine harder than a make world.

cesarb2• 7 years ago

The kernel has gotten bloated. Back in the day, it could easily run in 8 megabytes of RAM, with plenty left over for the userspace and cache. Nowadays, just the kernel code (before loading any modules) is already bigger than that.

l33tman2• 7 years ago

I was one of the first who worked on embedded Linux, and I put the kernel and a user-space program in a device with 512kb RAM once. It was too tight to do anything useful with, but just barely. It was no problem doing useful stuff on a 2 MB device (this was in 1999) :)

Though to Linux defence, it is not nearly as bloated over this period of time as, say, Windows, by orders of magnitude. And, you aren't required to compile in everything. It's just very tedious (and frankly a bit pointless) to trim it down..

ilaksh• 7 years ago

Nice. How long to build Chromium?

fortran77• 7 years ago

If you want to keep these compile times, better not rewrite the kernel in Rust.

Havoc• 7 years ago

Was recently googling what the Kernel guys were using out of curiosity. GKH mentioned that he had access to 32 core AWS instances for it....7 years ago.

Still...exciting times for consumers.

kstenerud• 7 years ago

Wow... I still remember recompiling the NetBSD kernel on my Amiga 3000 to tweak the timings on my Retina Z3. Took 14 hours to compile the kernel...

So if processors are now >1500x faster than they were in 1991, why is it that my Amiga 3000 has a faster, more responsive UI?

Why is it that an ssh session takes 8 seconds to establish?

Why is it that browsing a network share is 10x slower than viewing a file list was on a BBS?

fooker• 7 years ago

Because not everything is CPU bound.

yellowapple4• 7 years ago

But all the other bounds have improved massively, too:

- CPUs are multiple orders of magnitude faster

- GPUs are multiple orders of magnitude faster

- RAM is multiple orders of magnitude faster and more abundant

- Disks are multiple orders of magnitude faster and bigger

- Network connections are multiple orders of magnitude faster

Why is it that the end result feels the same?

The answer, of course, is that software complexity has kept pace: software, like a gas, expands until it fills its container.

vbezhenar• 7 years ago

Something's wrong with your DNS setup, I guess. SSH session takes less than one second to establish for me.

jandrese2• 7 years ago

Sounds like you have a slow network and are using protocols optimized for high throughput on LAN connections.

dr_dshiv• 7 years ago

You are getting good replies, but the main answer is... nobody knows (and you aren't alone). There are so many little reasons for slower modern computers, but it seems to come down to increased complexity and, I guess, entropy? It's frustrating.

ajford• 7 years ago

Regarding your SSH session, have you turned off gssapiauth (assuming you aren't using it)?

See https://coderwall.com/p/fukoew/speed-up-ssh-logon-by-disabli... for some details.

CodeWriter23• 7 years ago

How many -b bits did you specify for ssh-keygen?

And today’s software is more than 1500x the size.

ralgozino• 7 years ago

Edit your sshd config and add useDNS: no to it, don't forget to restart the deamon.

Either way, I get your point, I remember programming in Delphi, compiling, chatting on IRC and browsing the net with a 56K Winmodem (all the modulation was software driven) with a Pentium 2 all at the same time, and now I can't even have and IDE and slack open at the same time.

ncmncm• 7 years ago

IDE? Slack? "There's your problem right there."

GordonS• 7 years ago

I'm old enough to remember 14k, 28k and 56k modems - and while yh internet held great wonder, waiting for pages to load and files to download was really frustrating.

Oh, and having to hangup so my parents could make a phonecall wasn't fun either!

This at least is something that has got orders of magnitudes better, and is very much noticeable. At times I still shake my head in wonder at being able to download a GB in less than 2 mins on a residential FttC connection.

ddevault• 7 years ago

LLVM in under 2 minutes is way more impressive. That's a crazy CPU.

beagle3• 7 years ago

It's a beast, but ... that's not a very impressive measure. Bellard strikes again -

From [0], in 2004: TCCBOOT is a boot loader able to compile and boot a Linux kernel directly from its source code. It is only 138 KB big (uncompressed code) and it can compile and run a typical Linux kernel in less than 15 seconds on a 2.4 GHz Pentium 4.

[0] https://bellard.org/tcc/tccboot.html [2004]

anticensor• 7 years ago

I would prefer gccboot (with gcc -march=native -mtune=native) though, because tccboot cannot optimise the output.

Threadripper 3960X Compiles Linux Kernel in Under 30 Seconds

Related Stories