FASTRA II: the world’s most powerful desktop supercomputer

http://fastra2.ua.ac.be/

by jacquesm • 15 years ago

79 36 15 years ago

fastra2.ua.ac.be

wunki• 15 years ago

Although the computer itself is reason enough to be impressed, I must admit I more impressed with the use of the "super" computer. Beautiful that us geeks can contribute so much to every other discipline.

Immediately, back in my head, I got this gnarling feeling that I should also use my skills in such life improving projects instead of building the next social _______ life drain.

h0h0• 15 years ago

there are several, relatively well-paid, phd positions available at the vision group ;)

jvdb• 15 years ago

(keep in mind, 2009)

DiabloD39• 15 years ago

How is it the world's most powerful if its loaded with Nvidia cards?

200 series era Geforce cards were slower per watt and per dollar and per slot (all important here) than 58xx series Radeons on both single precision and double precision math (and integer as well, but integer is only useful for crypto and Radeons have native single cycle instructions common for crypto which makes them about 4-5x faster than Geforces, but not a typical use case).

And if we're comparing series 400/500 to 69xx, the beating is much worse.

risratorn• 15 years ago

Probably because some hasn't yet assembled a similar system with ATI cards? They claim they have the fastest desktop system and it probably is at the time of writing but as far as i can tell they didn't say it couldn't possibly be better/faster. There's always room for improvement no? :)

sp332• 15 years ago

Nvidia and ATI cards have (approximately and with a few exceptions) the same performance per dollar when it comes to 3D graphics applications. But the tradeoffs made internally mean that they have radically different performance in other areas. For example, ATI chips are faster then Nvidia for bitcoin mining (edit: lots of sha256 hashes), but Nvidia is generally faster than ATI at physical modeling.

http://ewoah.com/technology/a-very-good-guide-to-building-a-...

http://stackoverflow.com/a/4642578/13652

All this is moot if the application they're running only supports CUDA - then they're stuck with Nvidia.

DiabloD39• 15 years ago

Author of DiabloMiner, the worlds most popular GPU miner, here... I think you mean SHA256.

If they're stuck on CUDA, I feel sorry for them. They'd be better off rewriting the app to use OpenCL to take advantage of AMD's superior chip design and not be stuck on Nvidia/Windows (Nvidia Linux support, especially under CUDA, is horrible; rather depressing once you consider most CUDA clusters run Linux).

vilya2• 15 years ago

Because no-ones bothered to build one using the ATI cards, maybe?

I would guess the main attraction of Nvidia cards was probably CUDA, which is still a nicer environment to program for than OpenCL and seems to have a more lively ecosystem of libraries, tools, etc.

Also, floating point & integer math aren't the only relevant performance metrics. For their application domain, memory bandwidth & latency would probably be at least as important. Kernel switching time may be a factor too - there are many things you can't do with a single kernel. I don't know (but would be interested to hear) how the cards stack up against each other in those terms.

DiabloD39• 15 years ago

I'm not sure how you can claim CUDA is nicer. CUDA is not a public specification, it has poor documentation, and it only works on Nvidia.

I am a software engineer (or whatever the popular term this week is) who uses OpenCL frequently. I couldn't imagine being stuck on something like CUDA, between the C++ obsession and the fact it follows Nvidia Cg's way of syntax design instead of GLSL (OpenGL Shader Language) makes it difficult to use.

In addition, there never will be a native Linux implementation that is open source as Nvidia has basically tried to screw over Mesa/Gallium repeatedly with vague IP rights threats (see also, Nouveau, the X/Gallium driver for Nvidia cards, and how they unofficially threatened to sue if Nouveau continued reverse engineering hardware interfaces; they backpedaled when it got media coverage).

So, I dunno about you, but I don't like depending on companies that continuously screw developers who made the mistake of supporting their hardware.

OTOH, AMD has paid over $2 million in their open source efforts including having their legal department clear the programming manuals for use by X/Mesa/Gallium and also hiring multiple full time developers to work on the open source drivers, and has clearly supported FOSS at every turn.

microarchitect3• 15 years ago

When I was working for AMD, I heard from the GPU guys that one of big the reasons they weren't seeing a lot of HPC-space adoption for the Radeons was because they didn't (yet) support ECC for the DRAM.

ansgri• 15 years ago

A bit offtopic, but we of image processing do use integers a lot: they're fast, single SSE instruction processing whole 16 uint8s at once.

DiabloD39• 15 years ago

Except video cards do image processing every single day. Modern game engines use floating point pipelines the whole way through, the only integers in the GPU typically being texture data and the final output to the screen.

SSE2 integer usage may be fast, but GPUs are faster, typically by several magnitudes.

Quequau• 15 years ago

They've had enough time to build a FASTRA III since this was released...

ksadeghi• 15 years ago

It looks like they have 7 cards in there. If all the GPU's in that system where pegged at 100% for 30 minutes that system would overheat. There isn't enough space between the cards for free air flow.

7 x 5970's would give you just under 35 Teraflops instead of 12.

DanBC• 15 years ago

They have thermal image camera video:

(http://fastra2.ua.ac.be/?page_id=53)

They claim "slightly over 60 degrees".

DiabloD39• 15 years ago

To be fair, 5970s are dual GPU cards and would probably overheat packed in like that. 5970 have two 5870s, but are downclocked so the performance is closer to two 5850s.

8 5870s packed into two cases would survive if you keep the cases closed and put 3 Delta AFB1212s in the front pushing all the air through the cards and out their rear vents, this is how most Bitcoin miners build cases.*

If done right, 8 5870s should easily be kept under 85c even when marginally overclocked.

* Large scale Bitcoin miners have a habit of not using cases and use flexible risers to keep the cards away from each other out in the open. I do not recommend this as you lose out on high pressure cooling, a staple of enterprise/high performace computing.

hippich• 15 years ago

I used water cooling. With 5970 it do not adds too much to price, but helps keep cards cool in tight environment. Specifically, I had microATX case with two 5970 at 850 Mhz (it could handle more, but just wanted to feel safer). GPUs were staying at 55-65 C. I bet you can do better if you route separate water tubes to each card (which I did not because of tight environment and being lazy :))

Before adding water cooling - it would lock after 30 minutes of mining at standard 725 Mhz.

This system worked very well for ~4 months (and then I sold it to guy from Alaska, since it became too hot in Texas :))

SilasX• 15 years ago

FWIW, I set up a bitcoin mining rig with four 5870s by using liquid cooling. Full story here: http://silasx.blogspot.com/2011/12/exclusive-silass-bitcoin-...

microarchitect3• 15 years ago

Calling this a "supercomputer" is a bit disingenuous. The most powerful supercomputer today can do 10.5 Petaflops on the Linpack benchmark [1]. The 500th most powerful supercomputer can do 51 Teraflops.

If the authors actually tried to measure Linpack scores on their cluster, they'd find that peak performance would be much worse than the 12 teraflops they're claiming as "peak" performance. This link [2] seems to indicate that one S2050 along with a fairly fast CPU gives you about 0.7 teraflops. Even if they get perfect scaling across GPUs, which they won't, and if they could solve all the heating issues with their design, they won't get more than about 0.7*6.5 = 4.55 TFLOPS peak on linpack.

In any case, many (39 out of 500 to be precise) of the most powerful supercomputers in the world also use GPUs to accelerate certain applications so there's simply no way this thing would be faster than those computers.

[1] http://www.top500.org/lists/2011/11/press-release [2] http://hpl-calculator.sourceforge.net/Howto-HPL-GPU.pdf

willvarfar• 15 years ago

cough 12 terraflops, not petra

microarchitect3• 15 years ago

Oh right. Thanks for catching that typo.

tycho77• 15 years ago

One S2050 has four M2050s, not two (I inferred this assumption from your 6.5 multiple).

I'd like more information on the CPUs and CPU/GPU bus of this FASTRA II. CPUs are really gaining back a lot of their performance relative to GPUs - an M2090 is only about 5-10x faster than the newer Intel Xeons in a properly multithreaded program that pays attention to the NUMA nodes. This is efficient enough that for large simulations which do not fit on one GPU, running portions of the simulation on CPU easily overcomes the CPU/GPU communication overhead in terms of performance.

VladRussian• 15 years ago

>I'd like more information on the CPUs and CPU/GPU bus of this FASTRA II.

"the graphics cards, which are connected to the motherboard by flexible riser cables." - so this sounds like typical PCIe based design. Usually the bundle of GPU cards on PCIe backplane is placed into separate case which connected back to the CPU host, and in this case they engineered it into one big case thus each card getting it's own x16/x8 pipe.

In general such "supercomputer" is cheaper, hardware-wise, than equivalent CPU-based, yet it is still more expensive human labor (ie. programming) -wise.

scott_s• 15 years ago

Agreed, thank you. "Supercomputer" is a continually moving goal-post that basically means "more computing power than you can achieve with personal computers." Supercomputers push the limits of the performance that we can achieve. If you can fit it inside of a PC case, then you're not pushing the limits of the kind of performance we can achieve.

FASTRA II: the world’s most powerful desktop supercomputer

Related Stories