World’s top supercomputer from ‘09 is now obsolete, will be dismantled

http://arstechnica.com/information-technology/2013/03/worlds-fastest-supercomputer-from-09-is-now-obsolete-will-be-dismantled/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+arstechnica%2Findex+%28Ars+Technica+-+All+content%29

by ari_elle • 13 years ago

107 46 13 years ago

arstechnica.com

naftaliharris• 13 years ago

Something about these numbers doesn't quite make sense. The reason cited for dismantling the machine is that "it isn't energy-efficient enough to make the power bill worth it." But the supercomputer uses 2345 kilowatts, which at US prices of around 15 cents per kWh would cost $352 / hour to run in energy costs. By comparison, the $120 million cost of building roadrunner, amortized over the four years it's been running, comes out to $3400 / hour. The article makes it sound like the power bill is costing them a fortune, at $3 million a year, it isn't that much at all next to the $120 million price tag.

justincormack• 13 years ago

They probably need the space for a faster one...

ghshephard• 13 years ago

A 22,000 square foot data center costs about $5mm to build [1].

And these data centers can be located wherever is convenient. And clearly, forward planning is not an issue when we are talking $100mm+ investments.

[1] http://www.reedconstructiondata.com/rsmeans/models/data-cent...

miahi• 13 years ago

They already have the Titan, four times more efficient, with 17 pflop consuming only 8000 kwh.

FollowSteph3• 13 years ago

But at what price amortized over the same length of time?

davidmr2• 13 years ago

Titan is for open science, not classified research as roadrunner was. Sequoia is the new classified nuclear supercomputer.

sliverstorm2• 13 years ago

The cost of building the thing has already been paid. It is a sunk cost. If it costs $352/hour to run the roadrunner and they can only get $150/hour in income to operate it, it don't matter how much it cost to build it.

OGinparadise• 13 years ago

If it costs $352/hour to run the roadrunner and they can only get $150/hour in income to operate it, it don't matter how much it cost to build it.

Assuming they already paid for the replacement super-computer too etc. Otherwise, sure it matters. Instead of finding $xx-$xxx million to build a more efficient one, you might be better off to pay the electricity bill difference. This is still in the top 25 computers in the world, so it can be used for a lot of things, but we don't know all the details.

noahdesu• 13 years ago

The tri-labs have a rotating schedule for acquiring new machines, that include capacity and capability classes (the later being RR). The last time I was in the computing center at LANL the computing floor was quite packed. So, regardless of the claims made in the article for disposal reasons, there are other issues such as total machine weight, cooling requirements (which dominate), making room for new machines, etc...

I do not know if RR will be physically shredded, but I'm hoping that it is retired to the brand-new PRObE (http://nmc-probe.org/) computing center where RR would be available for use by academic researchers.

fintler• 13 years ago

PRObE will probably get a chunk of it, but they also have power limitations. Basically every piece of the machine has already been given to some researcher or another for some purpose -- there's a bit of a waiting list at the moment.

The hard drives will probably be sent to a military base to be melted down, I don't think they risk shredding them nowadays. The hardware that doesn't store data will probably be kept intact for quite some time.

Since Roadrunner takes up about 1/3 of the Metropolis datacenter, getting rid of it leaves some room for the 2015 "Trinity" machine (assuming LANL hosts it).

davidmr2• 13 years ago

Disclaimer: while I worked for a DOE supercomputing lab, it wasn't one of the NNSA (nuke) labs, so I don't have all the info. (And even if I did, I certainly wouldn't be allowed to comment on it.)

You're a bit right in that it's more complicated than they say in the article. The power of keeping it up and running is definitely costly, but there's more to it than that.

* It takes people to run it. They're expensive, and you don't want your people wasting a lot of time on the old super while your new one is being installed and accepted. I don't recall having seen any plans for a new super at LANL, but there's probably something new showing up soon.

* The architecture of roadrunner is outdated and obsolete. The NNSA isn't buying any more, and you can bet that with the new Blue Gene/Q at Lawrence Livermore (sequoia), all their new code is being written for different non-roadrunner architectures. There is considerable cost to porting new code to legacy hardware.

* Kind of in line with the last point, sequoia is >17x faster than roadrunner at only 3.5x the power. There's not a lot of point in paying such a premium in flops/watt by running on roadrunner since the new one is already online.

* 2.3MW is only what the computer itself uses. There's a ton of other supporting equipment needed to use it (chillers, disk, control nodes, etc.)

* This computer isn't being manufactured anymore, so maintenance is going to cost a fortune. When they bought it, they either paid for a hardware support contract or bought a ton of spares and did self-maintenance. If they did the former, IBM is either charging through the nose for replacements or refusing to support it outright. If they did the latter, they're probably running low on spares.

cbhl• 13 years ago

This is the "sunk cost fallacy" -- see http://en.wikipedia.org/wiki/Sunk_costs#Loss_aversion_and_th...

If it's cheaper to build (or rent) and run a new supercomputer, then it makes sense to dismantle the old one.

masklinn• 13 years ago

The $120m is sunk costs from a previous building budget, it's not relevant. The more relevant piece is more recent supercomputer (with architectures which are not dead ends) being 4 times as power efficient.

brudgers• 13 years ago

The 2.5 megawatts Road Runner consumes are not available for other purposes, such as running more efficient computers. In other words there is an opportunity cost associated with a power budget.

Retiring Road Runner may be significantly cheaper than adding additional megawatts of capacity, particularly when considering redundant power services and backup power supplies.

m_mueller4• 13 years ago

There's one thing you're forgetting here: Cell architecture. Noone writes software for that anymore, since it's been discontinued (either officially or inofficially). Most new supercomputers at that scale are powered by a mix of NVIDIA Tesla (with Intel MIC now coming as well) and Opterons/Xeons. This dictates the software stack of present and future research[1]. There's no reason to keep a computer running when it's severely underused.

[1] Plug: That's also what I'm working on currently: https://github.com/muellermichel/Hybrid-Fortran

orangethirty• 13 years ago

Interesting project. Do you have a blog?

peterjancelis• 13 years ago

An economic decision only takes into account marginal expenses (and income for that matter).

At this point the 120M investment is a sunk cost. And the investment was so bad that the operational income can not even cover just the operational expenses, so shutting it down is the wise thing to do.

danielrhodes• 13 years ago

Actually if you compare the Roadrunner against what it would cost to get the equivalent horsepower on Amazon (hypothetically, of course), that price tag is quite a deal.

n00b101• 13 years ago

RIP Cell processor.

hamoid• 13 years ago

"At more than one quadrillion floating point operations per second..." - how fast is that when cracking typical passwords or mining bitcoins? If we have an encrypted drive, how long would it take to find the passphrase with it?

dmm• 13 years ago

Bitcoin uses double SHA256 which is all integer operations, so that's not directly comparable to flops.

As far as I know most passwords hashes are also integer operations.

The cell processor in the ps3, despite being optimized for floating point ops, can do about 22.23 Mh/s. Assuming each of the 12,960 PowerXCell 8is in Roadrunner can do the same: 12960 * 23.23 ~= 288100 Mh/s which would earn you about 21.6 BTC/day at the current difficulty. That's about $2000/day at current prices.

The PowerXCells also have 8 SPUs vs 6 in the ps3, so they are potentially 25% faster.

moconnor• 13 years ago

For bitcoin mining you probably want to use Titan's 56k GPUs. I've run jobs at full scale on Titan, sometimes I forget it can be pretty cool to be in HPC!

DannoHung• 13 years ago

Well, that's four years. Only a year ahead of normal amortization schedules.

supervillain2• 13 years ago

Oh I wish I have a supercomputer and could play with one right now.

supervillain2• 13 years ago

I wonder what's wrong with this comment, I get downvoted for wishing out loud for a supercomputer to code on?

rwg• 13 years ago

It's interesting to read articles like this since I came from bizarro not-so-HPC world where "we" (academic department) didn't pay for electricity (the university did!), so there was no incentive at all to retire obsolete hardware. Right up until I left many months ago, I was running jobs on a cluster made up of 84 servers on death's door, each with dual-processor (not dual-core!) Nocona Xeons or Opteron 240s.

"Oh, but there's a cost to support obsolete hardware!" Yeah, sure, but the person supporting everything was me, and I was a constant cost to keep around whether I supported crappy obsolete hardware or shiny new hardware.

happycube• 13 years ago

Yup, if you're not paying for power or space you can afford to keep those dinosaurs around, if they can do the job. Which they might still be able to do, just nowhere near as well as a Sandy Bridge E platform.

Heck, some of the developers might prefer to use a Haswell+7990 in a few months over time on those things... if they can get one on their desk.

Meanwhile in the colo world, everything up to Core 2 Xeons were being tossed out in lots when Nehalem came out because it was just that much more efficient.

m_mueller4• 13 years ago

In terms of age this system isn't a dinosaur - the problem is the hardware architecture as stated in the article: Cell is now obsolete, noone wants to program new software for it anymore. X86 systems on the other hand, even older ones, can still be targeted by new Software without problems, so it may still make sense to keep them. To be blunt, the accelerator market simply moved to NVIDIA Tesla.

kislayverma• 13 years ago

What better way to spend the public dime than this.

auctiontheory• 13 years ago

How many years till everyone's cell phone has this much computing power?

kragen2• 13 years ago

Let's suppose current medium-spec cellphones have a gigahertz CPU, and that it can do one flop per cycle, peak performance. That's probably a tenth of a flop per cycle at LINPACK, which is how the Top500 list is ranked. So that's a hundred megaflops in your cell phone today, guessing.

This was the first petaflop machine. A petaflop is a million gigaflops, or ten million current cellphones. That's roughly 23 doublings in performance. If Moore's Law were to continue at one doubling per two years, that would be 46 years, or about 2059. Another ten or fifteen years might pass before low-end cellphones have it.

But 46 years is a long time. 46 years ago was 1967. There were no packet-switched computer networks. Nobody had ever seen a video arcade. Computers mostly interacted with people via punched cards, or at best a teletype. The Bloom filter had not yet been discovered. Timesharing and high-level languages were radical, risky concepts; the popular high-level languages of the time were all domain-specific, not general-purpose like C. There was no such thing as a mouse. There were graphical user interfaces and virtual reality. Humans had never walked on the moon.

46 years from now, there may not be cellphones, or humans. We certainly won't still be doubling the density of planar silicon chips every 18 months.

paulnechifor• 13 years ago

>Humans had never walked on the moon.

Yes, but they were 2 years away. We are now 41 years from the last moon mission.

davidiach• 13 years ago

Not sure we will still have cellphones 46 years from now.

Elhana• 13 years ago

flops = floating operations per second, why all this tech articles keep using terms like "petaflop"? It is not plural, it doesn't make sense.

World’s top supercomputer from ‘09 is now obsolete, will be dismantled

Related Stories