The Explosion of the Ariane 5

http://www.ima.umn.edu/~arnold/disasters/ariane.html

by pstadler • 13 years ago

87 58 13 years ago

ima.umn.edu

eps• 13 years ago

Reminds me of the (alleged) reason why first Soviet Mars missions missed the planet - there was an erroneous period instead of a comma at some part of its nav program written in Fortran.

jacquesm• 13 years ago

Reminds me of the (confirmed) reason why US mission http://en.wikipedia.org/wiki/Mars_Climate_Orbiter hit the planet at the wrong angle and desintegrated in the atmosphere - there was a mix-up in imperial and SI units.

FoeNyx• 13 years ago

or was it NASA Mariner 1 ? http://en.wikipedia.org/wiki/Mariner_1#Overbar_transcription...

Anyway, as always wikipedia has an interesting list of software bugs : http://en.wikipedia.org/wiki/List_of_software_bugs

I found interesting the computer crash of F22-Raptors after crossing the International Date Line.

huhtenberg3• 13 years ago

They don't have a Mercedes Smart bug where they mixed left and right causing the car to throw itself on its side when going through a turn.

(edit) ... and I'm downvoted. Lovely. Totally makes sense.

Aardwolf• 13 years ago

Don't they test these programs in virtual cases with simulations of realistic data, speeds, angles, altitudes, etc...?

ctdonath3• 13 years ago

There was a very expensive stock-trade mistake when someone issued a sell order, typing 'b' instead of 'm' for 'million'. Market dropped a significant fraction in seconds.

fr0sty• 13 years ago

If you are talking about the US "Flash Crash" that theory is little more than urban legend.

Here is one of the "b" instead of "m" stories published on the day of the crash:

http://www.cnbc.com/id/36998463

Here is the Wikipedia summary of the event which lists "fat finger trade" as one of the discredited theories:

http://en.wikipedia.org/wiki/2010_Flash_Crash#Early_theories

Uncompetative• 13 years ago

Incorrect.

According to Expert C Programming : Deep C Secrets by Peter Van Der Linden page 61:

    'Table 2-2 The Truth About Two Famous Space Software Failures

    WHEN            MISSION     ERROR                        RESULT                               CAUSE

    Summer 1961     Mercury     . used instead of ,          nothing; error found before flight   Flaw in Fortran language

    July 22, 1962   Mariner 1   "R" instead of "R̅"           $12M rocket and probe destroyed      programmer followed error
                    (to Venus)  written in specification                                          in specification'

I have no idea whether the formatting of this table will be preserved, its inclusion is to save you looking for the book, you could either download the following .pdf and search for 'mariner' due to the lack of page numbers

http://www.e-reading-lib.org/bookreader.php/138815/Linden_-_...

or just use what little Google Books provides

http://books.google.co.uk/books?id=4vm2xK3yn34C&q=fortra...

As you can see neither mission involved Mars and the faulty . being used instead of a , was found before it caused any harm.

More information follows on the failed Ariane 5 launch - which Bertrand Meyer attributes to "insufficient specification":

http://se.inf.ethz.ch/~meyer/publications/computer/ariane.pd...

http://en.wikipedia.org/wiki/Ariane_5_Flight_501

XorNot• 13 years ago

I'd really love more details about this. What did the surrounding code look like, why wasn't there a compiler warning being produced by this code etc.

There's certainly a much larger - and probably quite informative - story here.

lmm• 13 years ago

It was ADA, there was a trap that would have caught it but had been disabled (I believe for performance?) AIUI the decision to disable it had been made when it was Ariane IV code (and if an Ariane IV had ever been travelling that fast then it would have already been lost) - and then never reevaluated when the code was reused for Ariane V.

david_p2• 13 years ago

A friend of mine did a presentation about this. What he told me is that apparently, the developpers who wrote and tested the code that overflowed were designing for a value in miles.

When stored in miles, the value would never got outside the range of a 16 bit unsigned int, but the actual value used was in kilometers, and when converted to kilometers, the value would overflow.

marco-fiset• 13 years ago

If a simple multiplication by 1.60934 (miles to kilometers) can create an overflow, than you are not using a big enough data type. I can't believe that's the real reason.

david_p2• 13 years ago

Apparently I mixed up the story with Mars Climate Orbiter ...

More details about the Ariane 5 failure here: http://people.cs.clemson.edu/~steve/Spiro/arianesiam.htm

Tloewald• 13 years ago

This seems unlikely for a European/French system. Are you perhaps confusing this story with the NASA Mars Observer mission, mentioned elsewhere in the thread, which failed because of a metric vs imperial error?

The value was horizontal velocity. I imagine in metric it would be expressed in m/s while in imperial it's going to be feet per second (mph seems highly improbable).

ctdonath3• 13 years ago

The Mars Climate Orbiter mixed up feet with meters. Seems it calculated how far off the surface it was during landing, and when it decided it was at 0 units above the surface the retro-rockets were shut off as intended. Except that it was, in fact, still a few miles up. Code re-use struck again.

wazoox• 13 years ago

Indeed there are code samples on the WP page: https://en.wikipedia.org/wiki/Cluster_%28spacecraft%29

ctdonath3• 13 years ago

I've told the story to my students many times (probably mangling some minor details over the years, but getting the point across).

AFAIK (correct me if wrong)...

- The code was re-used from the Ariane 4, which was designed for launch only in the northern hemisphere. The Ariane 5 was launched from south of the equator, hence the importance of unsigned vs signed.

- A run-time warning was generated. Problem was where to put the warning, which wasn't specified, and which thru dangling pointer overwrote current position/velocity data. Watchdog code noticed the now-impossible trajectory, decided the system was very very confused, and initiated self-destruct for (relative) safety.

InclinedPlane7• 13 years ago

Both Ariane 4 and Ariane 5 launch from the same facility (in Kourou, Guyana, almost right on the equator (5.2 deg. N latitude)). The specific problem was that the Ariane 5 had more horizontal motion than the Ariane 4. Within the code there was a conversion of a 64-bit float to a 16-bit signed integer, which overflowed on the Ariane 5 flight causing the inertial measurement unit to start spewing out diagnostic information which the flight control software merrily interpreted as data and drove the vehicle off course.

callahad• 13 years ago

The Therac-25 is also a fascinating case of software failures causing tangible loss: http://courses.cs.vt.edu/cs3604/lib/Therac_25/Therac_1

nraynaud• 13 years ago

You know, also I'm slightly tired of that story (I mean it stings, my family works in the field), sometimes I feel like it's a good thing. Here in France, and with the elite political clique at the power even more, are afraid of risk. Our constitution was even emended towards risk-averseness. I think blowing up the GNP of an african country had various positive side effect: 1) risk is there, wether you have correctly signed the process paperwork or not 2) innovation feeds on blowups 3) be humble, stop being cocky on TV before a test launch, sending back the champaign and buffet was very painful to watch and there is no need for that.

avar• 13 years ago

They should have celebrated that they learned something that day with that champaign and buffet.

webreac• 13 years ago

"R1...More generally, no software function should run during flight unless it is needed."

This means, that even using the most reliable language and trying to test as much as possible, there is always a risk of an overseen bug.

troymc2• 13 years ago

The Ariane 5 rocket was carrying four ESA spacecraft known as "Cluster" (because they were to work together, in a tetrahedral formation). The bug and subsequent failure give another meaning to the word "clusterf%#k".

https://en.wikipedia.org/wiki/Cluster_%28spacecraft%29

Edit: The above Wikipedia article has the Ada source code that caused the problem.

obviouslygreen• 13 years ago

+1 for excellent reverse etymology.

tobych• 13 years ago

I worked on the Cluster project as a software developer for many years, at the University of Sussex in Brighton. My first job. And it blew up. Great first job. I was watching live with hundreds of engineers and scientists at Rutherford Appleton Laboratory. There was silence. All we could hear were birds singing, coming over the satellite link. Quite an experience. Amazingly, the Cluster project got up and running with replacement hardware a few years later, and I was back on the job.

smackfu• 13 years ago

Don't pretty much all launches have insurance? The rate of failure is high enough that it seems necessary unless you are a gambler.

troymc2• 13 years ago

Indeed, Cluster II was a brilliant success, with over 10 years of successful scientific operations in space. Kudos for any part you played in that.

youngerdryas• 13 years ago

From the linked James Gleick article:

"the programmers had decided that this particular velocity figure would never be large enough to cause trouble. After all, it never had been before. Unluckily, Ariane 5 was a faster rocket than Ariane 4. One extra absurdity: the calculation containing the bug, which shut down the guidance system, which confused the on-board computer, which forced the rocket off course, actually served no purpose once the rocket was in the air. Its only function was to align the system before launch. So it should have been turned off. But engineers chose long ago, in an earlier version of the Ariane, to leave this function running for the first 40 seconds of flight -- a "special feature" meant to make it easy to restart the system in the event of a brief hold in the countdown."

michielvoo• 13 years ago

That seems like a contradiction: it was caused by a calculation regarding velocity, but that calculation served no purpose once in the air.

Based on the quote I* have to agree with the developer then: this particular 'horizontal velocity' figure would never be large enough to cause overflow, it would always be zero since the rocket should still be on the platform.

So maybe the existence of the routine was the root cause, and not so much the potential for overflow inside the routine?

* I am not a rocket scientist

Martimus• 13 years ago

Yeah it certainly seems like that based on the way it's written. However, it does say "horizontal velocity of the rocket with respect to the platform".

Perhaps it had something to do with calibration prior to launch. The report states: "The Operand Error occurred due to an unexpected high value of an internal alignment function result called BH, Horizontal Bias, related to the horizontal velocity sensed by the platform. This value is calculated as an indicator for alignment precision over time."

InclinedPlane7• 13 years ago

> "this particular 'horizontal velocity' figure would never be large enough to cause overflow, it would always be zero since the rocket should still be on the platform."

Except the Earth is not stationary, nor is the surface of the Earth moving at a constant velocity.

nernst• 13 years ago

There is a good article examining the various possible causes of the Ariane-5 disaster by Bashar Nuseibeh: "Ariane-5: Who-Dunnit?". See PDF here: http://www.inf.ed.ac.uk/teaching/courses/seoc/2007_2008/reso...

Gravityloss• 13 years ago

I've heard from people in the space sector that it was the exception, not the overflow per se that caused the problem. Had it not been caught the flight could have made it to orbit (if there weren't other problems). Wikipedia says it was a hardware exception but http://www.ima.umn.edu/~arnold/disasters/ariane5rep.html says it was a software one, and it was only in code that was needed in pre-flight so it seems likely to not cause problems if there wasn't that crippling exception.

These systems have become so big and expensive. This was the case since ICBM:s and it got only worse with Apollo.

Yet they are so vulnerable since there is no way to abort intactly once you have flown something like 0.1 seconds. (At least Saturn V had some redundancy.) You do not get a second try.

Both issues create a perfect recipe for stagnation - everything has to be checked and rechecked for years before and after a software or hardware change. If someone tries something new, and there is a launch or spacecraft failure, it is a political issue and heads will roll. People's technical and political careers are destroyed.

In short, this way is not likely to reach real spacefaring.

A more organic approach with lots of smaller actors working in parallel and trying and failing a lot more - but with better processes built in to handle said failures (technical, political and cultural) could be much more conducive to real progress like increase in operational flexibility, shortened schedules, better reliability and lowered price.

Reasonable sized reusable rockets with good intact abort capability in a testing and development program could up the launch rate hugely, and all kinds of different solutions could be quickly tested. I find it likely that this will eventually happen, but it is frustrating how long it is taking.

In this "horizontal velocity overflow" case, you could do an intact abort if you had a fallback to some alternate control law or even manual control. Those are not incorporated to current expendable space launchers but they exist in aircraft. (Saturn V and also the Lunar Module did have manual backup. You could fly the Saturn to orbit. The LM was hard got get to the right orbit where the CM was waiting...)

neurotech1• 13 years ago

There has been several references that SpaceX "fly" their Falcon 9 computer systems to test for bugs like this. The idea being that as far as the computer is concerned, it is a real flight and should act accordingly. Most of the problems to date, have been related to a mechanical problem. During the first docking, there was a minor issue with the sensor "field of vision" but this was fixed.

The point is that SpaceX procedures seem to be able to prevent similar software bugs in the Ariane 5 from causing a catastrophic abort or failure.

kiba2• 13 years ago

How come the flight system at Ariane 5 wasn't tested like this?

InclinedPlane7• 13 years ago

Resources? Afterward they simulated flights using the Ariane 5 systems and duplicated the errors.

axusgrad• 13 years ago

They have to fool the sensor inputs to look like a real flight, right? They can't necessarily think of every realistic but unexpected input. Definitely worth trying, though. I'd actually be surprised if they hadn't done that with the Ariane.

jaxb• 13 years ago

For more stories like this, get the book by David M. Harland, "Space Systems Failures: Disasters and Rescues of Satellites, Rocket and Space Probes"

3327• 13 years ago

happens to the best of us... If its any condolence my first game app crashed after 10k points for a similar reason. check it out it should still be on the android store - Alliegator

jhonkola• 13 years ago

Another (probable) software failure due to unexpected scenario was the Mars polar lander http://en.wikipedia.org/wiki/Mars_Polar_Lander#Loss_of_commu...

The failure review concluded that the probable cause of loss was that the landing system software apparently interpreted the deployment of lander's legs as touchdown and shut down the descent engines. The vibrations caused by the deployment of the legs was not taken into account when designing the software.

flagnog• 13 years ago

compare this to the failures experienced recently with SpaceX: Elon's launch, while not perfect, recovered. I think this shows the power of E's vision, and how he's going to change the launch market.

The Explosion of the Ariane 5

Related Stories