Toyotas Unintended Acceleration and the Big Bowl of “Spaghetti” Code (2013)

http://www.safetyresearch.net/blog/articles/toyota-unintended-acceleration-and-big-bowl-%E2%80%9Cspaghetti%E2%80%9D-code

by UberMouse • 11 years ago

152 144 11 years ago

safetyresearch.net

btilly• 11 years ago

Please add (2013) to the title.

People may have forgotten this because it was in the news a few years back. But Toyotas randomly experienced "unintended acceleration" due to software bugs.

Many reported incidents were undoubtably due to the problem that people in a panic can step on the wrong pedal and will misremember what they did. Others were due to a floor mat that could jam the pedal. But some were Toyota's fault.

nikanj• 11 years ago

Was that ever proven? Did Toyota ever come out with a software patch that had to be installed in all cars made by them?

I remember them replacing carpets that might get stuck, and this http://www.thetruthaboutcars.com/2010/03/the-best-of-ttac-th...

tomcorrigan• 11 years ago

It was never proven because it never happened. The wikipedia article has much more detail.: http://en.wikipedia.org/wiki/2009%E2%80%9311_Toyota_vehicle_...

It is not merely coincidental that all unintended acceleration events occurred in automatic cars. Also, a widespread unintended acceleration problem has not been observed in Toyota cars outside the USA.

It's a shame that so many on HN are so swift to condemn Toyota

mannykannot• 11 years ago

I, too, am left wondering over the paradox of the brake supposedly being powerful enough to hold back any amount of torque the engine and transmission can apply - unless, in the name of anti-skid braking or traction control, the system has the ability to override the driver and release or limit the braking torque.

Furthermore, those explanations of the floor-mat recall that I have checked all say that the mats were suspected of having interfered with the accelerator, not the brake. Toyota subsequently performed a second recall, to correct problems with the accelerator pedal. These both imply that unintended acceleration was at least a significant causative factor in the accidents - unless Toyota and the NHTSA were simply acting in order to be seen doing something.

With regard to proof, the article covers the issue in some detail: after a far-from-exhastive series of investigations, there have already been found so many ways for the software to get into unintended states that only a vanishingly small number of the vast range of possible causes have been considered. Asking for proof of an error is the wrong question to ask: the burden of proof clearly rests with the manufacturer to show that their systems are safe - and Toyota cannot do that with this software.

That Toyota replaced floor-mats is certainly not evidence for the correctness of their software, and nor is the non-appearance of a software patch. The article quotes a concerned Toyota engineer whose statements indicate a culture of denial within the company.

UberMouse• 11 years ago

Doh, all the reddit links were from today didn't realize it was from 2013. Updated the title.

swang• 11 years ago

So many questions remain...

How do I tell which Toyotas are affected? Do they all have these problems? What, if anything, did Toyota do to fix their software engineering processes?

I ask because my parents have a 2005 or 2006 Toyota Sienna, and I don't feel very comfortable with them driving it now.

MichaelGG• 11 years ago

If you're concerned about safety, look into the stats. With the number of accidents happening overall, I wouldn't be surprised if even a known flaw doesn't increase risk a whole lot. There's probably dominating factors, such as location (weather, etc.) and distance driven at which times, or which model of vehicle.

Maybe I'm wrong and this is a significant risk, but the approach should be the same. Even if they found and fixed one problem, the stats would still tell you which cars are safest. It could very well be that other vendors are worse, just haven't been investigated.

peterfirefly3• 11 years ago

Toyota cars are some of the safest on the road in the US according to the safety statistics -- even if (big if) some of them had problems with "unintended acceleration".

This tells us that even if (big if) the issue was real, it was actually never important. Also funny that it never was an issue outside of the US.

userbinator2• 11 years ago

Things were so much simpler when the only problems that could cause unintended acceleration were mechanical in nature. A broken return spring, seized linkage, etc. My daily driver has a mechanical throttle (although it is electronically fuel-injected), and honestly I can't see much point in "drive-by-wire" for cars.

But, on the other hand, even despite so many possible bugs, AFAIK no one has been able to demonstrate one instance of unintended acceleration even with extensive testing.

SoftwareMaven• 11 years ago

A 150 foot long skid mark seems like pretty compelling evidence of unintended acceleration.

spydum• 11 years ago

Have you never encountered such a bug? It isn't difficult to understand how hard a bug might be to track down when it only affects such a small subset of your users and only under specific circumstances. It is the dreaded Heisenbug

bri3d• 11 years ago

Drive-by-wire is good for a lot of efficiency technologies because it enables the elimination of the throttle plate, and therefore of many kinds of throttling losses (see MultiAir, ValveTronic, ValveMatic, etc.). It also enables simpler cruise control and traction/stability control systems. The addition of computers to cars is mostly a plus (IMO) but the hidden complexity is frightening to say the least.

thrownaway24244• 11 years ago

There are no mechanical throttles for electric cars.

hliyan6• 11 years ago

This is appalling:

    Toyota had more than 10,000 global variables.

    “And in practice, five, ten, okay, fine. 10,000, no, we're done. 
    It is not safe, and I don't need to see all 10,000 global 
    variables to know that that is a problem,” Koopman testified.

and:

    Toyota’s failure to check the source code of its second CPU, 
    supplied by Denso —even as executives assured Congress and 
    NHTSA that the cause of UA couldn’t be in the engine software

and:

    He was critical of Toyota watchdog supervisor – software to 
    detect the death of a task -- design. He testified that Toyota’s
    watchdog supervisor “is incapable of ever detecting the death 
    of a major task. That's its whole job. It doesn't do it.

    Instead, Toyota designed it to monitor CPU overload, and, 
    Barr testified: “it doesn't even do that right.

and:

    Barr also testified that Toyota’s software threw away error codes
    from the operating system, ignoring codes identifying a problem with 
    a task.

When the news first broke a few years ago, given Toyota's reputation for quality and process, I thought this was an American industry lead witch-hunt of a Japanese competitor. But if this testimony is correct, what Toyota engineers have done is unforgivable.

elmin• 11 years ago

I wouldn't be so quick to blame the engineers. They might be working within a managerial structure which doesn't give them the latitude they need to do their jobs. Yes engineers have a duty, but in software it's often hard to know exactly what the impact of cruft will be, and often all you can do is quit.

icelancer2• 11 years ago

Blow the whistle? Contact regulatory agencies?

hliyan6• 11 years ago

I agree, but I also believe that as an engineer I have an duty to refuse to build something under conditions that, once built, would cause harm to others. Or at least to relay my concerns to management in writing before proceeding.

And sometimes, quitting is the ethical thing to do.

e402• 11 years ago

And, at large Japanese companies, people are shuffled around every two years. I had heard this and didn't believe it until I saw it. A distributor of my company would replace people with novices once the previous person had just gotten really good at the job.

If Toyota is doing this, it would explain a lot.

shard972• 11 years ago

When the news first broke a few years ago, given Toyota's reputation for quality and process, I thought this was an American industry lead witch-hunt of a Japanese competitor.

You. You were one of the many people I had no clue where they came from. When I first saw the reports they seemed legitimate and Toyota seemed to be hiding something but so many people just didn't believe it and the news just sort of went under the radar.

hliyan6• 11 years ago

There's no need to be emotional. Some people jumped to biased conclusions. Others based it on NASA's software review (which we have now found to be flawed). See the report: http://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUB...

song• 11 years ago

After having lived and worked in Japan, this doesn't surprise me. Software engineering education is really weak there, managers who became managers because of their seniority tend to impose their own antiquated decisions and do not listen to best practices. The other problem is that status wise, it's considered lower level to be a software engineer than to be a mechanical or electrical engineer, so the smartest kids do not study computer science (this might have changed since).

ild2• 11 years ago

it's considered lower level to be a software engineer than to be a mechanical or electrical engineer, so the smartest kids do not study computer science (this might have changed since).

Which is very interesting, because in US it is completely opposite (and frankly it is not good either). Is it because US deindustrialized itself last 40 years?

rdtsc3• 11 years ago

There was an article about this (in New Yorker, I think...) a few years back, t how Japan has this strange relationship with software.

How it is a hardware-first country. Many reasons were given -- cultural, how "real men build real things", infatuation with appliances, which are seen as being "hardware" in a way, perhaps the langauge barrier i.e. existing software tools using English for menus and inputs etc.

And it is kind of interesting, looking back at a country that many in the 80's feared with surpass and leave the Western world in the dust technology-wise, we haven't seen that much software produced there. Can you think of large well known software from Japan (excluding console games). Vine Linux, SoftEther VPN, TrendMicro, anything else?

One wonders if they see Western software companies and if there is any desire to catch up or try to keep up.

Maybe anyone from Japan can provide a better perspective?

awongh• 11 years ago

This is also an interesting contrast to the (I assume higher quality) kind of work being put out by their other engineering departments- how can they make other sound engineering decisions on the rest of the car and it's manufacture, but fail at the software?

Is it something specific to software vs. traditional engineering in Japanese culture, or is it something inherent to software in general? Given the parent's complaints about workplace culture in general, shouldn't the entire car be badly designed?

abawany• 11 years ago

I can state that this matched my experience though I was in the US - this was in the 90s, and in the automotive industry. I used to work at Motorola Automotive and Industrial and our team built the Computer-Aided Manufacturing systems that managed the production lines. It always felt like a tier-2 job compared to process engineering.

enupten• 11 years ago

This is a genuine question: why can't they outsource their software devel ?

Niten• 11 years ago

When the news first broke a few years ago, given Toyota's reputation for quality and process, I thought this was an American industry lead witch-hunt of a Japanese competitor. But if this testimony is correct, what Toyota engineers have done is unforgivable.

These two possibilities aren't mutually exclusive.

Software engineering practices aside, was it ever demonstrated that unintended acceleration could actually arise from the behavior of this software?

Schwolop• 11 years ago

Yes, another part of Barr's testimony was a series of proof-of-concept demonstrations on the ECU to demonstrate the ability to kill tasks, leaving the hardware those tasks were controlling in unintended states.

bsder• 11 years ago

was it ever demonstrated that unintended acceleration could actually arise from the behavior of this software?

No, but the software engineering practices were so bad that it was a moot point.

tremon2• 11 years ago

was it ever demonstrated that unintended acceleration could actually arise from the behavior of this software?

That's entirely the wrong question to ask. The correct question is "has the system design been proven to make it impossible for unintended acceleration to occur?"

superuser2• 11 years ago

Toyota has a reputation for mechanical engineering quality and process which it probably still deserves.

Software engineering is apparently the opposite.

engi_nerd2• 11 years ago

“And in practice, five, ten, okay, fine. 10,000, no, we're done. It is not safe, and I don't need to see all 10,000 global variables to know that that is a problem,”

At an old job, my boss asked me to take a look at a business critical application to see if it could be improved upon. It had some deficiencies that were really hampering things.

I got the source from a coworker (this is when I worked on rockets, so none of us were or are professional software people, but even my dumb engineer self knows this is not how it should be). I opened up the folder. Lots and lots of files with little apparent structure. Ah, there was a file called "MAIN". I opened it. Visual Basic 6. Over 29,000 lines for global variable declarations alone. The actual program logic heavily used a commercial library designed to (shudder) make VB6 development more like making macros in spreadsheets. The original programmer then implemented the main program logic in this hellish abomination. The logic took up another 50,000 lines or so.

I told the boss that it would be easier to write a new program from scratch then attempt to understand it.

Has anyone seen a program with more globals than this?

r00fus2• 11 years ago

Sounds like you need to submit this to DailyWTF...

scrollaway2• 11 years ago

Nowhere near as many, but I had to deal with this very recently and this little WSJ gem showed up:

https://github.com/ewfelten/Tracking-Report-Card/blob/master...

I'll let the code speak for itself.

tankenmate• 11 years ago

I once worked for a billion dollar retail company. Their core back office applications (sales tracking, logistics, orders and fulfillment, staff rosters, etc) were written in COBOL and the inputs and outputs were read from / written to temporary files; kind of like pipes but via temp files in case of job failure. The real problem was that there were about 3,500 COBOL programmes and 2,500 shell scripts all in one directory. All the executables, COBOL and shell alike, were given a 4 digit filename; there was no up to date documentation on what program did what (apart from the source), or in which order they should be run, or the dependencies there in. To top it off end of day processing would take between 12 to 16 hours to run; any major problems and it would start affecting the next day's EOD. There was only one guy in the office (who had been there 15 years) who understood how it all fit together; when he died things got much worse (no he wasn't that old, how he died is another whole war story all together).

I only lasted there 6 months before I left for a start up, in many ways it was too much to bear. A while after I left taxation for retail changed substantially, there was a massive IT stuff up and the billion dollar business wound up being carved up and sold off; several thousand jobs affected, many lost their jobs either directly or indirectly as a result.

Sure this is probably at the worst end of the scale (short of causing deaths) and is rare but it does happen. Billion dollar company succumbs largely because of IT failures. It got to the point that the cost of the technical debt was higher than the carve up losses...

agumonkey• 11 years ago

It's a strange feeling when some small webdev are using clean modular testable code while real life businesses are abusing VB in creative ways. Really makes me feel paradigms do matter. My last job involved a huge pile of VB (VBA to be exact) code and again, it was an unstructured mess.

thrillgore2• 11 years ago

Yes, just about every damn PHP project I inherited that wasn't bound to a framework, or wasn't an existing OSS project.

coupdejarnac• 11 years ago

The last company I worked for outsourced a large firmware project to Denso based on their relationship with Toyota.

Turns out they spent 18 months writing unmaintainable code that barely worked, and the codebase had to be totally scrapped with product version 2.0. The code was chock full of cut and past, globals, and hacked mutexes- they did not even use the mutexes built into the RTOS.

Perhaps Denso stuck their B-team on the project, who knows.

_pmf_• 11 years ago

In C, there may be static variables with infinite lifetime which are locally scoped (module scope or function scope).

This is done to reduce stack usage; it does not mean that these 10000 variables can be accessed from everywhere else.

But I don't want to ruin the "hurr-durr, stupid C programmers" party that the rockstar full-stack webdevs here like to celebrate. Obviously, they know more about embedded software development than people who have worked in this field for several years.

leoedin• 11 years ago

The criticism of global variable count comes straight from Phillip Koopman, "a Carnegie Mellon University professor in computer engineering, a safety critical embedded systems specialist, authored a textbook, Better Embedded System Software".

So this isn't a web developer providing criticism, but someone with extensive experience in embedded software development. Perhaps reading the article linked before jumping to conclusions might be useful!

zb• 11 years ago

Locally-scoped (i.e. static) variables are, by definition, not global variables. The article is pretty clear that the 10,0000 figure refers to truly global variables, and I would hope that the original expert witness has not misled us by referring to static variables as 'global'. (Of course, without being able to see the source, there's no way to make our own judgement.)

Sadly, you would be surprised at the number of embedded systems that store and pass around their state using global variables, despite the obvious stupidity of that approach.

(For the record, I am not a web dev and I did indeed spend several years working in the field of embedded development.)

novaleaf• 11 years ago

I was a victim of this. Thankfully only minor damage resulted. I owned a Toyota minivan in Thailand and was in stop-and-go traffic, when suddenly the engine started revving up. I slammed on the brake and the engine downshifted to first to compensate, so I ended up bumping the car in front of me. After the bump (total stop?) the engine recovered it's sanity I guess, because the revving stopped at that point.

Naysayers will point out that the engine wouldn't rev up if I'm putting on the brake. That seems to be part of the symptoms of this bug. Oh, and I didn't accidentally push the accelerator instead of the brake, otherwise I wouldn't have just "bumped" the car in front.

Edit: I took the vehicle into the dealership and they said "user error".... whatever, no consumer protection in Thailand anyway, so I just moved on.

thaumaturgy3• 11 years ago

There are a bunch of neat comments from past threads about this if you search HN for "Michael Barr":

"On a cyclomatic-complexity scale, a rating of 10 is considered workable code, with 15 being the upper limit for some exceptional cases. Toyota’s code had dozens upon dozens of functions that rated higher than 50. Tellingly, the throttle-angle sensor function scored more than 100, making it completely and utterly untestable." https://news.ycombinator.com/item?id=7711771

"For example, http://www.edn.com/design/automotive/4423428/Toyota-s-killer... quotes Barr's claims: 'Toyota’s electronic throttle control system (ETCS) source code is of unreasonable quality.' 'Toyota’s source code is defective and contains bugs, including bugs that can cause unintended acceleration (UA).'" https://news.ycombinator.com/item?id=8906513 (and the linked article has a link to slides which are enlightening)

A number of comments at https://news.ycombinator.com/item?id=6636811, "Toyota's firmware: Bad design and its consequences"

PhantomGremlin• 11 years ago

There is, to me, a more interesting question:

If not Toyota, then who? Which auto manufacturers do it right? And is there any public evidence to support that? If not, then I'm probably just as "safe" or "unsafe" in any other brand of car as in a Toyota.

alexvoda• 11 years ago

This is mere speculation but the software in Tesla cars is probably at least decent since they dared reuse some of the technology on the Spacex Dragon V2 capsule.

I think this is one field in desperate need of open sourcing. Anyone who has the capability and the desire should be able to audit the software. Modifications should be prevented by signing, unless the user signs that he assumes full legal responsibility for the effects of the modifications and has the vehicle recertified as roadworthy.

Cthulhu_3• 11 years ago

Open sourcing it is nice and all from an idealist standpoint, but given the complexity of the software (millions of LOC), the fact that all the software would be used in commercial companies, and simply the area itself (cars), do you really think the community would pick up on it? I doubt it. Plus, the researcher did audit it - 18-20 months' work, and that was just analyzing it, not actually improving or fixing it.

Plus there is no car manufacturer ever that would allow anyone to change the firmware on their own.

bliti• 11 years ago

These are safety-critical systems. It still is mind numbing that the quality of the code is so bad. After this came out, I've assumed all modern fuel-injected Toyota's suffer from the same lack of safety. I don't think other automakers may be better.

I own a Toyota, but it has a mechanical throttle body. :)

plumeria• 11 years ago

Here are the slides Barr used in his presentation: http://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUB...

_ati• 11 years ago

Similar to the aerospace and medical may be there should be strict regulations in terms of processes to be followed for development in the automobile industry also. Usually in medical, failing the audit would restrict the producer to sell the product in market for 1 year. May be something similar could be enforced in automobile industry also. Ofcourse it might increase the cost of cars but atleast they will be lot more safer.

meesterdude2• 11 years ago

So, take this into account, and add in self driving cars. The complexity involved in these, and the coordination required, is nontrivial and its clear from the article they do NOT have a good handle on things.

I mean, most cars do seem to get around OK, so I'm partly amazed that under the hood, things could really be so scary. But we can't be lazy about adding complexity and systems, especially as more faith is placed on them.

That this is not NASA grade code (or anything close) does not give me any warm fuzzies. It reeks of unprofessionalism, greed, and laziness.

There needs to be a sane-software assesment. People rely on products with an ever increasing amount of source code - it would be interesting if a third party could come along and certify that a given codebase is not a big scary unmaintainable mess; not to say there won't be bugs or issues, but that these guys are at least trying to make a well designed system, and aren't doing a bunch of crazy stupid things.

thaumaturgy3• 11 years ago

The companies currently most actively involved in automated automobiles (hah) are pretty good at what they do. Volvo has been producing big trucks for several years now that have remarkable safety features, like automatically detecting stopped traffic ahead and bringing the fully loaded truck to a complete stop in what seems like an impossibly short distance (https://www.youtube.com/watch?v=HoCknasKdRU).

There are plenty of things I dislike about Google but I can't say they're bad at software. If anybody can build a safe automated vehicle, they can.

Tesla, too, who have a lot riding on their hard-earned reputation for building next-generation vehicles.

I'm an old-school car guy, I haven't been a big fan of the complexity added to cars ever since the really bad emissions control systems in 80s cars, but even I have to admit that all that complexity has probably saved a lot more lives than it has cost -- antilock brakes, airbags, better pollution control, better fuel economy, way better structural safety features.

userbinator2• 11 years ago

If you've ever driven a vehicle with them you'd know that air brakes are very, very powerful.

Sanddancer2• 11 years ago

There are standards, Toyota simply chose to ignore all of them. Standards such as MISRA [1] and DO-178C [2] exist for the purpose of ensuring software quality in safety-critical situations. Most embedded software development environments even include tooling to help verify that you're not doing the things that are unsafe. The problem lies more in that automakers aren't required to use any such standard, unlike what the FAA has required for decades.

[1] http://en.wikipedia.org/wiki/Motor_Industry_Software_Reliabi... [2] http://en.wikipedia.org/wiki/DO-178C

meesterdude2• 11 years ago

Very awesome comment! I didn't know there were standards, but it's great to see there are. However, Toyota should have followed them. I mean, did they think they knew better? I think with self driving cars, the standards will become more requirements; though I doubt before a series of deaths result from software failure.

rainforest• 11 years ago

Given that a lot of computer vision work is based on randomised algorithms, do you think that these standards would be enough? You could demonstrate 100% MC/DC coverage through a neural net implementation, but the weights are where the faults probably exist, for example.

lambdaelite6• 11 years ago

Toyota couldn't ignore these (voluntary) standards given that MISRA-C nor the earlier DO-178 didn't exist at the time the code was developed.

derefr• 11 years ago

Self-driving cars seem less scary to me. You can take unreliable components (like, say, a car with crappy firmware) and make a reliable system out of it if the software running on top is fault-tolerant. The "autopilot", unlike a human, will pull over the moment the car's brakes begin to fail, etc.

michaelfeathers• 11 years ago

Toyota had more than 10,000 global variables. “And in practice, five, ten, okay, fine. 10,000, no, we're done. It is not safe, and I don't need to see all 10,000 global variables to know that that is a problem,” Koopman testified.

I wonder why people want to believe 5 or 10 are okay.

jfoutz• 11 years ago

Check engine light? Passenger without a seatbelt?

I mean, it's an embedded system, resources are scarce. There are more elegant solutions, yes. But a pattern of setting an error condition from a bunch of different sensors, that's checked from a bunch of locations seems reasonable. At least, not insane.

ericye16• 11 years ago

For simple configurations that have global affects, global variables might be easier than some other methods of keeping track of settings. It's a bit more frequent in embedded software than other software though.

icelancer2• 11 years ago

He's not sanctioning 5-10 globals, only saying he could deal with that number of them without freaking out. Which is certainly true.

thrownaway24244• 11 years ago

Counting global variables is just silly. I think any number of global variables is perfectly fine.

thrownaway24244• 11 years ago

Lots of downvotes, no rebuttals.

Crito• 11 years ago

You might as well claim that you think that dogshit makes for a fine breakfast. It doesn't warrant a rebuttal.

chopin• 11 years ago

Maybe you should cite something to back your claim made upthread.

mercurial• 11 years ago

That's a consequence of the use global variables having been recognized as an anti-pattern for a very long time now, and people downvoting the obvious instead of writing a rebuttal.

xutopia• 11 years ago

You're using a throwaway in what seems to be an apparent troll. For newbie programmers out there the reason we don't use global variables willy nilly is because it is very easy to lose track of all the places where one can be accessed. The result is that your code becomes very fragile.

bootload3• 11 years ago

"Defects in the car’s electronic throttle control system (ETCS) were directly responsible for the Camry’s sudden acceleration and resulting crash." [0]

I couldn't find an exact description of how the driver crashed. Was it using the on-board cruise control or normal throttle use? Never use the cruise control. Must ask when I get my car serviced if it has electronic or mechanical throttle body cf @bliti

[0] http://www.usatoday.com/story/money/cars/2013/10/25/toyota-s...

catshirt• 11 years ago

it was a symptom of neither cruise control or normal throttle use. the way the issue manifested is actually pretty terrifying:

"Koua insists that his 1996 Toyota Camry sped up to between 70 and 90 mph despite heavy braking."

http://en.wikipedia.org/wiki/2009%E2%80%9311_Toyota_vehicle_...

out of context, but "we're in trouble, there's no break"

https://www.youtube.com/watch?v=03m7fmnhO0I

bootload3• 11 years ago

thx @catshirt, I note this about my Camry: "Toyota Australia announced that its accelerator pedals are made by a different supplier and that there is no need for a recall of Australian made vehicles".

Reading through the notes I see trying to turn the engine off while going effects various electrical sub-systems. Me, I'd try putting the car in neutral (auto).

A couple of years ago the alternator blew in my car. The battery had enough charge to let me drive +30km home after a re-start by a local RAC. When I drove the car the three kilometres to the shop, the charge started to drop and battery failure light (alternator failure) clicked on.

First the car started loosing systems: seat-belts, overdrive, ABS... etc. This continued, till I bunny hopped the car into the garage where everything stopped working. No doubt a sticky accelerator is scarier than the lurching I had in busy traffic, kept going till the end. Power steering was the last to go, then the engine itself.

hackuser• 11 years ago

In the 1990s an engineer for one the U.S. automakers told me that the code in their cars was a black box. Nobody knew how it functioned; all they could do was watch the input and output, and add patches as needed to modify the output.

Maybe the engineer was talking about code for a specific component, but perhaps Toyota's software isn't unique.

r00fus2• 11 years ago

Exactly my thoughts. I bet any company when put under the spotlight would show some really bad practices/anti-patterns.

Is that however, a problem with the industry? I can't help think of Fight Club's "recall" equation here...

tulb• 11 years ago

yaakov344• 11 years ago

This issue returns to Hacker News and to public consciousness every few months, and each time there is something missing: namely, some kind of direct proof that an actual failure of this software resulted in unintended acceleration, anywhere, ever. Let alone a case of runaway acceleration which the driver is unable to stop with the brakes (requiring a simultaneous brake failure). Since the accelerator pedals in these Toyota vehicles have been pressed no fewer than tens of trillions of times in recent years, that's not an appalling safety record at all. In fact, stuck pedals (whether due to bad lubricants or rolled-up floor mats) are many orders of magnitude more likely to cause stuck throttle than software failures, while runaway vehicles (ones with brakes not working either) are generally caused by "pedal misapplication", i.e. stomping on the gas instead of the brakes.

The fact that software has an execution pathway leading to something bad does not mean that this pathway can ever be entered, since in a closed realtime system like this, it is not possible to receive every combination of inputs, unlike in a system loading user data from a file. This is not to say that Toyota shouldn't clean up and verify its code, but the moral panics over what this code says about programmers, the human condition, Japan, etc. etc. are unwarranted.

Quote from http://en.wikipedia.org/wiki/2009%E2%80%9311_Toyota_vehicle_...:

On February 8, 2011, NASA and the NHTSA announced the findings of a ten-month study concerning the causes of the Toyota malfunctions of 2009. According to their findings, there were no electronic faults in the cars that could have caused the sudden-acceleration problems.

tunesmith• 11 years ago

Every time this article comes up on hacker news there are a few comments that show up talking about how no one was able to prove that the bugs definitely caused the crashes, and thus the criticisms of Toyota are overblown. I'm left wondering how on earth to reconcile those viewpoints with general software engineering (which is a huge part of what this community is about), where we regularly see articles about new concurrency frameworks, fault-tolerant programming, formal methods to prove correctness of code, etc. If it's important enough for Amazon to use TLA+ to prove correctness of DyanmoDB then shouldn't it be considered outrageous that Toyota doesn't do the same thing for something that is potentially an accelerating death machine?

Gibbon1• 11 years ago

The reason generally given is that in at least standard cars the brakes can apply far more torque than the engine can. It's telling how naive people are, they think the problem is 'unintended acceleration' A mechanical engineer thinks the problem describe is 'unintended brake failure'

yaakov344• 11 years ago

Sure, they should improve or clean up the code on general principles. However, there are 2 issues here:

1) A lot of these articles imply that some expert has demonstrated how a coding error will cause unintended acceleration. Then when you look at the actual source, it becomes clear that the "demonstration" involves changing the internal state of the controller in all sorts of arbitrary ways, and sometimes rewiring its sensors in an invalid way as well. In other words, this is not a recipe along the lines of "blip the throttle while changing from N to D, press the brake within 0.2 seconds, the throttle will now be wide open". The fact that no such recipe has been found, despite many millions of dollars spent on expert analysis, suggests that it doesn't exist in the wild. Implying that this code is killing people somewhere out there is misleading.

2) Yes, it has been overblown. We know that throttles can stick open, usually due to jammed or sticking linkages and pedals. That's not a huge problem, since brakes are powerful enough to stop cars in this state. I don't think it makes a huge amount of sense to rewrite software to stop software-induced unintended acceleration, which probably doesn't even happen, instead of writing software for lane departure/collision warning/assisted emergency braking and other safety systems, which we know can help drivers avoid accidents.

I guess a third issue here is the American litigation system, which can turn companies into villains without the slightest indication that their products cause any more issues than anybody else's - all it takes is a non-zero probability of failure (true for most products) and a media + legal frenzy.

lambdaelite6• 11 years ago

The HN community at large isn't experienced with software engineering. It seems like there are a handful of commenters who are, but they are a very small minority.

Personally, I'm surprised that the majority of the HN community overlooks or ignores the reports from NASA and Exponent (Toyota's outside expert witness) in favor of the testimony from Barr, the witness for the plaintiffs in the Oklahoma case. (I'm honestly amazed that Barr was allowed as an expert witness) Barr did show that the code "smelled" and that it was possible to inject specific faults to induce an uncommanded acceleration, but he did not show that his proposed failure mode occurred nor that it was probable to occur. In fact, part of his failure mode was that it left no record of a DTC. It's the perfect kind of failure mode for a sympathetic plaintiff to wield against an arrogant defendant with deep pockets: possible that it occurred, possible to demonstrate, understandable for a jury (who hasn't had a BSOD?), and impossible to disprove.

You bring up the practice of software engineering, but I think you need to place it in context of how software engineering was practiced around the time that the electronic throttle system was developed. MISRA, which HN readers are becoming aware of, didn't exist. Proof assistants, which today remain challenging to use and integrate into an SLDC, were even more arcane. In fact, your example of TLA+ wouldn't fly even today since TLA+ doesn't do code generation, and that would be a necessary component of the verification process of a regulated SLDC. I don't think TLA+ existed at the time either. Things certainly have changed!

tremon2• 11 years ago

Oh please. Your arguments are exactly the same as creationists' arguments against evolution. There exists no definitive proof for a chaotic system.

The whole point of software engineering is to manage and avoid complexity, not to obfuscate/complexify your software beyond any control and then weasel your way out of it by saying that your software can't be proven wrong. The fact that no proof can be given either way is a failure of the design process, not of the claimant.

"The fact that software has an execution pathway leading to something bad does not mean that this pathway can ever be entered" On the contrary, that's exactly what the term "execution pathway" implies.

"there were no electronic faults in the cars that could have caused the sudden-acceleration problems." No electronic faults? Sounds to me like they tested for wire insulation, not runaway code.

taejo• 11 years ago

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies.

-Tony Hoare

lambdaelite6• 11 years ago

Un bon mot ne prouve rien. (A witty saying proves nothing.)

—Voltaire

kensai• 11 years ago

"In 1998 that standard had a Rule No. 70 called -- I don't remember the exact language. But function should call themselves. And the rules basically are the same in but they changed the numbering system, so in the standard this rule, same rule is No. 16.2. So this violation of the MISRA C rule."

"What was NASA's view about this recursion? A So NASA's view, NASA was concerned about stack -- possible stack overflow. They had a couple of pages devoted to it, about five pages. I pulled some quotes here. Recursion could exhaust the stack space leading to memory corruption and run time failures that may be Yes. In what way? The stack can overflow due to this recursion in the Camry. And create memory corruption? And that would create memory corruption, that's difficult to test -- detect in testing."

Man, some elementary stuff here. They changed the specs numbering and did not update their manuals as well as forgetting that recursion in very costly in memory and could overwrite the stack... Jee, these are things a C programmer learns from day one, even in CS50...

JabavuAdams• 11 years ago

Please edit the title to read "Toyota's", not "Toyotas".

ilitirit• 11 years ago

Software that can modify any aspects of a vehicle's handling should be subjected to a higher level of testing and scrutiny than things like airbags and safety belts. A faulty cruise-control component is demonstrably more dangerous than a faulty safety belt.

e402• 11 years ago

Any ideas if other Japanese car makers have the same problem with their software?

thrillgore2• 11 years ago

10,000 god damn global variables. It would be very difficult to expect some kind of unit testing with that many straggling race conditions bouncing around. Wow.

cpeterso• 11 years ago

Does Google (or Tesla or Uber) use Ada for their self-driving car projects? I am afraid that answer is likely C++.

I noticed that this article mentions MISRA-C but not Ada. I thought I had read the Toyata used SPARK Ada. Michael Barr's slides have more technical details about Toyota's C code:

http://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUB...

Toyotas Unintended Acceleration and the Big Bowl of “Spaghetti” Code (2013)

Related Stories