Arizona Government Suspends Uber's Self Driving Cars from Roads

http://www.chicagotribune.com/news/nationworld/ct-arizona-uber-testing-20180326-story.html

by ghshephard • 8 years ago

374 313 8 years ago

chicagotribune.com

asdfologist• 8 years ago

Let’s suspend human driving too, since it kills over 100 people per day.

vkou2• 8 years ago

Humans kill 1 person every ~100m miles.

Uber has killed 1 person after ~3m miles.

Yes, I would absolutely suspend a system that kills 3,000 people/day in favor of a system that kills 100 people/day.

jazoom6• 8 years ago

n=1 (To reduce the number of email notifications I'm getting I'd best put this message in parentheses right next to this statement. NOTE: I'm not a statistician and don't believe statistics are all that important here! All I meant to say here is that this is a single incident. That's it. Nothing more.)

Edit: I'm getting the impression some people think I am suggesting there aren't grounds to suspend Uber's driving based on my 3 character comment above, so I'll paste my comment to a child comment here. Also, I'm not a statistician and I don't really care about the statistics here all that much when there's video evidence showing this was poor driving.

"Same reason clinical trials stop early if someone dies. It might just be bad luck. Unfortunately we'll never know and that drug might never be tested again. It might have been an amazing drug. In this case, based on how it played out, I personally wouldn't want Uber's self-driving cars near me."

RoyTyrell• 8 years ago

People that drive recklessly and kill people can, indeed, have their license pulled. This is not unreasonable and we'll see what the investigation finds. If Uber did not take needless risks and this was some kind of edge-case where everything that could go wrong, went wrong, then perhaps it would be acceptable to allow them to re-continue their testing.

Given Uber's track record, however, I believe we'll see this was more of their "don't give a fuck" brogrammer attitude. When you're only really risking people not being charged a correct fee or not being picked up for a ride it's one thing. Risking public safety is another.

coinjobber2• 8 years ago

Amen. Uber's fatal accident rate is now 50x that of sober human driver's. At their current rate of driving, they can't have a _lower_ rate until 2028! Inexcusable failure of their technology to prevent a human death in the most basic collision avoidance scenario.

setra• 8 years ago

A fatal accident at 50x the rate of a sober human driver with a study size where N = 1.

coinjobber2• 8 years ago

No. 3 million miles of observation. You can get a pretty exact and conservative estimate with a bayesian poisson process model. I don't have the time to run the numbers right now, but my guess is the posterior estimate that Uber's fatal accident rate is higher than a human's is >90%, even if taking the human accident rate as a starting prior.

tedsanders3• 8 years ago

It's a reasonable gut feeling to not generalize from n=1, but the numerical evidence - with either a Bayesian or frequentist approach - is actually quite strong and statistically significant. Math here: https://news.ycombinator.com/item?id=16655081

pgreenwood4• 8 years ago

I'll have a go at seeing what we can conclude from the data. Others, check my thinking please. Now we have 1 death in 3m miles for Uber, versus 1.18 deaths in 100m miles for sober drivers.

The expected rate for 100m miles for Uber is 33.333...

But how confident can we be? To answer that let's compute a poisson confidence interval around that rate, as in https://stats.stackexchange.com/questions/10926/how-to-calcu....

Let's see what a 95% confidence interval for 1 death in 3m miles looks like:

  > poisson.test(1,conf.level = 0.95)$conf.int
  [1] 0.02531781 5.57164339
  attr(,"conf.level")
  [1] 0.95

Multiply that by 33.333 to convert to deaths per 100m miles:

  > 33.333333*0.02531781
  [1] 0.843927
  > 33.333333*5.57164339
  [1] 185.7214

So 95% confidence that the rate per 100m miles is from 0.84 to 185.72. That's pretty wide! And since the lower bound crosses 1.18, the difference is not significant at the .05 level (if we must make that particular comparison). However, let's look at 90% CI:

  > poisson.test(1,conf.level = 0.9)$conf.int
  [1] 0.05129329 4.74386452
  attr(,"conf.level")
  [1] 0.9

Which gives a CI of 1.71 to 158.13. So with 90% confidence we can say Uber is less safe than sober drivers. Ok.

Now let's look at 93% CI:

  > poisson.test(1,conf.level = 0.93)$conf.int
 [1] 0.03562718 5.17251332
 attr(,"conf.level")
 [1] 0.93

That gives a CI of 1.188 to 172.417. The lower bound being just a bit worse than sober drivers.

So we can conclude with 93% certainty from this data that Uber is less safe than sober drivers. Probably a LOT less safe. Although the CI is really wide, this is shocking data for Uber, in my opinion.

savanaly• 8 years ago

What is it compared to human drivers in general? That seems to be the fair comparison as "computer will never be drunk/tired/distracted" is usually cited as one of the benefits of computer-driven vehicles.

galdosdi10• 8 years ago

Haha, as I joked in another article comment thread, their current crash rate ironically means a somewhat drunk but otherwise defensive driver is probably much safer! What a world.

caf3• 8 years ago

If you want to be legally allowed on the road, I think your benchmark has to be other drivers that are legally allowed on the road.

agildehaus• 8 years ago

Before the accident they were infinitely better than a sober human driver.

Understand statistics before employing them please. We don't have enough data and a single data point doesn't change that.

raverbashing• 8 years ago

I wonder if this type of events are modeled by Poisson processes and measured by MTB(A?)

random314• 8 years ago

Understand statistics before employing them please

Do you understand statistics?

https://news.ycombinator.com/item?id=16685929

simion3145• 8 years ago

It is not 1 data pint though, say you shot at a target 100 times and hit only 1 time , is this 1 data point only and I can't make any conclusion?

Similar if you drive 1 million km and kill 1 person, if I drive 10 km and kill 1 person is still 1 data point and I can make no conclusion? I think it would have been 1 data point if this was the first km a Uber self driving car has driven.

mv4• 8 years ago

reminds me of this xkcd https://xkcd.com/605/

buvanshak4• 8 years ago

Good riddance. But I think this is not enough to send a strong message to all those who want to put half baked tech out there in hopes of "disrupting" something..

Also, I really hope all self driving testing is suspended until there is sufficient legal and testing ground work before these things are allowed on roads...

ineedasername5• 8 years ago

Precisely. Disruption can be a wonderful force for change and innovation, but "minimum viable product" is an astronomically higher bar when injury & death are the default failure modes.

sokoloff10• 8 years ago

It might surprise you that we were testing (selectively) autonomous cars on the autobahn in Germany in 1991 as part of the Prometheus project. Our tech was nowhere near as good as what's available now and so we compensated by having attentive drivers hovering over the E-stop button.

I don't think that stopping testing over a single crash is warranted. Maybe suspend it for a short time while you look into it, but if we stopped medicine every time a patient died in a trial, society would be worse off.

buvanshak4• 8 years ago

I am not asking to stop it for all time.

Just device proper tests that deem a self driving vehicle to be worthy/competent enough to be on the road with a normal average human being behind the wheel.

And then have every single self driving vehicle to pass before it can be put on the road. Is that too much to ask?

I am just asking for a driving test for self driving tech. Humans undergo driving tests. Why give machines a free pass? As we have seen here, having a human on stand by just won't cut it..

dahart• 8 years ago

if we stopped medicine every time a patient died in a trial, society would be worse off.

What on earth do you mean? Clinical trials are stopped all the time when people have adverse reactions & die. And people who die in clinical medical trials are volunteers, consciously aware of their participation, and having signed a waiver. In the U.S., medications may not be sold or administered to the public before being approved via trials. If Uber had to meet the same standards as the medical industry, Elaine Herzberg would be alive, and we wouldn't see autonomous vehicles on the roads for several more years.

oldgradstudent• 8 years ago

Drug trials are done on volunteers with informed consent.

Testing on public roads is like kidnapping random people and forcing them to take the drug.

swang720• 8 years ago

This is interesting, but can we avoid posting links to articles behind paywalls?

Here's one on the subject that isn't: http://www.chicagotribune.com/news/nationworld/ct-arizona-ub...

samschooler• 8 years ago

Just click the "web" link then click the WSJ link. If you have no cookies from them, it gets you around the paywall most of the time.

stordoff• 8 years ago

FWIW, complaints about paywalls are generally off-topic, though I agree, a non-paywalled source can be useful.

https://news.ycombinator.com/newsfaq.html

ineedasername5• 8 years ago

I think courtesy should dictate that paywalled articles be accompanied by at least a brief description. Perhaps a change to the FAQ to recommend such?

dang3• 8 years ago

It's helpful to post better URLs, and we've changed the submission to that one from https://www.wsj.com/articles/arizona-governor-suspends-ubers.... Thanks!

Please don't post complaints about paywalls, though: https://news.ycombinator.com/item?id=10178989. They're off-topic, and as long as standard workarounds exist (which users routinely post to these threads), paywalled articles are ok. Not that they aren't annoying; they suck. But HN would be quite a bit worse if all such articles were cleared out of here.

owenversteeg2• 8 years ago

Saying nothing about Google/Waymo, it appears they're allowed to continue? Their self-driving technology seems far more advanced, I think it would be unfair to punish all the players for the bad actions of a few.

In particular, Uber seems to have been operating self-driving cars recklessly and in an attempt to "move fast and break things". When those things are people's lives, that's unacceptable. I'm all for interesting technology, but sometimes existing rules are there for a reason.

I particularly hope this doesn't dissuade individual states from making leaps to embrace a particular technology. I think that's one benefit of the US: individual states have a lot of power to allow a new technology or permit testing that's illegal or in a gray area in the rest of the country. Arizona made a bet on self-driving technology. I applaud Arizona, even if the bet in this case might have turned out badly.

onetimemanytime• 8 years ago

Or maybe Uber got unlucky and they had to be suspended. I wonder how many miles they had and every how many miles does an accident happens with human drivers.

EpicEng• 8 years ago

It's been posted in related threads, bit many, many more human miles are driven per fatality. The problem is that the number of self driving car incidents is too low to reliably extrapolate out.

ggg9990• 8 years ago

Uber did not get unlucky -- they are simply an evil company. Their cars need human intervention every 13 miles as opposed to Google's every 5,600 miles -- they were not fit for the road but Uber let them out anyways.

Uber should be shut down and the principals should get life without parole.

Erlich_Bachman• 8 years ago

Watch the video of the accident. Even though it's dark (llooks to have been manipulated afterwards), you can still judge the actual situations in terms of the position of the lights, and the car and pedestrian - to see that it was obviously the car's fault because it should have seen the pedestrian with the lidar, or if the lidar was deactivated - it should have driven with much lower velocity.

spiderPig• 8 years ago

Which makes me think of it would be prudent for the government or military to invest in this tech as well. Companies like Northrop Grumman while being bureaucratic nightmares, take safety pretty seriously.

zitterbewegung• 8 years ago

This whole thing was started by a DARPA grand challenge. https://en.wikipedia.org/wiki/DARPA_Grand_Challenge

Industry takes the lead after DARPA's investment which makes sense.

dawnerd• 8 years ago

I kind of assumed they already were. In fact I'd be surprised if they weren't.

cbron2• 8 years ago

Yes Waymo is still active. I live in the testing zone and see an enormous amount of their vans. About 1 for every 5 minutes of driving.

Cheyana• 8 years ago

I drove down a main road here in Chandler, Arizona yesterday and saw three in five minutes. Based on all my experiences on the road with them in the last few months I’d say the chances of them having an incident similar to Uber are much less.

walrus01• 8 years ago

I wonder if this is because waymo has shown more details of how their lidar operates (and how it would have detected the pedestrian who was hit by the Uber car), but Uber is refusing to show the inner workings of their lidar's abilities to the regulators.

The manufacturer of uber's lidar system recently came out and said that they do not believe it failed, but that the input from it failed somewhere inside Uber's software.

bitumen2• 8 years ago

I think the public will probably feel the same way, but if this happens again with another company, I think there will be sustained backlash. Hopefully Waymo is as advanced as they’re advertising, and given what a terrible company Uber is by comparison, that doesn’t seem to be an unreasonable assumption.

You know what they say though, once is a mistake, twice is a habit.

foobarian2• 8 years ago

Hopefully Waymo has test drivers that actually pay attention.

visarga• 8 years ago

You know what they say though, once is a mistake, twice is a habit.

This line of thinking might end up with more people dead by delaying the advancement of SDC's.

qwerty4561272• 8 years ago

bad actions of a few

What bad actions? If you mean an AI car killing a person - that's hardly "an action", that's an accident. Classic human-driven cars kill people every day yet are allowed to continue operation. Do people really expect self-driving cars to be not just more safe than classic but actually 100% safe?

By the way I myself don't drive at all and only use public transport (and a bicycle to ride the countryside). A major reason stopping me from driving is I can't logically understand how can I make sure I am not going to kill anybody if I drive at a speed considered normal (rather than something ridiculous like 15 km/h) and somebody may jump out right in front of my car suddenly (which is very very far from an impossible encounter).

I am totally car-phobic yet I really doubt suspending self-driving cars operation can make the roads more safe. I trust AIs much more than humans when it comes to subjects like this - they can fail occasionally (especially in guaranteed failure situations like when a pedestrian emerges right in front of you all of a sudden) but are seemingly less prone to fail than humans are.

knuththetruth2• 8 years ago

What bad actions?

Uber disabled the safety systems included with the car that was involved in the incident. They relied solely on their own, half-baked tech when these systems could have saved the life of person who was killed:

https://www.bloomberg.com/news/articles/2018-03-26/uber-disa...

jonathanstrange• 8 years ago

One problem with these comparisons is the misinterpretation of statistical data. For general policy making statistics are useful, but for personal decisions you need to take them with a grain of salt and switch on your brain.

Human traffic accident statistics include the totality of reckless drivers, street racers, and drunken idiots. Would you readily be a passenger when they drive? I wouldn't. At the same time, the statistics also include the calmest, best, and most reasonable drivers you've ever met in your life.

I think it's reasonable to expect self-driving cars to always drive better than your average drunk college student with 1 year of driving experience and instead perform more like the best drivers you've ever met. So yes, self-driving cars should perform above the human average.

simion3145• 8 years ago

This incident brings to light the fact that this Uber car is blind, the person did not jumped in front of the car from behind some obstacle.

I can't drive a car because of my bad eyes so it makes sense that when we see that blind cars( or with blind spots) drive on roads we don't like it.

z3t42• 8 years ago

"always be hustling" is/was the Uber motto. " move fast and break things" is/was Facebook.

galdosdi10• 8 years ago

True, but the concept still applies here. Uber has gained a well deserved reputation for being careless as long as what they are being careless with belongs to other people.

mtgx3• 8 years ago

Well, do these comments sound like "hustling" to you or more like "move fast and kill things"?

https://www.theverge.com/2018/3/20/17144090/uber-car-acciden...

lern_too_spel4• 8 years ago

Arizona did something reckless and stupid. Unfortunately, somebody other than the decision maker paid the price. Arizona stupidly allowed Uber to operate after California kicked Uber out and after seeing the videos of Uber cars running red lights in SF.

For what? For a few temporary (the plan is to be fully autonomous quickly) low paying test driver jobs? The engineers making the big salaries stayed in California (or Pittsburgh in Uber's case).

https://www.google.com/amp/amp.fox10phoenix.com/news/arizona...

galdosdi10• 8 years ago

This is what's always confused me. The usual play is to suck up to a corporation in the hopes that a bunch of jobs or other investment will show up, but self-driving testing is kinda a perfect example of something that will not give any economic benefit to the testbed state at all. It takes very little local investment, and it's a temporary thing that will only yield benefit for the corporation doing the testing, and as soon as they have as much data as they want they can leave and use that data anywhere else.

And if what Arizona hoped for is to get self driving taxis early, well, other states can simply let the sucker states like Arizona do the testing, then wait to legalize self driving till the day after it proves itself, thus reaping close to 100% of the benefits with close to 0% of the disadvantages.

I honestly think Arizona politicians just blundered because they were following the typical playbook out of pure habit. I hope the explanation is not less innocent than that, but that's also very possible. Petty corruption is very common in USA state governments.

hueving5• 8 years ago

For what?

To advance technology and make AZ attractive to other R&D. Not everything has to be tied to a direct economic boom.

humanrebar2• 8 years ago

Arizona did something reckless and stupid. Unfortunately, somebody other than the decision maker paid the price.

Well, crossing the street in that manner wasn't the smartest thing to do either. The person shouldn't have been run over, but we accept some risks while walking bicycles across multi-lane highways at night without watching for oncoming traffic.

xvf22• 8 years ago

The police department was also quick to defend Uber by saying that the crash was 'unavoidable'.

From the reports it seems like Uber were heavily focused on putting in miles trying to make up ground. The NTSB report will hopefully expose the failures (if any) of the Uber self driving system or the organization.

They've said that self driving cars are critical to their survival and if it turns out they cut corners / mis-represented things then it'll be hard for them to get out of that hole.

test6554• 8 years ago

The decision to allow self-driving cars in Arizona is going to save far more lives than it will take. I firmly believe it was a massively good decision. but time will tell.

bowlich• 8 years ago

I heard Phoenix took quite a big hits in the tech sector in 2008.

I live out in the boonies in Arizona, I would say at least if I go off of for recruiter spam, there's been an uptick in engineering positions related to the self driving car companies.

Whether those positions are actually real I don't know, never actually replied to them.

galdosdi10• 8 years ago

Actually, it may be an instance of a "tragedy of the commons" but I think the most strategically optimal move for a state when there are 49 others is to ban it and let "sucker" states do the testing, and then immediately legalize it as soon as it shows promise.

Just like how I never upgrade to a new software version right away, letting the other "suckers" test it in production for me, creating value for me at their expense with no repercussions whatsoever, since the delay is very small.

Another strategically effective move might be to charge corporations an appropriate fee to capture back the value of the data they are collecting by experimenting on public roads. Given how valuable driverless technology could be, the fees could be large.

slivym• 8 years ago

I don't believe this because whilst there are costs for the state of allowing self-driving cars, there are also massive upsides. The CompSci jobs that are involved in self-driving cars are highly sought after and by enticing testing of these cars Arizona is also likely bringing in high value tech jobs that would normally go to somewhere like silicon valley. Once they've established a center of excellence for that skill set it becomes self sustaining too. So you'll probably find that years after every state has self-driving cars Arizona still have more of those jobs.

losvedir• 8 years ago

I had thought it was a brilliant move by Arizona. They were trying to be for autonomous vehicles like what Detroit was for cars back in the day, or Boston is now for biotech, or New York for finance.

But I never thought through: which jobs are moving to Arizona? Autonomous vehicle production and maintenance? LIDAR and other sensor R&D? AI software development? Or is it just the test driver and the cars get developed elsewhere?

ducktoller4• 8 years ago

There was a person at the wheel.

That person wasn't paying full attention to the road. From the video of the crash, it looks like the woman at the wheel was looking at their phone.

Why is no one acknowledging that this woman was responsible for taking over the wheel if necessary for any reason. When I have cruise control on, I take my feet off of the pedals and rest them on the floor. But if, say, a homeless person with a bicycle walks out into the middle of the road at night with a black sweatshirt on in front of my car, then I step on the brakes.

And why is no one acknowledging that a homeless person walked out into the middle of the street at night in a black sweatshirt. I definitely don't know whether I, even if I wasn't on my phone like the car operator was, would've seen the person in the road with enough time to slam on the brakes. I might've killed that person walking across the road by mistake, too.

Like millions of other people on the road every single day, the car operator was on their smartphone while they were at the wheel. And like 3-5 million other people a year, the homeless person walking across the street was killed in a car accident.

The woman walking across the street and the woman at the wheel of the car broke the law. Uber didn't break any laws as far as I can tell. They were as responsible as any other company building self-driving cars as far as I can tell, too. I don't think it's fair to put the fault on Uber here.

amelius2• 8 years ago

Cruise control is different because you still have your hands on the wheel.

salvar2• 8 years ago

What is the significance of the person being homeless?

seren• 8 years ago

They are also responsible for the test setup, the behavior and training of their employees.

If all their drivers, which ultimately are the latest safety when everything fails, are routinely not following the process that is an issue.

Maybe you need to have 2 drivers, maybe you need to randomly monitor the behavior of the driver remotely, but you have to demonstrate some active measures your took, to demonstrate the safety of the test. You cannot just say "not my fault".

I am not saying that Uber have not followed a rigorous procedure I don't know. But saying it is their fault or not seems a bit premature.

pjc50• 8 years ago

Asserting that the person who was run over broke the law is victim-blaming of the highest order.

The claim that the ""driver"" was texting suggests that they are at fault, but the whole "safety driver" concept is flawed; the less the driver has to intervene and take control, the less likely they are to remain alert and willing to jump in and take control. You almost need a "fault injection" system that gives back control once every 5-10 minutes just to keep them awake.

LogicalBorg2• 8 years ago

I think it's incorrect to say that the woman was looking at her phone. She was looking down and to the right of the steering wheel, which is where Uber places a monitoring tablet (an iPad) in their self-driving vehicles (where the radio is).

Here's an article that shows the sort of thing she would be looking at, an animation of road conditions: http://www.businessinsider.com/uber-driverless-car-interior-...

If you actually examine her behavior, in the space of ten seconds, she manages to:

* Look down at the tablet and watch the animation

* Look up at the road

* Scan the lane to the left (she was in the right lane)

* Look forward again

* Look down at the tablet again

* Look up again just as the accident occurs

She is systematically scanning her environment and appears to be alert and engaged.

The problem is that there is only one test driver, who has to divide her attention between the tablet and the road. Often two drivers are used, and then one watches the tablet and the other watches the road.

I think it is unfair to accuse her of inattention when she appears to be doing the job exactly as it is designed.

The reason everyone says "she was looking at her phone!" is because they don't understand the setup of how Uber's self-driving car works. They are only familiar with being bored and looking at their phone while driving. They think the test driver's job is to just stare out of the window all day, which is not correct.

crispyporkbites• 8 years ago

From the video of the crash, it looks like the woman at the wheel was looking at their phone.

How do you know that? They could have been monitoring the car in another way.

Also, it’s pretty obvious that if a car is fully automated, the driver isn’t going to pay attention. If that was you, or any other human the situation would have been the same.

The car screwed up, it should have seen the pedestrian. The safety mechanism of having a driver behind the wheel didn’t work either, and we all know it doesn’t work.

michaelt2• 8 years ago

  Why is no one acknowledging that this woman was
  responsible for taking over the wheel if necessary
  for any reason.

Nobody in the industry really believes safety drivers work, beyond the early point in testing where they're intervening regularly.

It is widely known [1, 2, 3] that it's extremely difficult for humans to remain vigilant when monitoring systems for rare events. Airport baggage screening systems perform "threat image projection" where they show the operator knives and guns from time to time, just to break up the monotony and reward attentiveness.

Beyond early testing, safety drivers are there to make the public/politicians feel safer, for situations where a slow response is better than none at all, and for more cynical companies, so you can use them as scapegoats in situations like this one.

[1] https://pdfs.semanticscholar.org/ece2/465ed2258585ebb8055fd7... [2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5633607/ [3] http://journals.sagepub.com/doi/full/10.1177/001872081350155...

arkades• 8 years ago

There was a person at the wheel. That person wasn't paying full attention to the road. From the video of the crash, it looks like the woman at the wheel was looking at their phone. Why is no one acknowledging that this woman was responsible for taking over the wheel if necessary for any reason.

The woman was part of their safety program, but ultimate responsibility for the effectiveness and failures of said program lies with them. For instance,

Other driver assist programs (eg, Cadillac) have eye tracking and hand sensors in the wheel. If your eyes leave the road or hands leave the wheel escalating alarms are initiated until driver assist is deactivated. This is a technology that takes into account normal human behavior. Uber did not do this.

Uber previously used two person teams, to minimize tedium and lapses in awareness that emerge when a single person as the job of staring out an uneventful window for hour(s) at a time. This is a policy that takes into account normal human behavior. They did not do this.

Either or both of these could have prevented this death. Both of these decisions occur at a level above the proximal driver-pedestrian interaction.

YeGoblynQueenne9• 8 years ago

> And why is no one acknowledging that a homeless person walked out into the middle of the street at night in a black sweatshirt.

I'm guessing, because a lot of people wear black clothes and self-driving cars must be able to deal with that, too. If safety-critical systems can't react to a common real-world situation then they 're just not safe enough to operate in the real world.

You might as well ask what was this woman doing crossing a road at night where even human drivers have reduced visibility. Well- people might cross roads at any time of day (legal issues aside- jaywalking is a very US-centric thing btw). Roads and cars must be designed so as to be safe under this assumption and drivers must be trained enough to deal with the situation.

What is the alternative? Make everyone wear high-visibility vests, or carry special self-driving car beacons when they go outside? I've actually heard both of those ideas since Elen Herzberg was killed and they're both on the near side of the absurd (so I'm sure someone will come up with something even more out there).

But, if tech that is supposed to make our lives easier is too dangerous, the answer is not to make our lives harder to accommodate it. That would defeate the purpose of the technology in the first place.

mtgx3• 8 years ago

Because Uber and possibly most other self-driving car makers aren't doing it right.

First off, Waymo, Ford, and even Volvo (whose car was in the accident) have said that they want to skip Level 3 because expecting people to actually watch the road while they're "self-driven" is not reasonable. They saw from initial testing that the drivers were dosing off.

https://www.bloomberg.com/news/articles/2017-02-17/ford-s-do...

And I didn't know this until now, but it looks like Toyota is now thinking the same:

http://www.motortrend.com/news/toyota-might-skip-level-3-aut...

Second, if they were to at least attempt to do it right (like Tesla is doing now, but didn't do in the beginning either) is to force drivers to keep their hands on the wheel at all times.

Also, Uber could have been watching its drivers with the cameras, and fired those that didn't stick with the program.

And all of this doesn't even mention how Uber's self-driving system seems to be terrible. Lidar wasn't working, the car didn't brake, and they've had one intervention per 13 miles while Waymo has one per 5600 miles.

So, my point is, Uber, the company, still seems to share most of the blame here, for all of those different reasons.

bryanlarsen2• 8 years ago

Even if the driver and the woman crossing the street were both to blame, it doesn't mean that Uber was any less to blame. Accidents in critical situations are typically multi-system failures, and every failure needs to be addressed.

There are four entities who could have and should have relatively straightforwardly avoided this death.

1. The woman shouldn't have crossed the street there and then.

2. The safety driver shouldn't have been looking at her phone (if that's indeed what she was doing).

3. Uber's automation should have caused the vehicle to brake much sooner.

4. That street should have been designed much safer. The design of a lit crosswalk on the median encourages people to cross there, so much stronger discouragement is required. Furthermore, a 35mph limit in an area with pedestrians is going to regularly cause pedestrian fatalities. That's a trade-off most people seem willing to make, but if you make that trade-off you have to own it. If the speed limit was 20mph that woman would be alive today.

As far as I can see it, all 4 entities are 100% responsible for the death of the pedestrian.

None of those 4 entities passed the "reasonable person" test with their actions, therefore all 4 are fully responsible.

Sure you can argue all you want on whether one entity's misbehaviour is more egregious than the others. It doesn't matter; all 4 engaged in behaviour that regularly kills people at a rate much higher than acceptable.

phyller• 8 years ago

This is not about legal or illegal. If the Uber program is unable to detect a large object like that in the middle of the road using lasers, they shouldn't be driving on public roads at lethal speeds. That's all there is to it. If autonomous cars mean that whenever a little kid runs out into the street after a ball and they aren't on a crosswalk, they get run down, or the cars smash full speed into any obstructions in the road, then I don't think they are ready for public testing, though I strongly suspect this is an Uber specific problem.

And not only are they apparently incapable of handling the most basic emergency you can think of, but they are trying to spin it like it's not their fault instead of immediately identifying what went wrong. This makes it appear that not only are they incapable but untrustworthy too. These are dangerous machines going down public roads, they have at least as much responsility as the rest of us to not hit people with their cars. And I don't get to hit people with my car just because they are Jay walking, it's night, and I'm not paying attention. Half the people in NYC would be dead already. The autonomous car can't see what is ahead well enough to avoid a potential collision? Just like a human driver, it should slow down. Otherwise these things should still be getting refined on a private course.

w0utert• 8 years ago

> And like 3-5 million other people a year, the homeless person walking across the street was killed in a car accident.

3-5 million people a year? Are you adding all road fatalities worldwide together here, including those in countries with fatality rates that are literally 100 times that of any developed country, and which are by causes completely unrelated to the kinds of problems autonomous cars try to solve?

The number of road fatalities in all of the US is close to 40K last year. Still a lot, but almost over 100 times less than the figure you quoted. The number of non-vehicle deaths (like pedestrians) is measured in the hundreds [1]

[1] https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...

YeGoblynQueenne9• 8 years ago

> Their self-driving technology seems far more advanced, I think it would be unfair to punish all the players for the bad actions of a few.

It's very unlikely that Uber and Waymo have radically different technology. The two companies probably differe only in business and testing practices- i.e. the environments and the conditions in which they are willing to test their cars.

Waymo simply seems to be more conscious about safety and therefore use its systems well within the safety limits of the technology. But, both companies have access to the state of the art (in the form of highly paid specialists) and you can't really expect one to be too far ahead of the other in terms of capabilities.

Edit: I'm guessing all this by comparison with results generally in machine learning and machine vision. With the exception of a big leap forward with CNNs in 2014, most improvements you ever see in the literature are in the order of a couple of percentile points at most so most systems differe by a very small margin. The same should go for industry entities using this technology in self-driving cars.

cbr2• 8 years ago

In the literature you see small margins because many of the things that really matter in practice (data quantity, data quality, engineering) aren't generally of academic interest.

Lewton• 8 years ago

I got the impression that Waymo has a ton of actual, human written, software on top of the ML core, "manually" handling a lot of edge cases.

I doubt Uber has this

tim3333• 8 years ago

Google, said that in tests on roads in California last year, its cars went an average of nearly 5,600 miles before the driver had to take control from the computer to steer out of trouble. As of March, Uber was struggling to meet its target of 13 miles per “intervention”

Quite a difference.

foobarbazetc• 8 years ago

You know who’s not applauding Arizona? The dead woman and her family.

Turns out when you let corporations do whatever they want without oversight for a weird PR “win” against SF people can die.

Game_Ender• 8 years ago

I wonder how long the investigations are going to take, they usually take months to a year in these cases. Or have for Tesla at least.

The political side is interesting to look at as well. The governor directly issued the order and he is up for election this year [0] and he played up the positive PR of the Uber and others coming to the state.

0 - https://en.m.wikipedia.org/wiki/Arizona_gubernatorial_electi...

notatoad• 8 years ago

Seems like kind of a pointless action to make it look like they're responding. Uber already suspended their testing in Arizona.

suresk2• 8 years ago

For all of the theoretical moral questions that have been brought up around self driving cars (the proverbial lever that lets you kill one person instead of a handful, for example), it's interesting/inevitable that this one came up first.

I'm sure lots of people will argue that self-driving cars will, in the long run, save a lot of lives, and therefore we can justify some loss of life in the process. They are probably right about self-driving cars saving more lives in the long run, but I don't know that the conclusion follows.

From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision. But self-driving cars have more sensors and, in theory, more capabilities to prevent something like this, and we absolutely should expect them to do better.

As someone with a little bit of experience with the machine learning side of things, I understand how this stuff is really hard, and I can imagine a handful of ways the system could have been confused, which makes it all the more important to understand what went wrong and how we could prevent it from happening in the future.

As self-driving cars become more prevalent, things like this will happen and it will be tough to walk the line between over-reacting to every incident and killing progress in this area and designing safe cars that don't make the problem worse. I think it is prudent to have Uber stop testing on public roads until they can explain how this happened and how they can keep it from happening again.

ars• 8 years ago

From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing

No, a human would have easily avoided this (unless they were not looking).

The video is much darker than what a person would see.

Even when you can see the bike on that dark video, the car still does not react!

MBCook• 8 years ago

Based on what came out of the last few days I don’t think it’s clear at all, I think a human would not have hit the woman.

The footage we saw was from a terrible dashcam and did not represent how dark the street actually was. Other people who have driven down the street with cameras show it to be perfectly reasonably lit, with streetlights and everything.

Second, the woman was crossing across two lanes, so any driver watching would have already seen her cross the left-hand lane. It’s not like she popped out from behind a tree suddenly. She wasn’t even going that fast.

This seems to be a massive failure of Uber’s system as well as a complete failure of the ‘safety’ driver.

I don’t think anyone’s overreacting. I think they’re under reacting. I think it’s pretty clear over was heavily negligent. There’s no way they don’t get sued. What comes out at trial is going to be up amazing, I think.

I think if this had been a “real” accident that would’ve been almost impossible to avoid, around it WAS impossible to avoid, or one where the human stepped in and it wasn’t good enough… We would be having a discussion we’re like what you’re saying. Maybe if this was what emai I think if this had been a “real“ accident that would’ve been almost impossible to avoid, around it WAS impossible to avoid, or one where the human stepped in and it wasn’t good enough… We would be having a discussion we’re like what you’re saying. Maybe if this was Waymo or one of the other comapnies.

But it looks like the dubious honor of first death by a autonomous car was totally unnecessary and preventable.

sokoloff10• 8 years ago

The pedestrian also has a significant contribution of negligence to this collision, IMO.

BurningFrog• 8 years ago

I think both these things are true:

Something was very wrong with how the Uber car handled this.

A human driver might well have avoided hitting the pedestrian, but would not have had any legal trouble if they did.

dontreact• 8 years ago

"From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision."

The camera footage is A LOT darker than what people have filmed there just a day or two after and is probably not a good representation of what it looked like to the human eye.

Even if it was fairly accurate, the car would be HEAVILY and EXTREMELY dangerously overdriving its headlights in a way that humans would not do.

takeda4• 8 years ago

From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision. But self-driving cars have more sensors and, in theory, more capabilities to prevent something like this, and we absolutely should expect them to do better.

The footage was artificially darkened, or it was extremely crappy dash cam or both. I suspect Uber just tampered with it (to get some sympathy) Someone posted a footage from that place few days later and that is very well lit road. On Uber footage you can see lamps on the sky, yet it is very dark.

Also note this dashcam is not what the car uses, and the car is also equipped with LIDAR so it should be able to spot pedestrian even in pitch black.

As self-driving cars become more prevalent, things like this will happen and it will be tough to walk the line between over-reacting to every incident and killing progress in this area and designing safe cars that don't make the problem worse. I think it is prudent to have Uber stop testing on public roads until they can explain how this happened and how they can keep it from happening again.

Your are missing the point here. Preventing accidents like this should be thing that AI would excel at, even today.

The fact that this failed means that there is something seriously wrong with how Uber is doing it.

Aspyre4• 8 years ago

How would the footage be tampered when it was taken by the police directly from the dashcam itself at the scene?

Also, there was no moon on that night and all of the video comparisons that have been put out are low-light cameras. They are not very realistic.

Also, noone seems to think it's a possibility that the agencies responsible for the lighting in this area to have made improvements immediately after the inicident before the public speculation started.

zer00eyz• 8 years ago

I'm going to take your post apart because I enjoyed it very much...

For all of the theoretical moral questions that have been brought up around self driving cars (the proverbial lever that lets you kill one person instead of a handful, for example), it's interesting/inevitable that this one came up first.

Your talking about the "trolley problem" and it is a ethical question as opposed to moral. This is literally a fuckup, bad programing or poor design. We are going to have real ethical concerns because at some point a car is going to have to make a choice pedestrian or occupant - and I know that my elderly father would choose himself if he were in the car, but I might make a different choice.... will the cars ethics ever reflect my own?

I'm sure lots of people will argue that self-driving cars will, in the long run, save a lot of lives, and therefore we can justify some loss of life in the process. They are probably right about self-driving cars saving more lives in the long run, but I don't know that the conclusion follows.

It is a tough one, and if there is more loss of life does using a self driving car a bit of fruit off a poison tree.

https://en.wikipedia.org/wiki/Eduard_Pernkopf springs to mind when you bring this up.

From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing

Except the fact that when you look at footage of the road when driven on with a decent camera recording rather than the crapy low quality and very much artificially dark uber footage this looks to have been very avoidable. If the guy who was supposed to be there to take over had been looking up right then, we might not be talking about this at all...

As someone with a little bit of experience with the machine learning side of things, I understand how this stuff is really hard, and I can imagine a handful of ways the system could have been confused, which makes it all the more important to understand what went wrong and how we could prevent it from happening in the future.

Were talking about uber here... it is also just as likely that they are in way over their heads.

As self-driving cars become more prevalent, things like this will happen and it will be tough to walk the line between over-reacting to every incident and killing progress in this area and designing safe cars that don't make the problem worse.

When I read this bit I felt compelled to respond - the life of one victim is too many. In the early days of aviation people knew the risks... But this wasn't the test pilot who died this was a woman walking down the road. I look forward to the NTSB report - they tend to be the adults in these situations.

suresk2• 8 years ago

Your talking about the "trolley problem" and it is a ethical question as opposed to moral. This is literally a fuckup

Right, thanks for the correction. The ethical question I was getting at is "What is the tradeoff of making sure self-driving cars are 100% safe and predictable before we allow them on the road at all, versus the time delay that will cause in which tens of thousands of people will die via human drivers?"

My thoughts behind this post came from reading comments elsewhere stating that the Arizona government was ultimately costing more lives by suspending Uber's program than it was saving. I disagree with that assertion, although it seems like my post came across in not quite the way I intended it.

My knee-jerk reaction to those comments (supporting Uber) is to liken it to advocating for ignoring deaths in Phase 1 of a clinical trial because there is hope that the drug will save more lives - but even then the comparison isn't apt, because people in clinical trials at least consent to some notion of the risk involved.

To me, it is a really difficult, uncomfortable question. And it isn't so much targeted at this particular case - where, as you say, it was more of a fuckup by a company known for not really being careful - but rather at more marginal cases, and there will be plenty of those. Driving is a lot more complicated than people give credit for, especially in areas where you are sharing the road with cyclists and pedestrians, and the ML approaches to self-driving are probabilistic in nature and will be wrong sometimes.

You can look at this crash and say "Simple safety measures available today would have prevented, or at least mitigated, this collision", and you'd be right, but that won't always be the case. When you say "the life of one victim is too many", it feels impossible to disagree with you, but it does also mean that we are likely decades away from having self-driving cars, during which millions will die from cars (worldwide). If you accept the premise that self-driving cars will be substantially safer than human drivers at some point, what is the right thing to do?

shakna5• 8 years ago

From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision. But self-driving cars have more sensors and, in theory, more capabilities to prevent something like this, and we absolutely should expect them to do better.

Intel and Volvo tested against the terrible dashcam footage. If Uber hadn't disabled the XC90's safety systems, this wouldn't have happened. [0]

The systems today can do better.

[0] https://www.bloomberg.com/news/articles/2018-03-26/uber-disa...

takeda4• 8 years ago

Based what I learned about Uber their self driving cars probably run on NodeJS.

I'm only half joking.

justicezyx• 8 years ago

O.M.G.

bryanrasmussen• 8 years ago

I don't actually know what a self driving car should run on, I would want to make it in Erlang or Rust but - what do you think a self driving car should be running on?

nafey• 8 years ago

VB Script

komali23• 8 years ago

Php

colejohnson66• 8 years ago

Batch

piinbinary• 8 years ago

Some kind of realtime system. A GC pause of 200ms in a car could easily cause a crash.

Sadly, I fear it may actually be written in C++.

golergka• 8 years ago

Excel spreadsheet

avip• 8 years ago

The blockchain

carlmr5• 8 years ago

For real-time embedded there are basically 3 options, C, C++ and Rust. And Rust isn't that widely used, so my guess would be on C/C++. You can't have a GC.

takeda4• 8 years ago

Hard real time system. Meaning every operation has a hard deadline and the car absolutely needs to react before then. No unknown delays because of garbage collection etc.

Sadly, I suspect that most of these self driving cars don't do that, because RT is hard.

piyushpr134• 8 years ago

I thought it was java and it had stop the world GC running when it killed

dang3• 8 years ago

We detached this subthread from https://news.ycombinator.com/item?id=16684651 and marked it off-topic.

johngalt2• 8 years ago

Several comments here making comparisons to human drivers. As if autonomous cars are ok so long as they are at or below parity with humans. This is statistically true but a complete misunderstanding of human nature.

Human risk tolerance varies drastically depending on control/participation. What is the acceptable casualty rate of elevators?

buvanshak4• 8 years ago

Yes. Extremely valid point, yet somehow overlooked in these discussion till now.

But can you explain how, if it is statistically true, can have a different real world outcome?

eyeing_see• 8 years ago

On top of that, I think it is absolutely rational to expect autonomous cars to perform at the current state of the art. We tolerate certain kinds of bad human drivers, like beginners, because there is hardly an alternative. A self-driving car with the driving skills of a beginner would be completely unacceptable if the state of the art has skills comparable to, say, somebody with a few years of experience.

carlmr5• 8 years ago

We don't really know yet what state of the art is for autonomous vehicles. Until we have gathered a bit more data from different companies we can't say anything.

This accident might have been a fluke, it might have been caused by bad engineering, it might have been caused by many things. We can only compare once we have a sizeable sample of incidents, or a long enough time of non-incidents that we're confident the system works well.

And every software update could change something about it.

emiliobumachar• 8 years ago

We should strive to be better as a society.

(not that it applies in this case. Uber's cars compare disfavourably to human drivers)

jMyles• 8 years ago

Several comments here making comparisons

On one hand, you are (reasonably) calling out comments for questionable comparisons, but on the other hand...

What is the acceptable casualty rate of elevators?

...you've made one of your own.

I haven't yet formed an opinion on the underlying question (of what kind of risk aversion is appropriate in the face of automation), but I do notice that it evokes questionable comparisons from all sides.

mtgx3• 8 years ago

His point was that machines should be held at much higher standards than humans.

This is a point I'e been making since the beginning myself. If you want self-driving cars to be accepted, they need to be at least 10x better than the best human drivers. Comments such as "self-driving cars only need to be 50% better than humans on average" are absolute nonsense. Humans will never go for that.

tim3333• 8 years ago

Dunno if someone/something is driving my cab I'd go with the one that was safer.

I note elevators used to be controlled by human operators - you don't get much of that these days https://en.wikipedia.org/wiki/Elevator#Manual_controls

kosei• 8 years ago

“Improving public safety has always been the emphasis of Arizona’s approach to autonomous vehicle testing, and my expectation is that public safety is also the top priority for all who operate this technology in the state of Arizona,” Mr. Ducey said in his letter. “The incident that took place on March 18 is an unquestionable failure to comply with this expectation.”

This after the police claimed no fault.[1] Not quite sure how those two statements square...

[1] http://fortune.com/2018/03/19/uber-self-driving-car-crash/

galdosdi10• 8 years ago

Two different and unrelated agencies and levels of government with different incentives.

The state has strong precedence over local authorities though, so whatever some garbage local keystone kops have to say is kind of irrelevant here. Local coppers are known in general to have a bias in favor of motorists for obvious reasons...

shakna5• 8 years ago

The investigation isn't complete, but Uber did disable the car's own safety features [0].

Both Volvo and Intel tested their software against the low-grade video that was released, and it was able to detect the impending accident and reduce the impact to where the pedestrian would likely have survived.

[0] https://www.bloomberg.com/news/articles/2018-03-26/uber-disa...

icebraining7• 8 years ago

Uber did disable the car's own safety features

Well, of course they did. Can you imagine testing a self-driving system when another system is also controlling the car? What happens when they send contradictory commands? Race conditions are bad enough outside of actual cars!

paul7986• 8 years ago

How is this company still in business...karma is chasing it all the way down to oblivion!

smnrchrds• 8 years ago

Outside of Hacker News, I have never encountered anyone who followed, was fully aware of, or cared about Uber scandals. From my conversations, it seems that people in my city hate taxi companies so much that it takes more than a couple of manslaughters to make them stop using Uber.

peterept• 8 years ago

I don't know much about the logic used in self-driving cars - but I do wonder how they will handle roundabouts. We have a lot of them here (Australia) and you have to give way to a car that is approaching or entering the roundabout on your right - which often is opposite you. How will a self-driving car cope with detecting a vehicle behind the concrete barrier of a roundabout?

pfarnsworth• 8 years ago

In Mountain View, I once saw a Google/Waymo car trying to cross traffic and turn left onto Rengstorff Ave, which is a pretty busy street in the evenings. As a human driver, I wouldn't know how to do it unless there was a miraculous break in traffic, I would have just turned right and go the opposite direction and make a u-turn elsewhere. So like you, I'm curious how it handled that situation as well.

kayoone• 8 years ago

In Europe we have a lot of roundabouts that work the opposite way, priority is given to anyone inside/exiting the roundabout, which would seem easier to handle for automated systems.

Symbiote• 8 years ago

I think that's what is meant, remember Australians drive on the left.

rimliu2• 8 years ago

Also in Europe it depends what road signs are at the roundabout. They have no special status in my country and can be looked as a "T" intersections connected to a circle. In most cases there is a yield sign for the cars approaching the roundabout so the cars already on it have right of the way, but it is not the case for all roundabouts. Also, some roundabout allow exit from the two rightmost lanes, others just from one—it also depends on signage.

caf3• 8 years ago

No, the roundabout rule is that a vehicle entering a roundabout must give way to any vehicle already on the roundabout, or a tram that is entering or approaching the roundabout.

See the Australian Road Rules, Part 9: https://www.pcc.gov.au/uniform/Australian-Road-Rules-19March...

If you are approaching the roundabout and someone enters it from your left before you get there, you have to give way to them.

chrononaut• 8 years ago

Isn't this what the original comment is saying though? Considering the cars are driving on the left-hand side of the road, cars in the roundabout are approaching from the right with respect to the vehicle, not the left. Or am I misunderstanding something unique for roundabouts in Australia? The way you describe it seems opposite of intuition that vehicles already within the roundabout do not have right of way.

mschuetz• 8 years ago

Things like that may vary from country to country, though. In Austria, roundabouts do not have a special rule and therefore a car approaching the roundabout theoretically has the right of way. But pretty much any roundabout has a yield sign, I've never encountered one without a yield sign at the entrance. Doesn't mean that there can't be a roundabout that does not have one.

grecy• 8 years ago

you have to give way to a car that is approaching or entering the roundabout on your right - which often is opposite you. How will a self-driving car cope with detecting a vehicle behind the concrete barrier of a roundabout?

How does a human driver detect a vehicle behind a concrete barrier?

Either humans can see it and the intersection makes sense, of humans (and machines) can't see it and the whole setup does not make any sense.

rjvs• 8 years ago

If they are still on the other side of the roundabout (and there are no other cars), can't you safely enter the roundabout anyway?

My understanding is that the goal behind a roundabout is to create an intersection where any vehicles that enter are traveling in a similar direction, improving the capability of people to avoid crashing and reducing the seriousness of crashes that do occur between vehicles.

PeterStuer• 8 years ago

Uber and responsibility doesn't seem like a great cultural match.

philip12092• 8 years ago

This seems like a lesson in why government regulation can be good.

coryfklein3• 8 years ago

The government regulation allowed the Uber on the road, so it may be a lesson in why government regulation doesn't work or provides a false sense of safety.

throwaway2048• 8 years ago

clearly the better alternative would have been no regulation, and uber continuing to operate (???)

philip12092• 8 years ago

I specifically mean that Arizona chose to allow self-driving cars on their roads with little regulation and that lack of regulation contributed to the death of a pedestrian.

Arizona Government Suspends Uber's Self Driving Cars from Roads

Related Stories