Self-driving Uber car kills Arizona woman crossing street

https://www.reuters.com/article/us-autos-selfdriving-uber/self-driving-uber-car-kills-arizona-woman-crossing-street-idUSKBN1GV296

by kgwgk • 8 years ago

2361 1727 8 years ago

reuters.com

bitxbitxbitcoin4• 8 years ago

On purpose to protect something "more valuable," or on accident?

cek• 8 years ago

There's an old saying around gun owners: "There's no such thing as a firearms accident. Only negligence."

I think the same thing applies to autonomous things.

si1entstill• 8 years ago

I feel that this sentiment could be applied to almost anything and isn't really meaningful. Almost every accident ever could be attributed to negligence somewhere.

jaxbot3• 8 years ago

In NYC the urbanists have a trend of saying "crash" instead of "accident" when referring to traffic collisions. They feel that "accident" gives the driver too much credit in situations where they were completely at fault for not looking, going too fast, running a light, etc.

neon_electro• 8 years ago

Agreed; more generally, there's a movement to avoid calling these incidents "crashes" instead of "accidents", since it's arguable that crashes are avoidable: https://www.crashnotaccident.com/

bitxbitxbitcoin4• 8 years ago

I agree completely. Here, there's perhaps negligence of the system, and the driver that was supposed to be overriding it in situations like this.

jakelazaroff3• 8 years ago

This is a dumb saying that ignores the fact that humans are fallible beings who make mistakes.

Sure, you might shoot yourself because you're just careless with your gun. Or you might be also thinking of an argument you had with your spouse, or you might be tired when you're handling your everyday carry on your way to work, or you might get startled, or a million and one other things that might prevent you from singly focusing your attention the gun you're handling.

Machines exist to serve us, not the other way around. If a product doesn't take human habits and flaws into account, it's a bad product, because over a large enough sample size someone will make a mistake.

xexers2• 8 years ago

Forgive me, I don't understand what you mean. Please elaborate.

bennyg• 8 years ago

They're basically asking if the car avoided doing something that would have been more destructive (e.g not hitting 2 people but only hitting one person instead). Or if there was no conflict decision making that happened and the car accidentally ran into the person and it resulted in a fatality.

- AI decides between one life lost as a better potential outcome

- AI made no decision and just happened to hurt someone

delete2• 8 years ago

Please don't, he's going to launch into some philosophical dilemma that's not relevant because these cars aren't making decisions.

hughes• 8 years ago

I think it's a reference to the imaginary trolley problem, which is often raised as an objection to self-driving capability.

The idea is that making a decision between killing one person vs killing something more valuable than one person (eg. multiple people) is something that drivers do on a regular basis. Computers may not be capable of correctly evaluating the ethics of that decision.

suresk• 8 years ago

One of the most oft-discussed questions around self-driving cars is what should happen if your car can choose between two options: 1) An action that will have a better chance of protecting the people inside the car, 2) An action that will protect people outside of the car, with a whole bunch of variations.

For example - What if someone steps out into the road and you can either hit them, or run into a wall? The former makes it far more likely the occupants of the car will be safe, the latter saves the life of a pedestrian at the expense of possible injury to people in the car.

It is kind of an interesting question, because people can't generally react quickly and rationally enough in a situation like that for it to matter, but in theory, a computer can. But it also dominates the discussion in a way that likely far overstates how frequently this will actually be a decision a computer has to make.

I think that is what the person you're replying to is asking. Given that this occurred at night and on (very likely) uncrowded streets, I feel like its far more likely the car (and backup driver) just somehow missed the pedestrian.

davidkuhta• 8 years ago

I believe they are asking whether that the vehicle struck pedestrian A in the course of avoiding something else like a pedestrian B.

mattlondon• 8 years ago

I suspect they mean the trolley car problem: "you are either going to crash into a group of school kids, or into a group of nuns. Who do you kill?"

I.e. did the uber car kill this 1 pedestrian, because it avoided killing 2 other people? Did the uber car opt for the "least killing" in this siutation?

I think this whole trolley car question - while a nice philosophical question - is silly though. I dont think humans would do any better in an emergency situation. In that split second when you suddenly find yourself in the "oh sh-<IMPACT>" situation, do you have time to a) make a deliberate, reasoned & fully-informed decision, and then b) control the vehicle effectively to follow through on that decision? I doubt it. If you had a second or two to think it through and then deliberately make a choice and steer towards that crowd of nuns, you'd probably be able to entirely avoid the accident anyway.

More likely is you'd probably do what I am sure most people do which is gasp, stamp on the brakes, shut their eyes and hope for the best ... assuming you even had time to realise there was about to be a crash before it happened. Some people might swerve, but I suspect they do that instinctively to avoid something in their way, not as a decision to hit something else.

I just dont think self-driving cars will get themselves into situations where they have to chose who to kill in the first place. And even if they did get into those situations, I really, really doubt it would be due their actions, and I'd certainly trust it to avoid the crash in the first place a whole lot more than the average human driver in the same scenario.

Some could argue "Ah but yes computers are so fast that they CAN make that informed decision about who to kill in milliseconds! So they have to make a choice! The question stands!". I'd argue back that in those milliseconds before the crash was inevitable, they'd act to prevent the accident before a human would even know what the hell was going on anyway. After that it just starts getting into a game of who can conjure up the most ludicrous hypothetical situation that rarely - if ever - happens in real life.

toofy• 8 years ago

Yeah, it will be interesting to see if this was an intentional (algorithmic) response or a failure of sensors somewhere.

I go back and forth with thinking the algorithms in things like self-driving cars should be public. Anything which must make a choice such as smash into bus or smash into bicycle — the public should be aware of the choices the company chooses to implement. Obviously this example is simplistic but it illustrates my point.

Today I’m definitely leaning towards it should be public and scrutinized. Next month I may be worried this would hamper progress.

bitxbitxbitcoin4• 8 years ago

I'm pretty sure everyone involved and observing also vacillate between those two mindsets. Regardless, events like this will likely hamper progress.

zawerf• 8 years ago

Is this the first death ever?

(Do we count Tesla's autopilot as self-driving?)

CocaKoala• 8 years ago

Hm, there have been other crashes where the 'driver' in the car died (I recall one where a car in autonomous mode rear-ended a truck, something about how the sensor was blinded by the setting sun? And it had alerted the driver to take control several times before the crash) but this is the first incident I can recall where a pedestrian was struck and killed by an autonomous vehicle.

stevenwoo3• 8 years ago

In the Tesla one the sensors did not recognize the white trailer of an 18 wheeler against the sky from news reports.

jandrese2• 8 years ago

IIRC in that case the driver T-boned a truck crossing the road, because the side of the truck was so shiny that it read as the horizon to the car's sensors.

twinkletwinkle• 8 years ago

I believe it's the first "third-party" death.

_m8fo• 8 years ago

Obviously more information is needed, but I thought the entire point of having a driver behind the wheel is to manually intervene to prevent this very situation?

I'm very curious to see how they'll investigate this and who will be determined to be at fault (person behind the wheel or Uber). It will likely set a precedent.

annabellish2• 8 years ago

Human beings simply cannot switch between "not focussed" and "in charge of a car, taken out of autonomous mode, and actively avoiding collision" fast enough to avoid most accidents, unfortunately. Neither can humans maintain the focus required to be ready to do that when 99.9% of the time they're not required to do anything.

Semi-autonomous cars have drawbacks.

sandworm1012• 8 years ago

Which is why the machine should be backup for the always-driving human, only leaping in to correct failings.

tluyben2• 8 years ago

Human beings simply cannot switch between "not focussed" and "in charge of a car

This is not how many people drive in non-autonomous cars? Commuting is on auto-pilot for many of us; you don't really remember how you got to work and I see people tying their shoelaces, calling, eating sandwiches, reading the paper, chatting on whatsapp all the time. Sure it's all illegal (in a lot of countries), but people are bored and drove that road 1000x.

pavon• 8 years ago

I agree with what you are saying, and that it would be very dangerous to sell cars where drivers may be lulled into complacency of thinking they don't need to pay attention to the road, until self driving cars get good enough that human attention really isn't needed.

However, these are test vehicles. The driver's full-time job is to be focused on the road and what the car is doing. They shouldn't have any misconceptions about the need for them to stay focused on the road. Now granted, even then attention will lapse occasionally, just like it does when you are actually driving. But I don't think that being physically in control of the wheel is necessary for maintaining focus at the same level as a good driver. Driver's Ed instructors need this skill. Also some back seat drivers I know are quite good at maintaining strong focus on the road regardless of who is driving :)

rocqua8• 8 years ago

How about autonomous on the highway, turnpikes, etc. And human-operated on streets where a car might ever need to yield to pedestrians or cyclists outside of red-lights.

spuz2• 8 years ago

Certainly, humans will never be perfect but there is the wicked problem of how we interact with systems that behave correctly 99% of the time but require human attention the other 1%. There was an excellent Econ Talk podcast about this topic a while ago:

http://www.econtalk.org/archives/2015/11/david_mindell_o.htm...

However, in my opinion, the driver and by extension his employer Uber must be held responsible in this case. Uber have a license to test their self-driving cars only if there is a human driver in control at all times. Regardless of the crazy things their software decides to do, the human is there as a person of ultimate legal liability.

TillE2• 8 years ago

The only way this is a reasonable failure of both the computer and the human driver is if they both physically had no time to react. Maybe that was the case, maybe it wasn't.

drunken-serval• 8 years ago

Given these cars have been blowing red lights, the human drivers aren't doing their jobs and overriding the car.

lazerpants• 8 years ago

I'm very curious to see how they'll investigate this and who will be determined to be at fault (person behind the wheel or Uber). It will likely set a precedent.

It's also plausible that the pedestrian was at fault. Under some circumstances it would be impossible for a human driver to avoid collision with a pedestrian, this may be no different.

mikepurvis• 8 years ago

Motor vehicle occupants are much more likely to survive than pedestrians and cyclists are. But blaming victims (especially dead ones) has a long and storied history as far as law enforcement and the road.

Hopefully with self driving vehicles we can start to move beyond this attitude, especially when there should be ample camera footage available to help dispel the usual claims that the person struck "came out of nowhere" or whatever.

ocschwar• 8 years ago

" Obviously more information is needed, but I thought the entire point of having a driver behind the wheel is to manually intervene to prevent this very situation?"

I think it's time to point out the obvious, and require that autonomous cars apply the brakes first, and THEN require driver intervention.

And that they be a whole lot quicker to err on the side of braking.

Cameras getting fuzzy ?

Slow down.

Your ML algorithms are showing lower confidence measures for how they classify nearby objects and trajectories ?

Slow down.

nearby vehicles slowing down and you don't know why?

Slow down.

This is inexcusable.

throwawaysecops• 8 years ago

Concioussness requires a body. This is how it is, here.

admax88q• 8 years ago

Nothing I love more than HackerNews armchair engineering.

It's not that simple, you're assuming the car even had some indication that something was wrong. For all we know the car's vision was showing high confidence it saw an open road.

wolverine12• 8 years ago

Car rapidly approaches you from behind.

Slow down?

dabeeeenster• 8 years ago

It could have been the fault of the pedestrian. I'm not saying it is, just saying there is a third possibility.

savanaly• 8 years ago

If humans can't prevent themselves from having accidents 100% of the time when they're in total control of the vehicle, why should we expect they could prevent an accident 100% of the time when they're in partial control of the vehicle?

ashleyn• 8 years ago

Uber's self-driving program seems to be one of the most unethical, most poorly-run in the entire industry. There were plenty of reports before warning us of the questionable quality of their research: cars changing lanes suddenly, blowing red lights, and now one finally killed someone. Every prior incident should have been treated as a serious matter.

Waymo et al has had none of these issues. Time to revoke their licence to test?

Mononokay4• 8 years ago

Uber's self-driving program seems to be one of the most unethical, most poorly-run in the entire industry.

I mean, it's Uber. I'm not sure exactly what people were expecting. Their entire schtick is being unethical and poorly-run.

reddit_clone• 8 years ago

Yeah. Their motto seems to be 'easier to ask forgiveness/pay fines instead of asking for permission'.

ProAm2• 8 years ago

Par for the course in regards to Uber.

SubiculumCode• 8 years ago

We do not yet know whether another autonomous vehicle or human for would have done better in this situation. THAT SAID, asshole company fields asshole autonomous driver. Did we expect differently? Uber cuts every corner.

synaesthesisx• 8 years ago

Yup - I've personally witnessed one of these almost zoom through a red light at an intersection and was appalled. I'm honestly not surprised...in their effort to race to the bottom Uber's autonomous vehicle program has been extremely aggressive - to the point of total disregard for safety. Their business model may be built on pipe dreams of autonomous travel (without drivers to pay), and hopefully this shutters that. In fact, the whole company should probably be shut down IMO - but I digress.

danso30• 8 years ago

IIRC, Uber initially launched its self driving cars by flouting California law: https://www.google.com/amp/s/www.cnet.com/google-amp/news/ub...

Mononokay4• 8 years ago

EDIT: Yeah this was a bit reactionary on my part. I get it, it's not over the $150.

It tells a lot about Uber that they weren't willing to fork over $150 for a permit.

brd• 8 years ago

Agreed. I've been in the cars in Pittsburg and the operators laughed about how poorly it performed on certain roads. Slanted telephone polls caused the vehicle to literally slalom down the road.

bitL• 8 years ago

We should wait for the results of investigation first. It could easily happen the killed person did some bonehead move that resulted in unavoidable crash. I was once told how a friend of mine fatally hit an older man that decided to cross the highway quickly, giving him no chance to react and a life-long trauma.

dmix4• 8 years ago

Calls to wait for facts never stopped the internet outrage machine in the early moments of news release. I doubt it will do much here, sadly. But it's worth pushing back against, even if it won't work IMO.

People will always relish in their biases being confirmed... even after a rare one-off event with a sample size of 1 in a burgeoning industry.

barkingcat• 8 years ago

Their San Francisco testing program didn't even have a license to test because they didn't want to pay for a permit.

https://www.theverge.com/2017/2/27/14698902/uber-self-drivin...

Nothing to revoke if they never had a license to begin with!

dmix4• 8 years ago

Not sure how that regulation would have made their cars safer in a way that's relevant to this accident?

Local governments have stopped other self-driving companies before with cease-and-desist letters with threats of serious punitive fines. I highly doubt by not buying a $150 license Uber would suddenly have carte-blanche to do what they want, that's not how it works.

If the state gov was really interested in stopping them they easily could have regardless.

tom_mellior• 8 years ago

Fully agreed regarding Uber's ethics, but...

Waymo et al has had none of these issues.

... absence of negative media coverage doesn't prove anything.

pyjammas• 8 years ago

You'd think Uber would be extra-vetted before they were allowed to do something like this.

amelius4• 8 years ago

Waymo et al has had none of these issues. Time to revoke their licence to test?

I think Alphabet is to blame here too in a sense (or at least the industry as a whole). Why didn't they push for more strict testing rules? It's in their interest too.

sevensor2• 8 years ago

This is how cars came to be on the road in the first place. Despite the fact that they were erratic and deadly, and spewed noxious gas everywhere, the manufacturers and automobiling clubs succeeded in convincing everybody that roads were for cars. Uber may well be the worst actor in the self-driving industry, but the industry as a whole is going to need to invent new crimes akin to jaywalking. This will allow it to shift blame to those killed by autonomous vehicles, just as jaywalking implies pedestrians are to blame for being hit by manned vehicles.

yaps8• 8 years ago

Yeah I missed that but it doesn't look good [1] (december 2016).

[1] https://www.theguardian.com/technology/2016/dec/19/uber-self...

nodesocket• 8 years ago

Striking a pedestrian in a crosswalk is horrible and terrible news. The reality though is how many pedestrians are hit by manual driver's vs automous drivers? On a percentage basis I gotta believe autonomous cars are orders of magnitude safer. Self driving cars aren't going to be perfect. How many people lost their lives in early factories to machines during the industral revolution? Imagine if they had pulled the plugs back then.

kamakazizuru• 8 years ago

yes, but with a human driver you have someone to take responsibility / punish for, what do you do with an alogrithm to provide justice?

etskinner• 8 years ago

Fine/sue the entity that made the algorithm, just like every other situation like that. Or, if you can prove negligence on the part of someone inside that company, prosecute them.

Maybestring3• 8 years ago

What do you do with an alogrithm to provide justice?

Assuming Arizona hasn't produced law specifically for fatalities in their public road self driving test program... The algorithms are just one detail of the total system design, and the design of the system isn't really the issue. People don't get killed by designs or code.

The verification of it's safety, and the decision to deploy that system on uncontrolled public streets will be the issue. People made these decisions, not an algorithm.

jandrese2• 8 years ago

It appears the woman was not using the crosswalk, but really that doesn't make much difference in the reaction to the report. Uber's self driving tech seems a bit under-baked at the moment.

cimmanom• 8 years ago

I always take the "outside the crosswalk" reports with a grain of salt. Hereabouts, if the pedestrian lands outside the crosswalk, they're assumed to have been crossing outside of it, even if later eyewitness reports and video footage show they were in the crosswalk and the car knocked them out of it.

bkohlmann• 8 years ago

Doesn't mitigate the tragedy or implications for autonomy, but per the article, the pedestrian was outside the crosswalk.

simias• 8 years ago

If that's something the software can't handle well then it's going to be a huge problem in many parts of the world. In many places in Europe, Africa and Asia for instance pedestrians will cross anywhere and everywhere at any time.

SllX2• 8 years ago

Does it really need to be said that an autonomous vehicle still needs to manage to not kill people regardless of their position relative to any nearby crosswalks?

Today you can cross in the middle of the street and take a calculated risk that the people approaching down the street will stop or slow down enough to not kill you. If they go ahead and kill you anyway, they'll still be prosecuted because their attention should be on the road, and the most basic assumption behind their driver's license is that this person is competent enough and fit enough to drive what amounts to a deadly weapon without killing the people around them. When you violate that assumption, the fault is probably on you. Obviously there are exceptions, such as when somebody intentionally jumps in front of your vehicle, but crossing outside of the crosswalk to get across the street is not even close to the same level as trying to commit suicide and fault will be found accordingly.

mattnewton• 8 years ago

I don’t know the number of hours their system has been driving but I highly doubt it is enough to draw comparisons to the average Arizona driver who hits 0 pedestrians. I also don’t think this works like this in the court of public opinion anyways. This system can never make a mistake of this magnitude and be publically accepted.

learnstats2• 8 years ago

The reality though is how many pedestrians are hit by manual driver's vs automous drivers? On a percentage basis I gotta believe autonomous cars are orders of magnitude safer.

That might be a fair question, except that one death at this early stage of limited realistic trials makes me doubt the correctness of your belief. On the statistics that we have right now, I'm not seeing how autonomous cars are safer.

Self driving cars aren't going to be perfect.

Shouldn't self-driving cars be held to a better standard rather than compared generously to objectively terrible existing standards? That we think of pedestrians regularly killed in "accidents" as acceptable collateral is already pretty horrific.

danso30• 8 years ago

Our society has evolved quite a bit from the times of the Industrial Revolution. It’s possible for progress to be made while having life-saving regulations.

JackFr• 8 years ago

On a percentage basis I gotta believe autonomous cars are orders of magnitude safer.

Why?

NHTSA reports 1.15 fatalities per 100 million vehicle miles travelled in 2015.

Waymo advertises >5 million road miles travelled. Let's say Waymo + Uber have driven 10 million vehicle miles and killed 1 person. That makes them 10 times more dangerous than human driven cars.

https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...

keymone• 8 years ago

what about tesla though? by november 2016 they've accumulated 1.3 billion self droven miles [1]. autopilot was released in september 2014, so that's 2 years, or 600 million miles per year. and that was ramping up during those years, so i think it's safe to assume at least 1 billion miles / year these days.

2 deaths in over 2 billion miles is roughly 10 times safer than human drivers.

[1] https://electrek.co/2016/11/13/tesla-autopilot-billion-miles...

lukev2• 8 years ago

This is the important statistic. Evidence so far points to autonomous drivers being worse than human drivers. In which they definitely have no place on the streets without a lot more R&D.

oldgradstudent• 8 years ago

The problem is that you 'believe' it to be orders of magnitude safer.

How about we test it properly before allowing it on public roads?

gowld17• 8 years ago

How do you test it not on a public road?

leggomylibro• 8 years ago

FTA:

"The Uber vehicle was reportedly driving early Monday when a woman walking outside of the crosswalk was struck.

...

Tempe Police says the vehicle was in autonomous mode at the time of the crash and a vehicle operator was also behind the wheel."

That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as when someone walks out into the road. Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.

On the other hand, it sounds like it happened very recently; I guess we'll have to wait and see what happened.

m3kw9• 8 years ago

They will have video for sure if since they are testing, then we will see. Your statement about reaction time is assuming so many things, may as well declare the AI guilty now right?

jakeogh8• 8 years ago

It's not "intelligent", it's a dumb computer that is programmed by humans. The people responsible for putting it on the road must be liable.

dragonwriter6• 8 years ago

may as well declare the AI guilty now right?

AI (short of AGI) is never guilty; it's creators and operators, OTOH...

giarc• 8 years ago

I imagine these vehicles (especially test vehicles) are also recording regular video and therefore getting a clear picture of what happened should be straight forward.

joosters• 8 years ago

This is Uber, who designed systems for remotely wiping entire offices when they thought that the data might be called for by the authorities.

Perhaps there will be unexpected 'issues' meaning that such data will have been lost in this case.

jonlucc• 8 years ago

I agree completely. One thing that is important to me is that the whole self-driving field will learn from every mistake. In other words, every self-driving car should only make the same mistake once, whereas with humans, each human has to learn only from their own mistakes.

babesh• 8 years ago

That isn't correct. Culture is learning from other people's mistakes. The only question is whether the human can and is willing to accept the lessons (and whether the lessons are correct).

guelo2• 8 years ago

Uber's algorithms might learn from this. None of the other self driving car companies will.

uptown• 8 years ago

If there's one company who hasn't demonstrated it's learned from its mistakes it's Uber.

But let's extrapolate. Say one day there are 20 self driving car companies. Should they be required to share what they learn so the same mistakes aren't repeated by each company or does the competitive advantage outweigh the public benefit from this type of information sharing?

dawnerd2• 8 years ago

Airlines are the same way. Every time there’s been a crash they’ve learned something and changed procedures to prevent it again. It’s made flying pretty safe, unless you’re an animal flying on United...

aetherson5• 8 years ago

I have seen no evidence that any autonomous vehicle currently deployed can react faster than an alert and aware human. The commentariat tends to imagine that they can, and it's certainly plausible that they may eventually be. But I've never seen anyone official claim it, and the cars certainly don't drive as though they can quickly understand and respond to situations.

jessriedel10• 8 years ago

I have seen no evidence that any autonomous vehicle currently deployed can react faster than an alert and aware human.

The argument I've always heard is that an autonomous systems will outperform humans mostly by being more vigilant (not getting distracted, sleepy, etc.) rather than using detectors with superhuman reaction times. Obviously, whether or not this outweighs the frequency of situations where the autonomous system gets confused when a human would not is an empirical question that will change as the tech improves.

wongarsu3• 8 years ago

"Alert and aware human" is already a high standard, given how most humans drive in routine mode, which is well understood to be much worse than "alert and aware".

From what I've seen I wouldn't trust autonomous cars to "understand" all situations. I would trust Waymo cars to understand enough to avoid hitting anything (at a level better than a human), at the risk of being rear-ended more often. Everything I've seen from Tesla and Uber has given me significantly less confidence than that.

dws• 8 years ago

I live and commute in Waymo country, and see evidence of quick reactions, though I can't say for sure whether it's an alert human taking over. Mostly, the Waymo vehicles still drive conservatively.

gldalmaso• 8 years ago

Not a truly autonomous vehicle example but this is a case where most likely the car reacted before the driver was even aware of a problem: https://www.youtube.com/watch?v=APnN2mClkmk

I agree with the sentiment though. This has been a major selling point for this technology, but it has not been sufficiently demonstrated yet.

0xfeba4• 8 years ago

Consider that autonomous braking systems know the distance required to stop and activate appropriately. Try getting a human to mimic that.

kerkeslager16• 8 years ago

That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as someone walking out into the road. Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.

That's an invalid conclusion to draw from this accident.

There were 34,439 fatal motor vehicle crashes in the United States in 2016 in which 37,461 deaths occurred. This resulted in 11.6 deaths per 100,000 people and 1.16 deaths per 100 million miles traveled.[1] As far as I know, the instance this article is about is the first death in an autonomous vehicle accident, meaning that all the 2016 accidents were humans driving.

Why is it that you see one death from an autonomous car and conclude that autonomous cars aren't ready to be driving, but you see 37,461 deaths from human drivers and don't conclude that humans aren't ready to be driving?

I admit that there just aren't enough autonomous cars on the road to prove conclusively that autonomous cars are safer than human-operated cars at this point. But there's absolutely no statistical evidence I know of that indicates the opposite.

[1] http://www.iihs.org/iihs/topics/t/general-statistics/fatalit...

EDIT: Let's be clear here: I'm not saying autonomous cars are safer than human drivers. I'm saying that one can't simply look at one death caused by an autonomous car and conclude that autonomous cars are less safe.

cryptoz• 8 years ago

don't conclude that humans aren't ready to be driving?

I'm not sure I see anyone here making that conclusion. I think you're the only one who's brought it up.

For one, I personally sure as fuck do not think humans are ready to be driving.

That's an invalid conclusion to draw from this accident.

The conclusion was not invalid at all. Other self-driving car companies have driven more miles than Uber has and they have done so safely. Uber has even taken its cars off the road, so even Uber agrees that their self-driving cars are not ready for the roads yet.

It is also important to take into account what Uber is like when it comes to safety and responsibility. They have knowingly hired convicted rapists for drivers, they have ridiculed and mocked and attempted to discredit their paying customers who have been raped by their employees/drivers, they have spied on journalists, they have completely disregarded human safety on numerous accounts. A company with a track record like Uber's probably not should be granted a license to experiment with technology like these self-driving cars on public roads.

They just aren't responsible enough.

chillingeffect• 8 years ago

1 July 2016 "Tesla driver dies in first fatal autonomous car crash in US"

https://www.newscientist.com/article/2095740-tesla-driver-di...

jdmichal3• 8 years ago

He's arguing from foundations. The hypothesis is that autonomous drivers should perform better at task X than the null hypothesis (aka human drivers). So any instances where autonomous drivers do not seem to perform better are all potential counter-arguments to that hypothesis.

The fact that human drivers aren't particularly good isn't really relevant, beyond setting a correspondingly low bar within the null hypothesis.

This all ties to regulation allowing these vehicles to drive on public roads, because that regulation was permitted due to the above hypothesis and hopeful expectations that it would be true.

Obviously, I haven't seen the entirety of the data set to know fatalities per car-mile. Which would be the relevant statistic here. I also didn't see such a number in your post, which I'm assuming means you are probably not aware either. But simply providing the numbers for the null hypothesis doesn't do anything.

vasilipupkin3• 8 years ago

you have to take into account miles driven. Yes, we have 37K fatalities, but trillions of miles driven. So, it comes out something like 1.2 fatalities per 100 million miles driven for human drivers. Which means, so far, self driving cars have a worse record per 100 million miles driven. So far.

rimliu2• 8 years ago

If solution does not solve anything, what's the point of the „solution“? Most of the deaths on the road I hear in my country are the result of someone doing something really reckless and stupid. „Normal“ drivers do not kill themselves or others. So if self-driving car is only as good as a dumb driver — I do not want these cars on the road. Also, interetingly enough there is little talk about human driver assisted by technology, not replaced by it. For some reason it is binary: either humand driver or self-driving. How about some human drivers plus collsion avoidance system, infrared sensing system (way too many people die there simply becaue they walk on the road in the dark without any reflectors/lights), etc.

danso30• 8 years ago

As of late 2017, Waymo/Google reported having driven a total of 4 million miles on roads. Given that Waymo is one of the biggest players, it's hard to see how all of autonomous cars have driven 100 million miles at this point.

https://techcrunch.com/2017/11/27/waymo-racks-up-4-million-s...

Nevermind that the initial rollouts almost always involve locales in which driving conditions are relatively safe (e.g. nice weather and infrastructure). Nevermind that the vast majority of autonomous testing so far has involved the presence of a human driver, and that there have been hundreds of disengagements:

https://www.theverge.com/2018/1/31/16956902/california-dmv-s...

Humans may be terrible at driving, but the stats are far from being in favor of autonomous vehicles. Which is to be expected given the early stage of the tech.

Twisell3• 8 years ago

Actually there was a metric of accident per distance drived. It was often pull out by Tesla before their first fatal accident last year (IA didn’t saw rear of truck because of sun or smthg like that).

This metric was often decried because it had poor statistical significance. It would however be nice to update this metric according to this death.

Maybe now this metric would indicate that IA are more dangerous than human, will be interesting if the perception of this flawed metric will evolve in reaction...

brbrodude2• 8 years ago

Apples and oranges.

randyrand• 8 years ago

Not bad even a self driving car can stop in time for deer jumping out from behind tree/bush cover.

People still need to look both ways when coming out from behind cars.

I want more details.

notheguyouthink• 8 years ago

That's a really good point that many (myself included) aren't mentioning. Stopping distance is a universal, reaction times be damned. Would be curious to see if that played a part.

jdietrich2• 8 years ago

That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as when someone walks out into the road.

Some of these accidents are unpreventable by the (autonomous) driver. If a pedestrian suddenly rushes out into the street or a cyclist swerves into your path, the deciding factor is often simply the coefficient of friction between your tyres and the road.

The autonomous vehicle and the human attendant might have made a glaring error, or they might have done everything correctly and still failed to prevent a fatality. It's far too early to say. It's undoubtedly a dent to the public image of autonomous vehicles, but hopefully the car's telemetry data will reveal whether this was a case of error, negligence or unavoidable tragedy.

Twisell3• 8 years ago

Yet this tragic accident will now set a precedent and finally start to scratch the delicate unspoken question: Who is responsible now?

soperj2• 8 years ago

The cars telemetry data will undoubtedly point to a unavoidable tragedy if it's just uber analyzing it.

oconnore• 8 years ago

If you have good judgement, it's quite easy to prevent a fatality. I.e. I go slower in places where I know pedestrians might jump out.

exelius• 8 years ago

I’m kind of in the Elon Musk camp here where you gotta break some eggs to make an omelette? Human-driven cars kill a lot of pedestrians today, but we can actually do something to improve the human-recognition algorithms in a self-driving car.

As long as self driving cars represent an improvement over human drivers, I’m ok with them having a non-zero accident date while we work out the kinks.

foolfoolz• 8 years ago

you seem to be giving uber a big benefit of the doubt. these autonomous cars generally go slow. tempe has flat roads with great line of sight and clear weather. coefficient of friction? highly doubt it. the sensors should be looking more than just straight a head

adamsea2• 8 years ago

It's not unreasonable for there to an expectation of basically zero accidents of this nature during testing in cities. The public puts a huge amount of trust in private companies when they do this. And, pragmatically, Google, Uber, etc, all know that it would be horrible publicity for something like this to happen. One would think they'd be overly cautious and conservative to avoid even the possibility of this.

Lastly, the whole point of the human operator is to be the final safety check.

You're right that we have no idea of the cause until the data is analyzed (and the human operator interviewed). Yet, my first thought was, "Of course it'd be Uber."

rockarage• 8 years ago

If a pedestrian suddenly rushes out into the street or a cyclist swerves into your path, the deciding factor is often simply the coefficient of friction between your tyres and the road.

This only true for the uninitiated, never let it get to that. I drove for Uber/Lyft/Via in New York city so I can experience and study these situation. These sort of accidents are preventable. The following is the basic:

1.) Drive much slower in areas where a pedestrian and cyclist can suddenly cross your past.

2.) Know the danger zone. Before people "jump into traffic" or a cyclist swerve in front of you, they have to get into position, this position is the danger zone.

3.) Extra diligence in surveying the area/danger zone to predict a potential accident.

4.) Make up for the reduce speed by using highways and parkways as much as possible.

It helps that Manhattan street traffic tends to be very slow to begin with. Ideally I will like use my knowledge to offer a service to help train autonomous vehicles to deal with these situation. It has to be simulated numerous times in a closed circuit for the machine to learn what I've learn intuitively driving professionally in NYC.

simplyluke2• 8 years ago

Agreed that it's too early to really say one way or another. In maybe 100k miles of urban driving I've had one cyclist run into my car and a girl on her phone walk directly into the corner of the front, I was at a complete stop watching them both times.

Until there's a detailed report it's really hard to say if it was preventable or not - but I think regardless the optics are bad and this is going to chill a lot of people's feelings on self driving whether or not that is an emotion backed by data.

zkms• 8 years ago

If a pedestrian suddenly rushes out into the street or a cyclist swerves into your path, the deciding factor is often simply the coefficient of friction between your tyres and the road.

This holds iff the control input you apply is only braking. Changing the steering angle is generally far more effective for the "pedestrian darts out from hidden place onto road" situation. It's far better to sharply swerve away to ensure that there's no way the pedestrian can get into your path before your car arrive there than it is to stand on the brakes and hope for the best.

Indeed, the faster you're moving, the more you should consider swerving away over braking -- take advantage of that lethal speed to clear the pedestrian's potential paths before he can get into yours.

Yes, this intentionally violates the letter of the traffic laws (and might involve colliding into a a parked or moving automobile on the other side of the road) and also involves potentially unusual manoeuvring on a very short deadline; but it's far better to avoid a vehicle-pedestrian collision even if it's at the cost of possibly busting into the opposing lane / driving off the road / hitting a parked car. Decently experienced drivers can do this, I can do this (and have successfully avoided a collision with a pedestrian who ran out between parked cars on a dark and raining night), there's no fundamental reason that computer-controlled cars can't do this.

acdha• 8 years ago

That’s technically true but the number of truly unavoidable cases is orders of magnitude lower. With human drivers, “jumped out in front of me” really means inattention 99.9% of the time and police departments historically tend to be unlikely to question such claims. (For example, here in DC there have been a number of cases where that was used to declare a cyclist at fault only to have private video footage show the opposite - which mattered a lot for medical insurance claims)

With self-driving cars this really seems to call for mandatory investigations by a third-party with access to the raw telemetry data. There’s just too much incentive for a company to say they weren’t at fault otherwise.

bargl2• 8 years ago

The hope is that AV can see 360 and observe things outside of blind spots further away. So the kid running from a yard into the road after a ball should be safer, but a person walking out from behind a parked truck wouldn't.

ubernostrum3• 8 years ago

If a pedestrian suddenly rushes out into the street or a cyclist swerves into your path, the deciding factor is often simply the coefficient of friction between your tyres and the road.

I mentioned in another comment that something I use to try to improve my own driving is watching videos from /r/roadcam on reddit, and trying to guess where the unexpected vehicle or pedestrian is going to come from.

Here's an example of a pedestrian suddenly appearing from between stopped cars (and coming from a traffic lane, not from a sidewalk), and a human driver spotting it and safely stopping:

https://www.youtube.com/watch?v=wYvKPMaz9rI

Why can't a self-driving car do this?

pishpash6• 8 years ago

The first martyr of the AI age, I suppose.

olympus2• 8 years ago

Could a human have reacted fast enough to stop for someone jumping out in front of them? If the person jumped out so fast that nobody could have possibly reacted in time, then it's not a stain on the technology- even with an instantaneous application of brakes, a car still takes a while to come to a stop. If the human jumped out ten seconds earlier and was waving her hands for help, then it's an issue.

"the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human"

This statement isn't really true. Do you think Uber is investing in this because it makes their passengers safer? No, they are pretty much immune to any car crashes with a human driver, the risk and insurance hit is assumed by the driver. They are doing this to save money. They don't have to pay a driver for each trip. The safety aspect is just a perk, but Uber will be pushing this full force as long as the self-driving cars are good enough.

I'm more concerned that Uber will "lose" the video evidence showing what kind of situation it was, and we'll never be able to know if a human would have had ample time to react.

pm24601• 8 years ago

The bicyclist did not "suddenly" jump out in front. The Uber car did not sense the bicyclist.

mtgx3• 8 years ago

I've been saying this from the beginning: it's not enough for self-driving cars to be "better than the average driver". They need to be at least 10x better than the best drivers.

I find it crazy that so many people think it is. First off, by definition like 49% of the drivers are technically better than the "average driver".

Second, just like with "human-like translation" from machines, errors made by machines tend to be very different than errors made by humans. Perhaps self-driving cars can never cause a "drunken accident", but they could cause accidents that would almost never happen with most drivers.

Third, and perhaps the most important to hear by fans of this "better than average driver" idea, is that self-driving cars are not going to take off if they "only" kill almost as many people as humans do. If you want self-driving cars to be successful then you should be demanding from the carmakers to make these systems flawless.

ingsoc793• 8 years ago

The main problem with self-driving cars is that they can't "read" humans' body language. A human driver can see pedestrians and cyclists (and other cars) and have a rough idea of what they're likely to do in the next few seconds, i.e. the pedestrian leaning out on the crosswalk curb is likely to step into the road soon. Even a reaction time of milliseconds can't make up for the (currently exclusive) human ability to read other humans and prepare accordingly.

sinatra2• 8 years ago

With sufficient data, I'd expect self-driving cars to be better at predicting what such leans mean. Moreover, for every one human driver who notices such a lean, there may be another human driver that doesn't even notice a pedestrian who has already started walking.

jupp0r2• 8 years ago

Are you sure that supervised learning does not create the same classification capabilities in self driving car AIs?

burfog2• 8 years ago

They also fail to "write" human body language. Nobody else can predict what the autonomous vehicle will do.

It gets worse when a person is sitting in what appears to be the driver's seat. If the car is stopped and that person is looking toward another passenger or down at a phone, nobody will expect the vehicle to begin moving. Making eye contact with the person in that seat is meaningless, but people will infer meaning.

gowld17• 8 years ago

Humans also can't read humans' body language. A pedestrian waiting at a corner isn't waiting for the weather to change. They are waiting for passing cars to stop, as required by law. But passing cars speed by instead of stopping, unless the pedestrian does a lunge into the street -- preferable a bluff lunge, since most drivers still won't stop, preferring to race the pedestrian to the middle of the lane.

danso30• 8 years ago

OT, but I would love to see how self-driving AIs handle something like Vietnam moped traffic and pedestrian crossings. The standard behavior for pedestrians is to walk slowly and keep walking -- stopping, even if it seems necessary to avoid a speeding driver, can be very dangerous, as generally, all of the street traffic is expecting you to go on your continuous path. It's less about reading body language than expectation of the status quo:

https://www.youtube.com/watch?v=nKPbl3tRf_U

mr_toad3• 8 years ago

I walk to work every day, and I can assure you that most human drivers have zero awareness of what pedestrians are about to do.

Even on crosswalks. If I just strolled out on a crosswalk without looking, and waiting for drivers who were paying no attention, I’d be long dead.

methehack• 8 years ago

Here's a times article you reminded me of that mentions Saab engineers putting on crash helmets and running the 9-5 into a concrete moose.

http://www.nytimes.com/1998/04/05/automobiles/behind-the-whe...

ohazi• 8 years ago

Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.

Who exactly do you think is granting Google, Uber, etc. approval for trials like this on public roads? It's going to be some bureocrat with zero ability to gauge what sort of safety standard these companies' projects have reached.

There are no standards here... what were you expecting would happen?

duxup• 8 years ago

It's an interesting dynamic. We want this tech to be much better than us terrible humans before we deploy it and anything like us terrible humans is not acceptable.

adamsea2• 8 years ago

I'd also add the goal of self-driving cars is to decrease costs for ride-sharing, home-delivery companies, etc, and also to decrease congestion via coordination amongst autonomous vehicles.

ForHackernews• 8 years ago

Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.

Uber cutting corners and playing fast and loose with legislation? Unheard of!

Here's hoping they get hit with a massive, massive wrongful death lawsuit.

YeGoblynQueenne3• 8 years ago

> Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.

They're on public roads because they've decided that deploying their tech is more important than safety.

I don't see any other explanation. We know that it's pretty much impossible to prove the safety of autonomous vehicles by driving them, so Uber (and almost everyone else) have decided that, well, they don't care. They'll deploy them anyway.

How do we know that? The report by RAND corporation:

https://www.rand.org/pubs/research_reports/RR1478.html

  Key Findings

  * Autonomous vehicles would have to be driven hundreds 
  of millions of miles and sometimes hundreds of billions 
  of miles to demonstrate their reliability in terms of 
  fatalities and injuries.

  * Under even aggressive testing assumptions, existing 
  fleets would take tens and sometimes hundreds of years 
  to drive these miles — an impossible proposition if the 
  aim is to demonstrate their performance prior to 
  releasing them on the roads for consumer use.

  * Therefore, at least for fatalities and injuries, 
  test- driving alone cannot provide sufficient evidence 
  for demonstrating autonomous vehicle safety.

dpark5• 8 years ago

That report ends by saying essentially, "it may not be possible to prove the safety of self-driving cars". [1] So the value here is questionable and the same logic could apply to anything with a low frequency of occurrence. The value of air bags by this measure was not proven until they were already mandated.

[1] "even with these methods, it may not be possible to establish the safety of autonomous vehicles prior to making them available for public use"

dragonwriter6• 8 years ago

That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as when someone walks out into the road.

No, the whole point of self-driving vehicles is that firms operating them can pay the cheaper capital and maintenance costs of the self-driving system rather than the labor cost of a driver.

Jeff_Brown• 8 years ago

On the bright side, with an autonomous vehicle, an accident is a learning experience that will benefit other autonomous vehicles.

iooi4• 8 years ago

The second ever [1] fatal car crash involving a self-driving car. This is the first one where a third party is killed.

[1] https://www.theguardian.com/technology/2016/jul/01/tesla-dri...

felippee2• 8 years ago

Tesla was not a self driving car, please stop confusing this.

This case is in fact the first autonomous test vehicle caused fatality.

There were at least several Tesla autopilot related fatalities and injuries, but I would seriously not put those into self driving bag.

iooi4• 8 years ago

Can you provide a source? From all the articles I can find, the language is all pretty much the same:

"Tesla driver killed while using autopilot"

"The Tesla driver killed in the first known fatal crash involving a self-driving car"

"It's been nearly a year and a half since Joshua Brown became the first person to die in a car driving itself."

"Tesla bears some blame for self-driving crash death, feds say"

imh• 8 years ago

There may be more than that. This one wasn't as heavily covered https://www.mercurynews.com/2016/09/14/second-fatal-crash-ma...

iooi4• 8 years ago

Interesting, apparently there's no way to know if autopilot was enabled or not.

A Tesla spokesperson said the car in the Chinese wreck was badly damaged and unable to transmit data. “We therefore have no way of knowing whether or not Autopilot was engaged at the time of the crash,” the spokesperson said in a statement.

nafizh2• 8 years ago

Tesla Autopilot != Self-driving(full autonomy)

jeremynixon• 8 years ago

Important and and missing:

"[Uber] said it had suspended testing of its self-driving cars in Tempe, Pittsburgh, San Francisco and Toronto"[1]

1: https://www.nytimes.com/2018/03/19/technology/uber-driverles...

kartan2• 8 years ago

Again... 'Uber has temporarily suspended the testing of its self-driving cars following the crash of one of them in Tempe, Ariz. The ride-hailing company had also been conducting experiments in Pittsburgh and San Francisco.'

I was resumed shortly after.

27 Mar 2017 (1 year ago)

https://spectrum.ieee.org/cars-that-think/transportation/sel...

basura0454785• 8 years ago

that's SOP for every av program. have incident? ground the fleet. doesn't matter why, or who's at fault.

itamarst• 8 years ago

The video says bicyclist and shows a bent bicycle. So pedestrian or bicyclist?

stingrae• 8 years ago

Perhaps they were walking their bicycle across the street.

jaxbot3• 8 years ago

It would explain the "outside a crosswalk" remark better if the person were riding the bike. This is one of those cases where video evidence should be subpoenaed, though. It's far too common for the police to accept the statement from the only party still alive, and come up with incorrect conclusions about what happened on the street.

AYBABTME• 8 years ago

Seems very unlikely to me that this case will be brushed off without police looking at video.

bmm6o• 8 years ago

I've heard that police reports often refer to all non-drivers as "pedestrians", even if they were riding a bike (which I learned after a neighbor was killed while riding a bike). I don't know if this is legalese or police jargon, but either way you would hope a journalist who covers these things could translate.

DFHippie• 8 years ago

A police officer in Nashville shouted at my wife for riding her bike in the road in accordance with Nashville law. Basically, he didn't know the law he was enforcing.

xapata2• 8 years ago

Unfortunately, many police forces don't train the police on how to handle bicycles. For example, in many places riding on the sidewalk is illegal, yet police will suggest a cyclist get off the road and ride on the sidewalk.

From studying many bicycle-related police reports, I was dismayed at how often I read the phrase "a pedestrian riding a bicycle."

Tepix• 8 years ago

That's dumb. The term means someone who is walking. "Per pedes" means "by foot".

justin66• 8 years ago

When I lived in California the press reported pedestrian and bicyclist deaths under the same "pedestrian" moniker.

danso30• 8 years ago

Given that so many of autonomous car accidents logged by the California DMV seem to be the fault of the human driver trying to take control, I’m surprised that the accident here reportedly happened in autonomous mode, which for all its flaws seems to have been good at not hitting things in front of it. In fact, seems like most of the CA accidents were the autonomous car being rear ended or side hit from other cars, usually for going too slow or stopping unexpectedly.

outside2344• 8 years ago

Given how incompetent Uber is at regulation of anything else in their company, is this really a surprise?

avoutthere• 8 years ago

It's a sad reality, but regardless of how many times per day a pedestrian is hit by a human-driven car, such incidents involving self-driving cars will be headline news for years to come.

ambivalents2• 8 years ago

Having a hard time sympathizing here. It's one thing to fear air travel based on the very few but very publicized plane incidents (considering all the data we have on the safety of air travel). It's another thing to hold these self-driving car companies accountable, considering a) the lack of data and history of such programs, and b) their touted benefit as a safer alternative to human-driven cars.

FeepingCreature• 8 years ago

It's not about sympathy. It's about one question: do they kill fewer or more people than humans per mile?

If they kill fewer people, they can be run by a joint venture of Satan and the Mafia for all I care.

asah• 8 years ago

As they should: humans fail independently, but autonomous cars can fail systemically, including hacking. Bicycles are known to be tricky for autonomous cars.

awakeasleep2• 8 years ago

Wow. This is dark but it feels real to me.

I had been assuming that self driving cars would be a 'reset' for our decision to value cars more than the countless lives they end. Self driving cars could take the irrationality and emotion out of driving. It would be America's "stop the murder of children" moment (The Netherlands rallying cry to reform auto traffic in 1970's).

But now I see that it can, and probably will go the other way. These giant companies could push to further restrict the rights of cyclists/pedestrians around roadways, vilify the 'stupid' people who made a reckless decision that resulted in an accident, and push our country even further into car culture.

waynecochran• 8 years ago

Even if its provable that self driving cars are 10x safer, every accident of this unfortunate kind will be used against autonomous cars. I don't see how to overcome this -- I don't expect reason will overcome the perceived danger.

brbrodude2• 8 years ago

You're goddamn blind. Yes, of course it matters more if I'm killed by an AI machine than it does if I'm killed by another human being. This is a new development, humans killing humans are not. What if it was autonomous warfare? It is surely coming about in a very near shipment, SOON. The world and history are not static, and techno-utopia is big fat lie, and the idea that things just get better as time advances frontwards is plain infantile. Reality is way more fucked up than you can imagine.

dang20• 8 years ago

You're goddamn blind.

We ban accounts that do this. Please read https://news.ycombinator.com/newsguidelines.html and follow the rules when posting if you want to keep commenting here.

timmaah• 8 years ago

The video mentions she was on a bike and shows a mangled bicycle. Also the line markings on the street indicate there is a bike lane to the left of a right turn lane. Definitely very bad.

https://www.google.com/maps/@33.4369934,-111.9429875,3a,75y,...

conductr5• 8 years ago

Not to be insensitive to the death. But, that bike lane design is just horrible. Are human drivers really known for respecting the "do not cross solid white line" rules? 9/10 drivers that realize at the last moment they need to turn right are going to quickly cross over this bike lane probably without much time to check their blind spot. If I was cycling on this path, I'd want to use the sidewalk and be extra cautious of peds, I don't trust drivers human or not.

mason55• 8 years ago

There aren't a lot of options when you need to have cars turning on roads with bike lanes. At some point the car lane and the bike lane have to cross paths unless you're going to build a fully grade-separated bike lane.

I suppose you could have a wall that separates the bike lane from the road except for a short area where cars can cross over, so that at least there's only a small space where the bikes & cars can interact, but that introduces a bunch of new problems (cars that can't see bikes behind the wall when they go to cross over, cars that run into the wall, probably even more swerving to get into the lane because your choices are do it RIGHT NOW or you're stuck/hit a wall).

avree• 8 years ago

Yes, it's been proven that un-protected bike lanes such as those are very dangerous. There are more and more separators coming out (https://peopleforbikes.org/blog/a-new-generation-of-bike-lan...) to allow for protected bike lanes.

However, I do need to mention that when you choose to bike on the sidewalk, you're creating the same type of problem that cars that ignore traffic rules create for bikes. There's a reason why biking on the sidewalk is illegal in most places, and why bike lanes/roads exist.

kerkeslager16• 8 years ago

I don't think that's insensitive to the death. It's properly placing (partial) blame on the cause of the accident.

A better design for that intersection would be to keep the bike lane against the sidewalk and the turn lane inside of that, so that they don't cross, and have a "no turn on red" sign, so that the paths of bicyclists and people turning in cars never cross as long as everyone obeys the stoplight and sign.

lsaferite• 8 years ago

AZ driving rules language on solid white lines is the same as FL (where I live). They use the words "should not cross", so it's not technically an infraction for a driver to cross a solid white line. I was in an accident once due to someone crossing a solid white while at a light and cutting me off so I'm familiar with the RAW on those lines.

mjevans• 8 years ago

I'm much more a fan of the design that places a buffer (either sidewalk or shrubs) between the CARS and any other type of vehicles. Sidewalks are tolerable as they are often raised above street grade and a vehicle usually has to be actively controlled to overtake such a raised divider.

sunsunsunsun• 8 years ago

I have been hit by a car in a bike lane by a car making a rogue left turn (In the UK). I'm always wary of riding in bike lanes like this now.

walshemj3• 8 years ago

Those sort of bike lanes teach cyclists that they should under take slower moving traffic - the problem is the majority of deaths in London are due to vehicles turning into undertaking cyclists.

rebeccaskinner• 8 years ago

Anec-dataly, I agree. We live at the edge of a residential area where there is a long stretch of road with bike lanes and turns into the residential area every block. Not only do people regularly drive in the bike lanes, the people who do so are typically the most aggressive and inattentive drivers who are trying to angrily speed around the lines of cars who are driving the speed limit and respecting the stop signs. I'm not aware of any recent accidents, but only because cyclists seem to just completely avoid the entire stretch of road.

prestonh• 8 years ago

That looks completely identical to all bike lanes I've seen in suburban areas. I think bike lanes are just inherently dangerous. Would be interesting to see what a well-designed suburban bike lane looks like.

biggc• 8 years ago

That bike lane is just a shoulder with bicycle symbols painted in it and it looks terrifying.

vorpalhex2• 8 years ago

This is both terrifying and normal. The bike lanes in my area are frequently used as turn lanes... by vehicles that don't fit in them - despite being clearly marked as bike lanes.

Human drivers are terrible.

LukeShu• 8 years ago

The car turn-lane is to the right of it, so at least it's maintained as part of the road. Bike lanes that are truly a shoulder with bike symbols painted on are often unmaintained, and full of pot-holes, debris, and such.

crankylinuxuser• 8 years ago

Yeah, they do the similar stupid "bike lane" crap here in Indiana as well.

Yeah, like a line is going to protect me from run amok car drivers who are too buy on their cell phones. Or it's going to stop the bus drivers from using them as pickup/dropoff areas, cutting bicyclists off.

Long story short, I bike on the sidewalks. I can be a responsible biker on sidewalks while heeding to pedestrians. My life is more important than to be biking on a vehicle road.

jdmichal3• 8 years ago

I've actually never seen a bike lane that looks different than that...

gowld17• 8 years ago

How about a "sharrow", where cyclists drive right-of-center, in hopes of avoiding the "parked car door lane", and encouraging drivers to cross the double-yellow to overtake.

https://en.wikipedia.org/wiki/Shared_lane_marking

username2232• 8 years ago

This design seems pretty common in the US. Some bike lanes on major streets in LA are like this, except there they also have to contend with buses pulling into stops. Biking on those streets would be terrifying, but I saw people doing it.

wdavew• 8 years ago

Yes, I bike almost daily on Venice Blvd in LA, and every intersection I encounter is setup like the one discussed here, along with buses weaving in and out. A vicious battle flared up last year when the city replaced a lane of traffic on Venice with protected bike lanes (some discussion here https://la.curbed.com/2017/10/25/16528864/road-diets-los-ang...). There seems to be a marked increase in the hostility between cyclists and drivers now.

jonnyreiss• 8 years ago

Unfortunately, this is a design that is pretty common in many of Tempe's major intersections. I pass through a similiar intersection biking home from ASU and have had multiple close calls with human drivers.

Anecdata: The times I have interacted with one of Uber's autonomous vehicles at this intersection, they tend to hit the brakes pretty hard as they are about to enter the turn lane and I am around 10ft behind them. All of my interactions with them, however, have been during the day, and it was unclear if it was the human driver in control.

steaknsteak2• 8 years ago

The video says bicyclist and shows a bicycle, while the written article says pedestrian and said the car hit "a woman walking"... I'm confused as to why there is a contradiction, but considering the video shot of the bicycle I assume you're correct

hardwaresofton• 8 years ago

It seems like a bunch of what-ifs that normally come up with self-driving cars are about to get answered and precedents are about to be set.

I assume this case will also be one of the most well-recorded cases of a fatal car accident in history as well, given the amount of sensors and equiment on-board a self-driving car, along with eye witness testimony from the operator on board.

Can't tell if Uber has just been incredibly unlucky as of late or if just enough of their employee-base is incompetent as to prevent them from just having a quiet year with no large failures.

odammit2• 8 years ago

Curious how the numbers work out of number of miles [1] to number of deaths of autonomous vs standard vehicles.

I know the scale is different but since this is the first death I’m curious if the percents fall in line.

[1] is distance the right metric?

jnbiche• 8 years ago

Perhaps time as opposed to distance? After all, we drive much fewer miles in a city, but there are much greater opportunities for accidents, particularly pedestrian accidents.

checkyoursudo2• 8 years ago

Maybe deaths per speed per time (or distance, since speed over time is the same as distance at speed). More deaths at higher speeds and fewer at lower would seem more likely. Purely using time or distance may skew the interpretation, if there are more deaths at some speed ranges than others.

prh8• 8 years ago

Yes, these statistics are typically based on miles driven.

jws• 8 years ago

This is an interesting metric. Although self driving test cars are rare, the whole point of their existence is to drive so they probably clock more hours than a normal car.

About 5400 pedestrians are killed each year in the US. US drivers go 3.1 trillion miles a year. So they kill a pedestrian about every 5.7 billion miles. Last November, Waymo said they had 4 million self driven miles, so well short of statistically expecting to hit a pedestrian. In September of some unspecified year Axios claims Uber had self driven over 1 million miles.

My estimate needs help, the distribution of pedestrian bearing roads and pedestrian free roads likely does not match from my total miles per year number and what carbot testers cover. Also, this may have been a cyclist death, which adds another 800 or so deaths per year.

But, in any event, in rough numbers, Uber appears to have beat their expected time to pedestrian fatality by two or three orders of magnitude.

haneefmubarak• 8 years ago

It occurs to me that because the deaths caused by autonomous vehicles may not follow the same distribution across types of deaths, it might make more sense to compare total deaths per million miles between human and autonomous drivers.

odammit2• 8 years ago

Wow.

If deaths are broken out by pedestrian vs passenger, I wonder how people will respond if safety skews heavily towards one of those groups with self driving cars.

frgtpsswrdlame2• 8 years ago

Distance is the metric used so far but also isn't appropriate for comparing self-driving cars to human driven cars. The human driven bucket contains all miles driven, highway cruise control, light snow, heavy rain, tricky merges, etc. The self-driving metric is only the easiest possible miles. Overtime those miles will expand and harder scenarios will be incorporated but to really know if self-driving cars are safer we need apples to apples comparisons which is going to require matching humans vs robots on miles driven and a categorization of those miles, maybe a count of tricky unexpected scenarios as well.

matt4077• 8 years ago

Why am I not surprised that, among all the teams competing in this space, many of which have clocked in far more miles than them, Uber is the first one to kill someone?

This may well just be bad luck. But I cannot shake the feeling that if Uber started an ice cream venture, they would store their molasses on a hill in a Boston. The only way to get “humanity” associated with Uber would involve an Uber zeppelin.

Or it’s a conspiracy, because nothing is as threatening to Uber than autonomous cars. This is certain to invite more regulatory scrutiny. Just kidding... I think..

Axsuul3• 8 years ago

You mentioned bad luck and that could very well be it. Unless Uber starts having more incidents, it isn't fair to say that Uber's technology is inferior.

jakeogh8• 8 years ago

Calling these pre-programmed multi-MJ wheeled computers "self driving" is foolish and deceptive.

orliesaurus• 8 years ago

Not to be the guy who says "I TOLD YOU SO" but it had to happen at some point, sadly & unfortunately - if this is the first (known) case of A.I. controlled cars killing humans, the number of casualties can only grow from now onwards

JumpCrisscross5• 8 years ago

if this is the first (known) case of A.I. controlled cars killing humans, the number of casualties can only grow from now onwards

This is how counting works.

wongarsu3• 8 years ago

Every year between 30000 and 40000 Americans are killed in car accidents. With more self-driving cars on the road it comes as no surprise that self-driving cars will cause some of them.

w_t_payne2• 8 years ago

To ensure that all automotive software incorporates lessons learned from such fatalities, it would be beneficial to develop a common data set of (mostly synthetic) data replicating accident and 'near miss' scenarios.

As we understand more about the risks associated with autonomous driving, we should expand and enrich this data-set, and to ensure public safety, testing against such a dataset should be part of NHTSA / Euro NCAP testing.

I.e. NHTSA and Euro NCAP should start getting into the business of software testing.

kaennar2• 8 years ago

Dr. Mary Cummings has been working on talking to NHTSA about implementing V and V for autopilots/AI in unmanned vehicles for a few years now. She's also been compiling a dataset exactly like what you are talking about.

I think the idea is to build a "Traincar of Reasonability" to test future autonomous vehicles with.

You might want to check out her research https://hal.pratt.duke.edu/research

w_t_payne2• 8 years ago

Thank you for that link. I will pass it along to my former colleagues who I suspect will be very interested in her work.

pishpash6• 8 years ago

How about not testing it on the unpaid public in incremental patches like this age of software "engineering" has decided it was a good idea to do?

Someone1234• 8 years ago

You ultimately have to at some stage, since any test track is a biased test by its nature.

It is more an issue of how sophisticated these vehicles should be before they're let loose on public roads. At some stage they have to be allowed onto public roads or they'd literally never make it into production.

rjvir4• 8 years ago

Why would Uber agree to such regulations?

They were unwilling to legally obtain a self-driving license in California because they did not want to report "disengagements" (situations in which a human driver has to intervene).

Uber would just set their self-driving cars free and eat whatever fine/punishment comes with it.

toomuchtodo3• 8 years ago

It would be required by law. Violating the law would hold corporate officers criminally responsible.

neaden• 8 years ago

I guess my hopeful answer would be they start getting treated like Enron and someone high up goes to prison until they start to comply.

hjnilsson• 8 years ago

Why would Uber agree to such regulations?

This is a strange question to ask. The regulation is not there to benefit Uber, it is to benefit public good. Very few companies would follow regulation if it was a choice. The setup of such regulation would be for it to be criminal to not comply. And if Uber could not operate in California (or the USA) if they did not comply, it would in their interest to provide the requested information.

LifeLiverTransp3• 8 years ago

Very cynical but - if your self-driving is way behind of your competitors- wouldnt it help to have your lousy car in a accident - so that your competitors get hit with over-regulation and you thus kill a market- on which you cant compete?

IncRnd2• 8 years ago

Why would Uber agree to such regulations?

Uber doesn't get to pick and choose what regulations they wish to follow.

Uber would just set their self-driving cars free and eat whatever fine/punishment comes with it.

That sounds quite negligent and a cause for heightened repercussions if anything happens.

The strange attitude you display is the _reason_ there are regulations.

danso30• 8 years ago

Uber ultimately backed down and applied for a California permit: http://fortune.com/2017/03/08/uber-permit-self-driving-cars-...

s2g2• 8 years ago

The start tossing executives in jail.

It’s the only real solution to corporations misbehaving.

adrianmonk7• 8 years ago

Possibly there's enough business risk that if Uber doesn't, someone else will, and then they will have SDCs but Uber won't, and then Uber will go bankrupt just about instantly.

nealabq2• 8 years ago

There may eventually be standard test suites that can be applied to any of the self-driving systems in simulation. This would give us a basis of comparison for safety, but also for speed and efficiency.

As well as some core set of tests that define minimum competence, these tests could include sensor failure, equipment failure (tire blowout, the gas pedal gets stuck, the brakes stop working) and unexpected environmental changes (ice on the road, a swerving bus).

Manufacturers could even let the public develop and run their own test cases.

peterwwillis2• 8 years ago

Situations no self-driving car can avoid:

  - philosophical dilemma
  - physics-constrained reaction times
  - actions that violate the rules of the system

Solutions:

  - protected car lanes
  - protected bike lanes
  - protected pedestrian lanes

Reasons why these solutions are not put in place:

  - cost

Until humans determine that the cost of human life is higher than the cost of upgrading infrastructure, we should accept human death as a regular part of autonomous driving, just the same as we do for non-autonomous driving. 37k dead people every year in the US due to human drivers.

Top reasons for auto accidents today include inclement weather, reckless driving, speeding, driving under the influence, and distracted driving. In theory, most of those could be solved by autonomous driving. But then the list of reasons for accidents would change to whatever new reasons cause autonomous car accidents, such as damaged sensors, programming errors, equipment failure, road hazards, etc.

Even with autonomous cars, we will still need protected lanes, and we will still never implement them, because we don't really care when people we don't know die.

lainga• 8 years ago

If by "protected passenger lane" you mean putting a Jersey barrier between the sidewalk and the road, no thanks. Maybe I'm a spoiled suburbanite, but I can't help but recall that, scant a century ago, pedestrians could walk along or on the streets wherever they pleased.

peterwwillis2• 8 years ago

Not specifically that one way.

In order to allow transportation to co-exist with pedestrians without collisions, you need some kind of separation between the two. With subways, the protected lane is literally underground, but it does definitely have a protected lane. If you don't go under ground, you can go above ground, like several subways and metros do around the world.

If you don't do either of these, you have to make concessions on the ground level. My personal preference would be tall fences around the roadway, and pedestrian bridges that go over or under the roadway (but both have problems). Another would be to still have the fences, but automate some sliding barricades that would activate when traffic halted, which is somewhat like how train crossings work. We could also implement hybrid methods, like that at Shibuya crossing, for very congested intersections.

johnstew• 8 years ago

Why is Uber allowed on public roads?

jakeogh8• 8 years ago

This has little to nothing to do with the specific company (although is a convenient deflection ATM). The very idea that pre-programmed computers can deal with driving is the fundamental problem. In time, people will realize that, but at the moment the "solution" is to "patch" and make more laws to hide the truth: AI has nothing to do with intelligence.

Ultimately they will blame humans, humans are the bugs in their code.

vokep2• 8 years ago

"Artificial Intelligence has nothing do to with intelligence" welp

I don't even know what to make of that

joefourier• 8 years ago

As far as I know all self-driving car programs use some form of machine learning, so they aren't exactly "pre-programmed" in the sense of good old-fashioned AI based on strict, pre-determined rules.

What exactly do you mean by "deal with driving"? Drive without a single accident, ever? That's obviously impossible in practice. Drive better than humans, who in 2016 killed 37,461 people in the US alone? I don't see how that would be impossible - human drivers have a limited field of view, slow response times (average time to break is ~2.3 seconds), and are frequently distracted, sleepy, drunk, etc.

pcthrowaway• 8 years ago

The people leading the development should demonstrate that they can stop for pedestrians by personally jumping out in front of them on a closed test road. If they're not able to demonstrate this, they shouldn't be putting them on public roads.

Alex39172• 8 years ago

Why? It’s not a problem if Uber kills pedestrians, even in situations where it’s completely avoidable. It’s only (legally) a problem if they’re violating the rules of the road while doing so.

wtetzner2• 8 years ago

It’s only (legally) a problem if they’re violating the rules of the road while doing so.

I doubt that's true, but even if it were, I believe the rules of the road are "always yield to pedestrians, even if you have the right-of-way".

gaius2• 8 years ago

The ancient Romans would have the civil engineer stand under the bridge they'd just built while a Legion marched over it. That's why Roman structures are still around today!

ranie93• 8 years ago

Do you know if this is factual? I've read that it's only a myth.

golergka• 8 years ago

Would they do it enough times for such observation to have any statistical significance?

jaclaz2• 8 years ago

The ancient Romans would have the civil engineer stand under the bridge they'd just built while a Legion marched over it. That's why Roman structures are still around today!

JFYI, it is most probably apocryphal: https://skeptics.stackexchange.com/questions/18558/were-roma...

still it's a nice one, I have heard the same about engineers/architects in ancient Babylon and Egypt.

flexie• 8 years ago

Heard a similar myth about a Danish king. We was tired of cannons blowing up so he ordered the manufacturer to sit on top of the cannon when it was fired the first time.

astrodust2• 8 years ago

A similar story comes from World War II where an alarmingly high number of parachutes were failing to open when deployed. They started picking random chutes and their packer and sent them up for a test drop. The malfunction rate dropped to near zero.

pferde• 8 years ago

Something similar, but self-imposed happened when they moved a whole building in Guadalajara: https://en.wikipedia.org/wiki/Jorge_Matute_Remus#Movement_of...

"In order to gain the trust of the employees he asked his wife to enter the building while the movement was taking place."

NewEntryHN• 8 years ago

The comparison is odd to me. Somehow building bridges seems more of an exact science to me than making cars drive themselves. I sure wouldn't step on a bridge if its engineer doesn't dare going under it. Shit is supposed to stand up.

aiCeivi9• 8 years ago

Isn't that just survival bias? The good ones didn't collapse?

joering2• 8 years ago

Said that few times: how does any AI recognize whether object in front of me is just a white plastic bag rolling in the street, or a baby that rolled out of crib. AI cannot know and will never know. And we cannot have cars drive smoth in traffic if every self driven car will stop before hitting an empty plastic bag.

darawk• 8 years ago

What? AI can know in the exact same way you can know. What are you talking about?

munk-a• 8 years ago

I'm surprisingly okay with self-driving cars stopping for plastic bags in the near term.

Self driving tech is up against the marketing arm of the auto-industry, they've got to be better than perfect to avoid public backlash. If they're a bit slower but far safer then I think they'll do well.

trhway2• 8 years ago

actually i slow down for plastic bags. And not because i care for their safety.

bonesss2• 8 years ago

How do we recognize whether an object in front of a car is just a plastic bag in the wind or a baby?

o At speed we're pretty ok with cars hitting things people drop on the road, examples of cars hitting wagons and babies are already plentiful

o Visual recognition & rapidly updated multi-mode sensor data, backed by hard-core neural networks and multi-'brained' ML solutions, have every reason to be way better at this job than we are given sufficient time... those models will be working with aggregate crash data of every accident real and simulated experienced by their systems and highly accurate mathematical models custom made to chose the least-poor solution to dilemmas human drivers fail routinely

o AIs have multiple vectors of possible improvement over human drivers in stressed situations. Assuming they will bluntly stop ignores how vital relative motion is to safe traffic, not to mention the car-to-car communication options available given a significant number of automated cars on the road -- arguably human drivers can never be as smooth as a fleet of automated cars, and "slamming the brakes" is how people handle uncertainty already

Presently the tech is immature, but the potential for automated transport systems lies well beyond the realms of human capabilities.

Splines2• 8 years ago

Reminds me of this: https://www.youtube.com/watch?v=Sr1ym96abkg

Johnny5554• 8 years ago

Self driving cars are still subject to the laws of physics... unless you're going to dictate that self-driving cars never go above 15mph, I wouldn't advocate jumping in front of even a "perfect" self-driving car.

Braking distance (without including any decision time) for a 15mph car is 11 ft, for a 30mph is 45 ft. Self driving cars won't change these limits. (well, they may be a little better than humans at maximizing braking power through threshold braking on all 4 wheels, but it won't be dramatically different)

So even with perfect reaction times, it will still be possible for a self-driving car to hit a human who enters its path unexpectedly.

pishpash6• 8 years ago

Then we should at least learn their capabilities by throwing jumping dummies at them. Call it the new dummy testing. It's the least we can do. Did Travis bro do this when he set the plan in motion?

bobthepanda6• 8 years ago

Indeed. This is why many cities are reducing speed limits.

In fact, self-driving cars may actually improve the situation if cars actually start complying with speed limits en masse.

astrodust2• 8 years ago

Unlike humans who have limited vision, self-driving cars are generally able to observe all obstacles in all directions and compute, in real-time, the probability of a collision.

If a car can't observe any potential hazards that might impact it using different threat models it should drive more slowly. Blowing down a narrow street with parked cars on both sides at precisely the speed limit is not a good plan.

ubernostrum3• 8 years ago

Once upon a time when I was learning to drive, one of the exercises my instructor used was to put me in the passenger seat while he drove, and have me try to point out every person or vehicle capable of entering the lane he was driving in, as soon as I became aware of them. Every parked vehicle along the side of a road. Every vehicle approaching or waiting to enter an intersection. Every pedestrian standing by a crosswalk or even walking along the sidewalk adjacent to the traffic lane. Every bicycle. Every vehicle traveling the opposite direction on streets without a hard median. And every time I missed one, he would point and say "what about that car over there?" or "what about that person on the sidewalk?" He made me do this until I didn't miss any.

And then he started me on watching for suspicious gaps in the parked cards along the side that could indicate a loading bay or a driveway or an alley or a hidden intersection. And so on though multiple categories of collision hazards, and then verbally indicating them to him while I was driving.

And the reason for that exercise was to drive home the point that if there's a vehicle or a person that could get into my lane, it's my job as a defensive driver to be aware of that and be ready to react. Which includes making sure I could stop or avoid in time if I needed to.

I don't know how driving is taught now, but I would hope a self-driving system could at the very least match what my human driving instructor was capable of.

francisofascii• 8 years ago

The speed limit for this particular stretch was 45 mph. (sounds high to me, considering there is a bike lane)

rout39574• 8 years ago

So, there's a performance envelope expected. Sure, someone can bungee jump off an overpass and not be avoidable. :)

But they should be willing to walk in front of it in an in-spec performance regime. There's some really good Volvo commercials along that line, with engineers standing in front of a truck.

Spare_account2• 8 years ago

Braking distance ... for a 30mph is 45 ft

Apologies for going off topic here, but I'm curious about this. I've tested every car I've ever owned and all of the recent cars with all-round disc brakes have outperformed this statistic, but I've never been able to get agreement from other people (unless I demonstrate it to them in person).

I'm talking about optimal conditions here, wet roads would change things obviously but each of these cars was able to stop within it's own car length (around 15 feet) from 30mph, simply by stamping on the brake pedal with maximum force, triggering the ABS until the car stops:

2001 Nissan Primera SE

2003 BMW 325i Touring (E46)

2007 Peugeot 307 1.6 S

2011 Ford S-Max

I can't work out how any modern car, even in the wet, could need 45 feet to stop. In case it's not obvious, this is only considering mechanical stopping distance, human reaction time (or indeed computer reaction time which is the main topic here) would extend this distance, but the usual 45 feet from 30mph statistic doesn't include reaction time either.

dlwdlw• 8 years ago

If self driving cars are limited to speeds that allow them to stop within their lidar max range is that too slow? Humans don't have the pinpoint accuracy of lidar but our visual algorithms are very flexible and robust and also have very strong confidence signals e.g driving more carefully in dark rain.

Cameras are not accurate enough though, their dynamic range being terrible. Wonder how humans would fare if forced to wear goggles that approximated a lidar sensors information.

PantaloonFlames• 8 years ago

The advantage self-drivers have is in 1. minimizing distraction / optimizing perception 2. minimizing reaction time.

Theoretically self-drivers will always see everything that is relevant, unlike a human driver. And theoretically a robot-driver will always react more quickly than even a hyper-attentive human driver, who has to move meat in order to apply the brake.

emodendroket14• 8 years ago

But is that the actual situation we're talking about? Or are we actually talking about a situation where the person may have been jaywalking but would have had a reasonable expectation that a human driver would stop? I walk a decent distance to work every day and I don't think anyone totally adheres to the lights and crosswalks (not least because if you do you will be running into the road just as everyone races to make a right turn into the same crosswalk you're in).

gkya2• 8 years ago

What about steering away, a la ABS?

weinzierl2• 8 years ago

My opinion is that we will come to a point where self-driving cars are demonstrably, but only marginally, safer road users than humans.

From an ethical standpoint the interesting phase will only start then. It‘s one thing to bring a small fleet of high tech (e.g. having LIDAR) vehicles to the road. It‘s another to bring that technology to a saturated mass market which is primarily cost driven. Yes, I assume self-driving cars will eventually compete with human driven ones.

Will we, as a society, accept some increase in traffic fatalities in return for considerable savings that self-driving cars will bring?

Will you or me as an individual accept a slightly higher risk in exchange for tangible time savings?

pishpash6• 8 years ago

We do all the time, and the answer to your question is, yes if marketed ingeniously. Humans are stupid.

smileysteve4• 8 years ago

demonstrably, but only marginally, safer road users than humans.

I believe the claim is 38 multitudes better than humans, significantly better than marginally.

accept some increase in traffic fatalities

No. And the question is more about "some" than "some increase"

accept a slightly higher risk in exchange for tangible time savings?

Texting while driving and even hands free talking were becoming laws in many states before smart phones -- and my experience is that many people readily accept this risk and the legal risk just to communicate faster. The same can be said for the risk of drunk driving -- it's a risk that thousands of Americans take all of the time.

rdlecler1• 8 years ago

There wouldn’t be many human drivers on the road if they had to pass this test as well.

TheGRS• 8 years ago

IIRC the people who programmed the first auto pilot for a major airliner were required to be on board the first test flight, so I have to think their testing methodology was pretty meticulous.

Moter8• 8 years ago

Auto Pilot is not a "Push a button and it goes from Airport A to Airport B". It only helps in a few cases.

tomswartz072• 8 years ago

The people leading the development of these horse-drawn carriages should demonstrate that they can stop for pedestrians by jumping in front of them on a closed test dirt-path. If they're not able to demonstrate this, they shouldn't be putting them on public carriageways

Sounds silly when compared against old tech.

Accidents happen, best we can do is try to prevent them.

GeorgeTirebiter• 8 years ago

Imagine if we fully separated car traffic from foot traffic. Would this have happened then?

larrykwg• 8 years ago

very good point, I would not be surprised if Uber put those cars out with a barely working model to collect enough training data, the human operator to correct such errors was tired and didn't intervene. Some basic driving test for self driving cars or other mitigating factors need to be added IMMEDIATELY otherwise tons more will probably die or be injured. Train operators need to prove they are awake and with attention using some button presses, similar things need to be required for those research vehicle if you want to allow them at all.

rsync• 8 years ago

"The people leading the development should demonstrate that they can stop for pedestrians by personally jumping out in front of them on a closed test road. If they're not able to demonstrate this, they shouldn't be putting them on public roads."

Although the actual logistics of your proposal might be challenging (child comments point out that some speeds/distances might be impossible to solve) your instinct is a correct one: the people designing and deploying these solutions need to have skin in the game.

I don't think truly autonomous cars are possible to deploy safely with our current level of technology but if I did ... I would want to see their family and their children driving in, and walking around, these cars before we consider wide adoption.

basura0454785• 8 years ago

goes to show, sdv's have to be perfect in the eyes of the public. you wouldn't seriously recommend adding a test like that to a regular driving licence.

also that testing does happen in the case of every av program i know of. closed obstacle courses with crap that pops out forcing the cars to react. look up gomentum station.

† i did it for you. http://gomentumstation.net/

danso30• 8 years ago

This isn't a good argument because it implies if these AVs have successfully not killed their CEOs during a closed (i.e. controlled) test, that they are safe on public roads. But it seems like the majority of AV accidents so far involve unpredictable and uncontrolled conditions.

IOW, setting this up as some kind of quality standard gives unjustified cover ("Hey, our own CEO risked his life to prove the car was safe!") if AVs fail on the open road, because the requirements of open and closed tests are so different.

DennisAleynikov• 8 years ago

human driven cars are currently, as we speak running people over on the streets and they have human drivers who don't particularly want to run over other humans. It was inevitable that this happened, and no matter how many people self driving cars run over will be worth it, since it will still be less than what cars are currently doing in terms of death toll.

supertrope4• 8 years ago

Bad idea. Even Uber corporate VPs shouldn't be exposed to unnecessary injury risk.

https://www.youtube.com/watch?v=_47utWAoupo

artursapek• 8 years ago

This reminds me of a story I heard in college... an owner of a company that builds table saws demonstrated his saw's safety feature - a killswitch that pulls the blade down into the machine as soon as it detects flesh - by putting his hand on it while it was spinning.

astura3• 8 years ago

Video: https://youtu.be/eiYoBbEZwlk

megy8• 8 years ago

This was a cyclist. So they should be on bikes.

Just like they take the repair mechanics on the first test flight after a major repair of a big airplane.

Avamander• 8 years ago

My sympathies to the victim, but this post with the XKCD substitution plugin made my sides leave orbit.

dang20• 8 years ago

Please don't do this here.

ocdtrekkie5• 8 years ago

The reason self-driving car companies have flocked to Arizona is that Arizona chose to pretty much completely deregulate them: They don't even have to report statistics to the state like they do in California.

It's unfortunate, but unsurprising, the first pedestrian killed by an experimental self-driving car allowed on public roads was in Arizona.

jonathanyc19• 8 years ago

What will the consequences of this person’s killing be? Will someone lose out on a promotion, or miss their performance bonus?

We need to discuss how the developers self-driving cars will be held accountable for the crimes they commit. There is no reason the person who programs or the person who makes money from a self-driving car should be held less accountable for a crime that if committed directly by a person would almost certainly result in jail time. You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.

cliffordkeeney• 8 years ago

This is crazy. The developers likely have no say in where and when the cars go out on public roads. That's obviously a decision for someone higher up in the company.

The executives should be held accountable, not the developers.

jessriedel10• 8 years ago

We already have plenty of case law and policy for cases where people are killed by mechanical equipment operated by businesses, and the type of penalties and compensation are appropriately different if the cause was negligence, malice, or impossible-to-eliminate fluke events. (Generally in the latter case, compensation is due but there are no criminal charges.) Obviously there will be policy adjustments and clarifications for the case of self-driving cars, but I don't think there's reason to think we can't apply normal and existing legal principles here.

ApolloFortyNine6• 8 years ago

If you hold developers responsible, you can kiss self driving cars goodbye.

What should be passed (but I can't see how) is a percentage of allowed deaths, at least in the early years, and set it to something like 5-10% of the current rate, reducing downwards to 1% after 20 years.

People will die from self driving cars, and undoubtably their will eventually be a case that is 100% the self driving car's fault. The benefit of self driving cars comes from the mistake being permanently fixed, while with human drivers it can be committed over and over again.

There needs to be some kind of protection on the companies (and obviously the developers, I've never heard someone try to say they should be held responsible before) from lawsuits. Otherwise all it'll take a is a small handful beefore companies will just let it die.

k3lsi3r• 8 years ago

I'm guessing people are downvoting because of the implication that the software team should be held responsible?

Something does feel wrong about punishing them when the decision to put the car on the road in the first place was almost certainly not their own.

Though I agree Uber should be held accountable for it and it shouldn't be a token fine since the whole point of punishing an accident like this is to discourage them from occurring in the first place.

This sort of accident orchestrated by a group of people probably won't be gracefully handled by our legal system.

chimeracoder8• 8 years ago

You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.

Look, I'm all for developers and (software companies in general) to be considering the ethical implications of the work they do, and the moral obligations that they take on as a result of it. However:

We need to discuss how the developers self-driving cars will be held accountable for the crimes they commit. There is no reason the person who programs or the person who makes money from a self-driving car should be held less accountable for a crime that if committed directly by a person would almost certainly result in jail time. You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.

This is a bad mentality to take with postmortems for software failures in general, at least from the outset. You need to look at the underlying factors that contributed to the issue, not simply looking for a person to assign blame to. It's possible that negligence is the underlying cause, but not necessarily - and even if negligence is a cause, what were the other cultural factors that led to the negligence happening, without being caught somewhere else in the pipeline? It's tempting to look to assign blame, but if you do that, you'll actually miss out on the systemic improvements that would be necessary to prevent similar incidents in the future.

But moreover, this is a bad outlook to take here, because this wouldn't be criminal behavior if committed by a human. From the best we can tell, given the details available so far, it's an accident, and it's very rare for criminal charges to even be considered in accidents like these, unless it's a hit-and-run.

wehadfun• 8 years ago

The developers have no control over the sensors, tires, weight of car, testing budget, human backup operator, or a million other things that went into this happening. Hell developers probably told who ever they could to not release this. Management at Uber and who ever approved this thing should be held accountable.

Consultant324523• 8 years ago

The realistic alternative isn't California-esque regulations, it's a country with poorer (more disposable) people. Arizona is a nice compromise, at least we can see what's happening.

lucb1e2• 8 years ago

You've got to be kidding or sarcastic about the poorer=disposable part, right?

InclinedPlane2• 8 years ago

This was inevitable. Some of the developers for self-driving vehicles are immensely careful, some are not. And the regulatory oversight is currently not sufficient to allow the one to operate but not the other.

lostlogin• 8 years ago

I wonder how companies handle these situations? Most places have clear requirements eg stop, check if anyone is injured, contact police, remove debris from road if safe to do so. These things aren’t so easy without a driver.

3pt141592• 8 years ago

Look, I'm all for taking self-driving cars seriously. I've written about how they're effectively WMDs if the software update computer gets owned. I've also frequently commented about the unethical practices that Uber has taken in the past.

That being said, let's not over-react. This is a single death. These are going to happen. Right now self-driving cars are essentially teenagers learning how to drive. En mass there is going to be a death here or there, but, also en mass, we're going to have a permanently safer road once they've learned, if we can secure them from cyber attack.

Try to avoid all-or-nothing thinking here. Definitely advocate for regulations, and maybe Uber wasn't safe enough, but people are either going to die from self-driving cars or they're going to die from self-driving cars taking too long because of public outrage.

gamblor956• 8 years ago

This didn't need to happen. It's indefensible that it did.

And there's still no proof that self-driving cars are safer than humans when confronted with novel situations, i.e., the situations most likely to result in injuries and fatalities. If anything, the evidence thus far is that self-driving cars are more dangerous than ones driving by humans.

Axsuul3• 8 years ago

How are we to judge that self-driving cars are more dangerous from simply one death?

The proper comparison to be made should be between miles driven from autonomous vehicles vs. those of humans and then looking at the incident rates.

codecamper• 8 years ago

Interesting nobody used the words jail or murder yet. Not even manslaughter.

Just because there was no driver does not keep it from being a crime. Do the engineers go to jail or do the managers or the investors?

stordoff• 8 years ago

In a typical crash, you don't jump straight to "this was murder/manslaughter"/"who is going to jail". At a bare minimum, you have to establish fault. I don't see why this case would be any different.

WHoWHo2• 8 years ago

What happens when a self driving car in the future from a service is driving empty back to some parking? And hits someone? Does it call ambulance or police?

WHoWHo2• 8 years ago

... this would make from some creepy film scene - someone shouting for help and the car only watching.

andy_boot• 8 years ago

several conventional luxury cars already have this feature - they detect crashes and call emergency services.

rjromero• 8 years ago

I’m pretty sure if Arizona’s lawmakers understood the underlying nature of machine learning and autonomous driving, they wouldn’t allow self driving cars to operate.

I don’t care how good your models and data are, until you’re able to write an algorithm that can fully handle and learn from a situation it was completely unprepared for, it will always encounter edge cases like this.

This is what happens when we have a society of people saying AI is already here, people like Elon Musk saying AI is a “significant threat” despite "AI" being essentially a black box statistical model in it's current form. From what I’ve seen, the general public thinks we’re 20+ years of AI than we really are.

You need general AI before AI can drive a car, full stop. Otherwise, you need to isolate roads from other human drivers pedestrians, and cyclists. It just won’t be reasonably safe until then.

ggggtez4• 8 years ago

You need general AI before AI can drive a car

1.3 million people died in the USA from car crashes in 2017. An additional 20-50 million are injured. You don't need to be perfect, you just need to be better than the baseline.

That said, and to be fair to your feelings, Uber certainly is not better. This is a fact I'm sure lawmakers in Arizona are discussing right now.

woliveirajr• 8 years ago

This comes just few days after this: "When an AI finally kills someone, who will be responsible?" url: https://news.ycombinator.com/item?id=16584862

justboxing• 8 years ago

Have to quote Prof. Teddy here.

Context: How new technology is introduced on public streets and we have no choice or say on whether we want it or not, even if it endangers us and gets us killed.

When motor vehicles were introduced they appeared to increase man’s freedom. They took no freedom away from the walking man, no one had to have an automobile if he didn’t want one, and anyone who did choose to buy an automobile could travel much faster and farther than a walking man. But the introduction of motorized transport soon changed society in such a way as to restrict greatly man’s freedom of locomotion. When automobiles became numerous, it became necessary to regulate their use extensively. In a car, especially in densely populated areas, one cannot just go where one likes at one’s own pace one’s movement is governed by the flow of traffic and by various traffic laws.

and

Even the walker’s freedom is now greatly restricted. In the city he continually has to stop to wait for traffic lights that are designed mainly to serve auto traffic. In the country, motor traffic makes it dangerous and unpleasant to walk along the highway. (Note this important point that we have just illustrated with the case of motorized transport: When a new item of technology is introduced as an option that an individual can accept or not as he chooses, it does not necessarily REMAIN optional. In many cases the new technology changes society in such a way that people eventually find themselves FORCED to use it.)

Source: The Mainfesto: INDUSTRIAL SOCIETY AND ITS FUTURE http://www.washingtonpost.com/wp-srv/national/longterm/unabo...

RyanZAG• 8 years ago

Great points and extremely important.

But at the same time, I don't personally want to go back to an age before automobiles became commonplace. They're too useful to me.

Is that selfish? Sure. But our lives are all shaped by our wants, not by our needs.

bobthepanda6• 8 years ago

I don't think that anyone is asking for a return to nature, but America has shown that adopting the automobile as the primary mode of transportation and adapting all of society wholesale for it has serious side effects. Rather than promote it as (essentially) the only option, automobiles should be a tool in a toolkit of transportation methods.

harrumph• 8 years ago

But at the same time, I don't personally want to go back to an age before automobiles became commonplace.

Neither would I, but I think the cautionary value in the discussion of the resulting inadvertent restriction of pedestrian freedom is not to visualize an auto-free world as an alternative but instead to visualize a new set of customs of urban development that aggressively protect pedestrian freedom. In the real estate world, the walkability movement is concerned with this and should achieve greater influence.

dwaltrip• 8 years ago

This is a great point. Note, though, this is not new.

It's been been happening since the invention of the first tool. Stone weaponry, agriculture, and so forth, all became largely non-optional after their introduction.

abakker• 8 years ago

Obligatory Mitchell and Webb skit: https://www.youtube.com/watch?v=-EGAtLGDU7M

bobthepanda6• 8 years ago

The major difference, though, is that most major technologies are not also accompanied by legislation that effectively penalizes the old methods. There's nothing legally penalizing me from using a bow and arrow, or buying hand-woven clothes, etc., whereas walking is effectively criminal in some parts of the US: https://www.cnn.com/2014/07/31/living/florida-mom-arrested-s...

dsacco2• 8 years ago

When a new item of technology is introduced as an option that an individual can accept or not as he chooses, it does not necessarily REMAIN optional.

This is offtopic, but I am reminded of how Slack initially supported IRC gateways and recently discontinued them.

It’s not a percect likeness to this quote’s point. But I think it’s interesting this pattern of initially optional technology, then de facto standardized technology emerges ubiquitously.

I suppose it’s arguable that embrace, extend and extinguish is the purposeful, deliberate manifestation of this pattern, arbitrated by a single entity (i.e. a company with a new technology).

Edit: Jeez an immediate downvote? The comment was barely posted a minute ago :)

markatkinson• 8 years ago

Just finished the new Unabomber series on Netflix. This is similar to his manifesto which I ended up sympathising with towards the end.

opportune• 8 years ago

It's actually a direct quote from his manifesto

qpooqpoo2• 8 years ago

It's so refreshing to actually read a truly free thinking and intelligent discussion on the problems of technology and its impacts on society.

Anyway, I just finished reading Kaczynski's first book, "Technological Slavery" (Feral House, 2010) It's an amazing work and it elaborates on this point quite a bit. Recommended.

ExcelSaga• 8 years ago

An “amazing work,” just to be very very clear, by Ted Kaczynski, the serial bomber. Not that he has nothing of value to say, but keep the source well in mind. He’s a smart guy, with a lot of insight, and he’s also batshit crazy and a remorseless killer.

df5t0rw2• 8 years ago

[unconstructive but still..] So the day I could be emprisoned because of an error of coding... Like a segfault on an highway... as finally came :D Is the disclamer "no waranty of anykind" shield us from that kind of thing ? ;) (I code on some mainstream C libraries)

Jack000• 8 years ago

you already can? our lives depend on all kinds of software every day. In Canada at least you'd need a P.Eng to work on stuff that directly endanger lives.

https://en.wikipedia.org/wiki/Therac-25

alacombe5• 8 years ago

If I'm not mistaken, P.Eng are merely needed the rubber stamping, not to do the grunt work. I doubt they'll go through all the specs, re-run all the simulations, and verify all the parameters.

Also, where is your P.Eng be any useful to justify the learning coefficients of your AI, or to protect about the cosmic ray bit-flip in the hardware your proven correct software is running on ?

gowld17• 8 years ago

Engineering for safety is about testing, not manufacturing quality. Or course you need to build quality in order to pass tests, but perfection is not a goal of any component. Tests measure quality, and failsafes and control layers reduce failure rates.

df5t0rw2• 8 years ago

Thanks for the Therac-25 reference :)

loup-vaillant• 8 years ago

If you don't offer warranties, your user will have to review your library so they can offer warranties.

Someone has to sign that stuff as "production worthy or I go to prison".

smallgovt3• 8 years ago

Not to dismiss the tragedy of this incident, but it should be expected that self-driving cars kill pedestrians -- just at a rate lower than what's expected from human drivers.

Perhaps there's a better metric to look at, but I'd like to see number of deaths caused per miles driven.

If Uber's self-driving cars are killing more pedestrians than a human driver would, we have a huge problem, but I'd be willing to bet they're at least an order of magnitude safer in this respect.

macintux3• 8 years ago

Fewer people will die (probably). The problem is that different people will die, and you can't celebrate with the families of the people who were slated to die and didn't.

falcolas9• 8 years ago

And different people will "kill". It won't be the drunk drivers, or the road ragers, or the people texting on phones, it will be anyone unfortunate enough to be in the car at the time. A businessman. A politician. A cop. A teacher.

And to say that "it's not their fault" and "it could have happened to anyone" will be cold comfort to those who just watched their car mow down an innocent, and there was nothing they could do to stop it.

paxys• 8 years ago

It's not that simple. Who is going to take responsibility for such incidents? The company that owns the car? The person behind the wheel (if any)? Or the engineer that designed the algorithm?

ghaff3• 8 years ago

I would think it will be pretty standard product liability. With some assumptions (proper maintenance, software updates, etc., used in accordance with directions), it's hard to see liability for deaths and injuries not resting with the manufacturer. What's unusual for a consumer product is that, with the somewhat exception of pharmaceuticals, there aren't a lot of examples of consumer products that routinely kill people even when properly used.

nafizh2• 8 years ago

There is no data available at the moment to draw conclusions that self-driving cars bring less fatalities, and rightly so, because they have not been tested on a beta stage. You are only guessing and hoping.

smallgovt3• 8 years ago

There is certainly at least SOME data. At the very least, let's find out how many miles Uber's self-driving program has driven. Then, we can, at least, compare deaths per miles driven to a human driver.

ApolloFortyNine6• 8 years ago

It'll likely never get passed, but we truly need an "acceptable death" metric to be passed to protect the companies (and apparently people too, since some in these comments what the developers held responsible) from deaths. In my opinion, a law should be passed that per mile driven, 5-10% of the current deaths per mile is allowed, and it should begin decreasing after 10 or so years to something low (maybe 1%, maybe .01%, we'd have to wait and see).

People will always make mistakes, the benefit with self driving cars however is that mistake should only be made once. Whatever bug caused it to be considered at fault can be patched and will never occur in that exact circumstance again. Meanwhile with humans you could have that same case occur over and over again.

With how much money people can win in lawsuits nowadays, all it would take is a handful of cases to totally destroy a manufacturer of self driving cars.

Erlangolem7• 8 years ago

That sounds like the market correcting itself, while you want to protect the likes of Uber from market forces, with new laws. Putting that aside though, good luck trying to get re-elected after passing a law like that. Few people are technocratic and emotionally detached enough to agree with you, from the looks of it, not even on this site.

tachyonbeam• 8 years ago

In general, I like having a diverse set of options available in terms of software implementations (eg: multiple C++ compilers to choose from). It does worry me a little bit, however, that so many companies are trying to implement their own self-driving cars from scratch. If you just look at how buggy new OS releases tend to be, how many blatantly obvious bugs you run into every month, it's clear that many software development teams don't properly test their stuff. Some companies, like uber here, will act irresponsibly and fuck up, resulting in the loss of human lives.

I would guess it's inevitable that in the medium/long-term, there will only be one or two companies developing and selling self-driving tech to every car manufacturer. Small players won't have enough data, everyone will be too afraid of the risk, the amount of government regulations surrounding self-driving cars will increase. More regulations will protect people but it will also mean it's much harder to start gathering the data you need and testing your product.

apeace5• 8 years ago

You would lose that bet. As another comment pointed out[1]:

The NHTSA reports a fatality rate of 1.25 deaths per 100 million miles[2], twenty five times the [4 million miles] Uber has driven.

So they should have driven around 75 million more miles before getting their first fatality, in order to remain even with humans. Not to mention they've been driving on the clearest/sunniest roads in only a few cities.

Of course a sample size of one is not enough data, but I'd say we should err on the side of "we have a huge problem".

There should have been so many safety precautions in place that nobody should have died from this yet.

[1] https://news.ycombinator.com/item?id=16620736

JustSomeNobody• 8 years ago

I'm pretty sure you don't want to compare right now. Self-driving cars are just too new.

I think the rate of overall vehicle crash deaths in the U.S. is less than 2 per 100 Million vehicle miles. Pedestrian traffic is a fraction of that.

tedsanders• 8 years ago

Human drivers kill humans at a rate of 1 per 100 million miles. Collisions occur at a rate of like 1 per 200k miles.

AI drivers have killed 1 human having driven a mere ~5 million miles (not even counting Tesla autopilot). Disengagements occur at a rate of 1 per 5k miles.

The evidence strongly, strongly points to self-driving cars being more dangerous than humans today.

vgf• 8 years ago

I bet people at Volvo are feeling a bit anxious at the moment for that deal:

https://www.media.volvocars.com/global/en-us/media/pressrele...

remind_me_again• 8 years ago

What I don't get is that if this is a test car, how come the the driver did not take over the helm immediately. The driver in that car should be in full alert as if driving the vehicle himself and jump in as needed. Test should not mean to let loose an autonomous car.

ggggtez4• 8 years ago

Unlike all the other comments discussing charges, this one actually has a good point. Presumably the safety driver is there specifically to prevent this from happening. They'd still have to show that the driver was personally at fault. That would mean showing that the driver was e.g. sleeping/not paying enough attention, or something similar.

At the very least, it might call into question whether safety drivers are actually providing any real safety. Localities may start to crack down on using that kind of excuse for self-driving tests.

phyzome5• 8 years ago

Do you think you could maintain the required state of alertness, focus, and engagement necessary to hit the brakes and turn the wheel after 30 minutes (or whatever) of just sitting passively as the car drove you around?

I don't think the "safety drivers" do much other than provide liability-shifting for the company.

gowld17• 8 years ago

the drivers are employed by the company, doing company work. They may (misleadingly) shift the optics away from the automation, but they don't shift the liability.

gowld17• 8 years ago

A test pseudo-"driver" ought to be doing stuff like orally making notes of the environment and car operation, collecting data the sensors might be missing, and monitoring the car's automated judgement. As a related effect, the human should be ready to emergency brake or swerve.

ape42• 8 years ago

It would be impossible to stay alert as a safety driver.

fabian2k• 8 years ago

Unless this was a situation that was impossible to anticipate and avoid, this looks bad. It would mean that both, the autonomous driving system and the safety driver failed.

Not hitting pedestrians should be close to the top in priority, so while failures of the software are certainly expected during this kind of test, this is one of the worst kinds of failures that should have received a lot of attention before ever putting the vehicles on a public road.

The safety driver failing makes me wonder how well qualified they are, or if there is an issue with staying alert for long times without actually driving the car.

grecy• 8 years ago

To play Devil's advocate, there were over 100 people killed in the US TODAY from non-autonomous vehicles [1].

Obviously we'd need to know the number of miles driven in each category to get a meaningful comparison, but let's all keep in mind a USA where only one person per day is killed on the roads would be a two order of magnitude improvement over today.

[1] https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...

phyzome5• 8 years ago

If you look elsewhere in the comments, you'll see some numbers. Autonomous vehicles are new enough that they come out as more dangerous even with one death.

mdev• 8 years ago

I wonder how software updates and are validated and tested before going to production in autonomous cars. This is scary - imagine being a developer responsible for a similar malfunction. I'm not suggesting that was the case here - just think this might need as much validation as medical equipment (hoping that it won't stifle innovation).

amelius4• 8 years ago

Car was autonomous with driver behind wheel

Let me make a prediction. Uber will claim that the driver was in control of the vehicle at the moment of the accident. And the "driver" will receive a bonus payment in the coming months.

swyx2• 8 years ago

I'm saddened by this incident but have thought a lot about this eventuality. there are a few levels of societal acceptance of self driving car death outcomes i can think of:

A - Human equivalent: self driving car obeys reasonable rules that a human also would eg minimum speed on a highway and kills someone

B - Trolley problem: (very artificial scenario) self driving car has to choose between killing N or M people where N > M and wrongly (in hindsight) chooses N

C - Car fault: self driving car kills someone in a situation where no human would have

As a society we would probably accept A easily but B starts to get shaky. C currently looks completely unacceptable, BUT I would argue that society has to get to a point where even C is ok conditional on the probable result that overall car deaths decline dramatically.

In other words we will have to get to a point where individual deaths are extremely regrettable but the overall death reduction of adopting self driving cars are so undeniable that the individual deaths can be discussed without also talking about banning SDCs in the same breath.

rimliu2• 8 years ago

How does it work out? I mean you got a situation wher SDC kills someone where no human driver would, but somehow it gets safer?

swyx2• 8 years ago

yes, because in aggregate the deaths are still reduced. we need society to get past individual death stories (which are very sad and we should do everything we can to prevent it) and not let them distract from the overall need for SDCs

RcouF1uZ4gsC• 8 years ago

Self driving cars are one of the things where we absolutely cannot afford to do "move fast and break things".

If there was any whiff of shortcutting, poeple need to go to jail and Uber should be bankrupted by penalties and lawsuits.

mtgx3• 8 years ago

The last time I heard this, the first thing that came to my mind was "I sure hope Facebook isn't going into self-driving cars."

And then I learned Facebook is going into self-driving cars.

Sigh.

cdolan• 8 years ago

Devil's Advocate argument here, but have you read books on our space race in the 60s? Should NASA have been bankrupted by penalties and lawsuits when Apollo 1 had its accident?

azdle• 8 years ago

If they were dropping rockets on people standing on street corners. Yes, 100%. Your example is different because it was deaths of people involved in the projects who knew there were risks and agreed to them.

Jagat• 8 years ago

Can we please not overreact and send the whole self-driving research down the drain?

What does the statistics say? How many miles have the self-driving cars driven, and how many deaths were they responsible for? How does it compare to a human driver?

When a million humans drive a car for a mile, and 1 of them results in a death, it's easy to pinpoint the blame on a "random drunk/distracted driver". How'bout start thinking of the self-driving software as a combination of all human drivers, with just a much smaller odds of having a drunk/sun-blinded/distracted driver than an average human driver.

objclxt• 8 years ago

What does the statistics say? How many miles have the self-driving cars driven, and how many deaths were they responsible for? How does it compare to a human driver?

As of December 2017 Uber had driven 2 million autonomous miles[1]. Let's be generous and double that, so 4 million.

The NHTSA reports a fatality rate (including pedestrians, cyclists, and drivers) of 1.25 deaths per 100 million miles[2], twenty five times the distance Uber has driven.

You probably shouldn't extrapolate or infer anything between those two statistics, they're pretty meaningless because we don't have nearly enough data on self driving cars. But since you asked the question, that's the benchmark: 1.25 deaths per 100 million miles.

[1] https://www.forbes.com/sites/bizcarson/2017/12/22/ubers-self... [2]: https://en.wikipedia.org/wiki/Transportation_safety_in_the_U...

rjvir4• 8 years ago

Scaling those numbers paints a poor picture for Uber. Assuming 3 million total miles autonomously driven thus far from Uber's program:

- Uber autonomous: 33.3 deaths per 100 million miles

- Waymo: 0 deaths per 100 million miles

- National average: 1.25 deaths per 100 million miles

Of course, the Uber and Waymo numbers are from a small sample size.

But there's also the bayesian prior that Uber has been grossly negligent and reckless in other aspects of their business, in addition to reports that their self-driving cars have had tons of other blatant issues like running red lights.

It seems reasonably possible that an Uber self-driving car is about as safe as a drunk driver. DUIs send people to jail - what's the punishment for Uber?

gpm• 8 years ago

We absolutely should extrapolate something from those statistics.

Let's assume that the chance of killing in any two intervals of of the same number of miles traveled is the same. Let's say that the threshold for self driving cars being "good enough" is the same death rate as human drivers.

If we assume Uber is good enough, then they should kill people at a rate of at most 1.25/100,000,000. The waiting time until they first kill someone should fit an exponential distribution. The probability that a death would occur in the first t miles is 1 - e^(-lambda t) where lambda is the rate of killing people, is 1.25/100,000,000. I.e. 1 - e^( -(1.25 / 100,000,000) x 4), which is 5 x 10^-8.

If Uber has only a 5 x 10^-8 probability of driving "safely enough" they should lose their license at the very least.

Edit: Oops, 4 != 4,000,000. It should be 1 - e^( -(1.25 / 100,000,000) x 4,000,000) which is about 0.049...

Still, I think we can ask for better than a 5% chance of being the same as a human driver.

(also replaced stars with 'x' because HN was making things italic)

asift• 8 years ago

1.25 per 100 million miles is almost certainly a bad benchmark since the majority of those miles are interstate miles. Fatality rate per mile of urban driving would be much better, although I'm not really sure whether I would expect that number to be higher or lower.

Edit: Actually, maybe I'm wrong in assuming (a) the majority of miles driven are interstate miles, or (b) that the majority of existing miles logged by self-driving cars have not been on the interstate. Would love to see some data if anyone has it, although I suspect Google, Uber, et al. are reluctant to share data at this point.

stevenwoo3• 8 years ago

Also, probably 95 percent of the autonomous miles are under the easiest conditions, a sunny day between 9-5 because most are at the Arizona/California test centers.

cup-of-tea2• 8 years ago

As of December 2017 Uber had driven 2 million autonomous miles[1]. Let's be generous and double that, so 4 million.

How often did the human have to take control?

hikarudo• 8 years ago

If the accident had happened with the autonomous vehicle of ANY company, we would still be talking about this and estimating the number of deaths per 100 million miles.

Therefore, I think it would be more fair to consider all miles run by all autonomous vehicles all over the world in the denominator.

It is for the same reason that we want to consider all miles driven everywhere, not just those in Arizona.

ocdtrekkie5• 8 years ago

Note that the statistics we have to work with are relatively terrible. For example: Waymo's favorite statistic is "x miles driven", which is a terrible/useless statistic, because it treats all miles equally, and fails to note that the most complex driving is often short distances (intersections, merges, etc.) and doesn't account for the fact that most of those miles were repeatedly driving on a very small number of roads. But it looks good on marketing copy because it's a big number.

Additionally, our self-driving car statistics we tend to see today also tries to ignore the presence of test drivers and how frequently they intervene. As long as they can intervene, the safety record of self-driving cars is being inflated by the fact that there's a second decisionmaker in the vehicle.

EDIT: Oh, and human driving statistics are also bad because a lot of accidents don't even get reported, and then when they do get reported, a lot of it is through different insurance companies, that's before we get into nobody centrally tracking "miles driven", and that's why most statistics for human driving safety are more or less just an educated guess.

jmull• 8 years ago

I think it’s fine to attribute miles to autonomous vehicles which have drivers that can intervene... as long as that is how they are used outside of tests as well.

Just a guess, but I doubt having a test driver that can intervene will help safety statistics for autonomous vehicles much. I think we’ll find test drivers will usually be unable to notice and react fast enough when they are needed.

throwaway123156• 8 years ago

I agree the statistics we have are pretty terrible. However, in this case, I think Waymo's statistic is actually quite useful. It's likely that Waymo's x miles driven statistic is largely driven by the fact that Waymo has tested their cars on a small number of roads, in fairly safe settings. But that paints Uber in an even worse light. Waymo is supposedly ahead, or at least on par with Uber in self driving technology, and they have chosen to limit their testing and driving to safer and a limited number of roads. Uber has not. That seems to underscore the fact that Uber has pushed beyond their tech's capabilities even though their competitors have determined the tech isn't there yet.

Also, if someone had posted a poll a day ago as to which company's self driving cars were likely to be the first to kill somebody, I think the vast majority of people would have predicted Uber. I don't think that's a coincidence.

__abc2• 8 years ago

The statistics may prove meaningless. It's the emotional reaction of our legislators. Take the recent legislation regarding a single killed dog by United Airlines (my condolences to the family members/owners of that dog) vs. deaths via [insert any other thing that has multiple deaths and is currently under regulated].

Ajedi322• 8 years ago

We don't really have any statistics on autonomous car deaths yet. One fatality is only one data point; that's not _nearly_ enough information to come to any solid conclusion on the overall safety of the technology. (Not to mention the fact that a failure of any one particular implementation of self-driving car tech doesn't necessarily mean the other implementations are similarly unsafe.)

mtgx3• 8 years ago

No, the time to overact is now, before millions of these get on the road.

Actually, the time to "overeact" was even earlier, but most of self-driving cars ignored any criticism. So now you have stupid car companies cutting corners and killing people so they can be "first" on the market or whatever.

mattbrewsbytes2• 8 years ago

You know how sometimes you get a gut feeling or your awareness picks up somehow?

You experience it sometimes where you just happen to glance in the right direction, as a driver, and avoid horrible things from happening. Or hesitate to go through an intersection when a light turns green and some person is running a red. You and your body somehow knew but it can't be explained.

Computers don't have that, whatever it is.

drngdds• 8 years ago

The thing you're describing is coincidence/confirmation bias, not a real phenomenon.

rdiddly• 8 years ago

What if you already thought from the beginning that cars, including human-driven ones, and their infrastructure, were always a huge boondoggle (the grossest misallocation of resources in human history, as Jim Kunstler calls it), and an instrument that posits itself as the solution to a problem it caused (not unlike an addictive drug), and that meanwhile serves chiefly to centralize wealth and rend the social fabric?

Please petulantly downvote all minority opinions!

VikingCoder• 8 years ago

"Uber has paused self-driving operations in Phoenix, Pittsburgh, San Francisco and Toronto, which is a standard move, the company says."

...there's a "standard move" when one of their cars kills someone?

I don't think that really came across the way they wanted it to.

richjdsmith• 8 years ago

This will be the first major test of liability and insurance with a self-driving vehicle.

tracker1• 8 years ago

Really? This is a completely new tech, and process. How is any reaction "a standard move?" I guess it could be standard for any new tech to result in a death from autonomous machinery. In any case, things like this are why I don't think fully autonomous is nearly ready in city environments. I'm not saying it isn't close, and shouldn't be worked on. I just don't think the intelligence systems are near as worked out as they should be. Not to mention the level of sensors needed.

Tobba_• 8 years ago

I'm sure that pile of data will gain intelligence if we stir it hard enough. Any moment now.

Axsuul3• 8 years ago

Sad to hear it's happened this soon but this was inevitable.

bryanlarsen• 8 years ago

Arizona had 10 pedestrian deaths in a single week: https://www.azcentral.com/story/news/local/phoenix-breaking/...

alacombe5• 8 years ago

As somebody else pointed out, what is the ratio of total miles driven in metro Phoenix over fatalities ?

austinhutch• 8 years ago

There may be no company I trust less to be open and honest about this than Uber. The arms race between companies will likely keep the most important lessons from this event proprietary. I'm very much not looking forward to hearing execs answer for this as I'm sure it will have the same amount of humanity as the rest of their comms.

WhompingWindows2• 8 years ago

This is my thought as well. Compared with Waymo, Cruise/GM, Tesla, Lyft, and the many other start-ups in this space, I think Uber is the least ethically scrupulous. I also doubt their self driving tech is as advanced as many of the other players, and I wonder if those other companies would have been able to handle this situation.

dmode• 8 years ago

I know when Uber was stopped by CA DMV to operate its self driving cars without a permit, the Arizona governor went all out to promote the state as a beacon of business friendliness and deregulation etc. etc. Perhaps, when something is as critical as self-driving cars going around along with pedestrians and bikes and other vehicles, a more cautious and deliberate approach is warranted ?

Tiktaalik• 8 years ago

In the early 20th century the automobile industry mobilized to downplay the deadly threat automobiles posed to pedestrians, and lobbied to create new laws to frame pedestrians as being at fault instead of car drivers (eg. jaywalking).

I expect we'll soon see tech companies do the same in order to favour their autonomous automobile businesses at the expense of pedestrians and cyclists.

burlesona2• 8 years ago

This. There are many posts on this thread suggesting we could fix all this by lowering speed limits, etc. That's been known for a hundred years at this point.

If you want to get up on the history you can grab a copy of "Fighting Traffic," by Peter Norton. He goes through all the efforts cities made in the 1920's to limit cars, and how the motor lobby formed in opposition to this and little by little reversed the tables to the situation we find ourselves in today.

https://mitpress.mit.edu/books/fighting-traffic

mlinksva• 8 years ago

Or shorter version https://www.researchgate.net/publication/236825193_Street_Ri... or podcast version https://99percentinvisible.org/episode/episode-76-the-modern...

Indeed it has been known for a long time. This is an opportunity to reverse bad decisions made a century ago, regardless of how quickly self-driving cars actually are adopted.

https://www.strongtowns.org/slowthecars/

lr4444lr• 8 years ago

Meanwhile...

https://www.azcentral.com/story/news/local/arizona/2018/03/0...

abalone8• 8 years ago

It's likely this was actually a bicyclist, not a pedestrian.

The police are saying pedestrian walking outside the crosswalk but here's a picture of the crumpled bicycle: https://twitter.com/daiwaka/status/975767445859287042

You can see it's next to a sign. That sign lets us determine exactly where it happened on Google street view: https://www.google.com/maps/@33.4370667,-111.9430321,3a,75y,...

It's exactly where the vehicle lane crosses the bike line approaching the intersection. Mostly likely the Uber and bike collided when the Uber was crossing into the right turn lane.

pro_zac• 8 years ago

The news video in the original post also says bicyclist. Speed limit goes up to 45 mph just before the bridge.

davesque2• 8 years ago

Isn't it reaching to see a picture of a bicycle near a sign and then conclude that the accident must have happened exactly next to or near the sign? Maybe the police/witnesses dragged the bicycle to that location?

abalone8• 8 years ago

Nah. The Uber likely stopped in its tracks and it's in the picture too. You can see an official collecting evidence at the scene. The bicycle may have been pulled onto the sidewalk but it would be the sidewalk near the accident, not far away.

dang20• 8 years ago

Could you please not use allcaps for emphasis in HN comments?

This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.

__abc2• 8 years ago

Updated. Thank you for pointing me towards that as I was currently unaware.

dang20• 8 years ago

Ok great. I've detached this subthread from https://news.ycombinator.com/item?id=16620865 and marked it off-topic.

sdfget434• 8 years ago

That car has to have a dashcam -- what's the current caselaw with regards to the 5th amendment?

CobrastanJorji2• 8 years ago

I'm no lawyer, but I imagine that it wouldn't be hard to get a warrant for the dashcam footage. The fifth amendment is about speech, not possessions. Otherwise, all warrants would violate the fifth amendment.

kaibones• 8 years ago

I see the Waymo cars while I walk to lunch (AZ) quite often. One time while walking through the crosswalk a Waymo turning right cut me off as I started into the crosswalk despite my having the walk indicator. This seems to be a very difficult scenario for a self-driving car as there are often people at the corner that are either crossing the other direction or just waiting on the corner. I sent them a note via their feedback link, but never heard back.

stevenwoo3• 8 years ago

This is a non trivial situation for human drivers, too, when I am on a bicycle or driving a car I have to wait and look at the pedestrian until I am sure of their intention. That seems like a very difficult problem in that a lot of times people are on their phone or watching a child or dog, I often have to slow to a standstill and speak to people to ask them what they are planning on doing.

mikelward• 8 years ago

Indeed. Right on red is unsafe for pedestrians. I guess at least the requirement to stop before turning helps ensure the speed of impact has minimal consequences.

As a driver, there are certain heuristics, e.g. did my traffic light just go red, or is the pedestrian signal/traffic light in the same direction about to go green.

gexla• 8 years ago

Uber: 1 Pedestrians: 0 Game on.

dang20• 8 years ago

Please don't do this here.

Ajedi322• 8 years ago

So this is it then? First fatal accident involving a fully self-driving car which might have actually been the car's fault?

Well, it was obviously going to happen sooner or later. It'll be interesting to see what the fallout is. Up until now regulators have been, surprisingly, taking a pretty relaxed approach to regulation of self-driving cars. Hopefully this one accident doesn't impede development on other autonomous car programs too much.

abalone8• 8 years ago

This may not have been an unexpected circumstance. The "pedestrian outside of the crosswalk" report is likely wrong. There is visual evidence that strongly supports that this was a case of the Uber hitting a bicyclist while merging into the right turn lane.

Here's a picture of the crumpled bicycle (note dent on the front right of the car): https://twitter.com/daiwaka/status/975767445859287042

The sign it is lying next to lets us locate the exact point of the collision on Google street view: https://www.google.com/maps/@33.4370667,-111.9430321,3a,75y,...

It's right where the vehicle lane crosses the bike lane approaching the intersection. Very unlikely a pedestrian was walking a bike there. Much more likely the Uber hit the bicycle while merging.

EDIT: I take back "very unlikely" a pedestrian was crossing there. There is in fact a very weird "X" pathway in the median that pedestrians are not supposed to be in. I didn't see that until I looked at the satellite view.[1] So possibly she emerged from there and was walking her bike across the street. That still looks pretty bad for Uber though, since that street is a straight shot with clear visibility and the dent's on the right side of the car. Meaning it would have swerved left to avoid a late-detected obstacle rather than swerve right away from her entering the roadway. She also would have literally had to walk straight into oncoming traffic. I still think bicyclist is the most likely scenario and it's odd the PD doesn't mention the crumpled bicycle.

[1] https://www.google.com/maps/@33.4365931,-111.9425027,198m/da... (street view shows a "no pedestrians" sign)

jessaustin17• 8 years ago

Lots of confusion! It seemed the video was for an injury collision rather than a fatality? This is what we get from posting a Cincinnati source for an event that occurred in Tempe.

omarforgotpwd• 8 years ago

Wow, that's really bad! Easy to see how the code didn't factor in the possibility of someone using the bike lane, which is a very bad look for Uber.

phkahler2• 8 years ago

It's not even about the bike lane. It's about not hitting a person. Beyond that, it's about understanding unforeseen circumstances.

Piskvorrr11• 8 years ago

Now, I'm not a fan of AV, but you're definitely jumping to conclusions here.

Mononokay4• 8 years ago

This is looking worse and worse for Uber.

hourislate• 8 years ago

I can't imagine how bad it looks for Non Autonomous cars considering the 100's of pedestrians that are killed everyday in the USA.

mojomark• 8 years ago

This is making Kitty Hawk look a lot more attractive.

radley• 8 years ago

He posted a tweet later, it wasn't a cyclist:

https://twitter.com/daiwaka/status/975771533745336320

scoggs4• 8 years ago

Wait, so that picture / video of the bicycle on the ground at what looked like the scene had nothing to do with the accident at all? It seemed really focused on from my POV. I'd find it rather strange if the bicycle had nothing to do with it although this is just reporting, it's not like it's the official police report or anything.

skarap• 8 years ago

To repeat my comment from a previous discussion which brought a lot of downvotes: what happens when (not if) a self-driving car runs-over and kills someone (e.g. because of a software bug)? Do such cases cause criminal penalties? Who is penalized? Or will all cases of autonomous car accidents with deaths become civil cases? If so - do human drivers get the same new rules or if they kill someone by accident (because they got distracted) they still go to jail? Is that fair?

In this particular case I assume the operator will be thrown under the bus, which is also unfair.

foobaw• 8 years ago

I know quite a people working on this at Uber ATG. They are in panic mode.

untog3• 8 years ago

This opens up an interesting question going forward. We can't rely on Uber themselves to analyse the telemetry data and come to a conclusion, they're biased. So really, we need self driving car companies to turn over accident telemetry data to the police. But the police are ill equipped to process that data.

We need law enforcement to be able to keep pace with advances in technology and let's face it, they're not going to have the money to employ data analysts with the required skills. Do we need a national body for this? Is there any hope of a Republican government spending the required money to do so? (no)

dmoy• 8 years ago

On the one hand I'd say give it to the NTSB, because historically they are really good at this sort of thing.

On the other hand, I'd wonder if increasing NTSB scope this much would drastically decrease the average quality of NTSB's work. Scaling ain't easy.

jeiting• 8 years ago

Yeah, pilot here. NTSB is the right organization to handle this. The investigators over there do an amazing job of determining root cause from forensic evidence. I assume that will be the process here.

martin_bech• 8 years ago

Shouldnt this actually be pretty easy? The system on the Uber, should have a massive amount of cameras, and lidar. Basicly dashcams on speed, recording multiple angles of the accident. I would assume that everything is beeing recorded, for debug/Testing purposes.

gowld17• 8 years ago

NTSB should expand to launch an automotive division.

Bonus: Job creation to replace the jobs lost to automation

Quenty• 8 years ago

State governments, especially those authorizing self driving cars, could authorize this to some extent.

openasocket• 8 years ago

Aren't you basically just proposing that the NTSB analyze automobile telemetry, the same way they analyze aircraft telemetry data? Doesn't seem wildly outside the realm of possibility.

untog3• 8 years ago

The NTSB could certainly do it. But they'd need to expand a lot in a world where self driving cars are an everyday reality, which again comes back to the question of funding.

dagurp• 8 years ago

NTSB? https://www.ntsb.gov

jv22222• 8 years ago

Perhaps a company like open ai or some other company setup by all the players involved could run the analysis in cases like this.

roywiggins• 8 years ago

This sounds like something the NHTSA should do.

pvg• 8 years ago

That sounds pretty much like how things are done as it is.

For instance, take a look at:

https://www.ntsb.gov/investigations/AccidentReports/Pages/HW...

"The Tesla was equipped with multiple electronic systems capable of recording and transmitting vehicle performance data. NTSB investigators will continue to collect and analyze these data, and use it along with other information collected during the investigation in evaluating the crash events."

cubano• 8 years ago

Don't we already have a huge infrastructure of police, the courts, and insurance companies in place to decide these very things everyday?

I mean, how is this different then all of the other accidents that occur every day? Yes a self-driving car is involved, but do people really think autonomous cars aren't going to be involved in fatal accidents?

Of course they are...but I've always thought that autonomous vehicles only have to be like 10% safer for them to make tons of sense to replace human drivers.

rocqua8• 8 years ago

For the same reason we don't leave the investigation of plane-crashes to the attorney general and the courts. We care about more than 'who should we punish for this' in this case.

We want to know what happened, how it happened, how we could have prevented it, how likely it is to happen again, what assumptions or overlooked details lie at the heart of this.

The required level of expertise, effort and precision here are higher than in a regular traffic accident. Moreover, the required skill-set, knowledge base, and willingness to work in a new area here make this an exceptional case.

Finally, the outcome of this will be much more than liability in a single case. This could set the precedent for an entire field of industry. This could be the moment we find out self-driving cars are nearly a pipe-dream, or it could be the moment we kill self-driving cars at the cost of millions of preventable traffic accidents. This investigation just might determine a lot, again, that makes it exceptional.

hnal943• 8 years ago

Who would be liable? The owner of the AV? The manufacturer?

dsfyu404ed4• 8 years ago

How about we wait for the problem to present it and actually cause harm before we throw law enforcement, government regulation, a regulatory body, etc at it?

ryandrake• 8 years ago

A pedestrian struck and killed doesn’t count as “harm”?

pdabbadabba• 8 years ago

NTSB has already announced that it will investigate: https://techcrunch.com/2018/03/19/ubers-fatal-self-driving-c...

I guess we'll find out how competent they are at this, but I think they did a surprisingly good job with the previous Tesla investigation: https://dms.ntsb.gov/pubdms/search/hitlist.cfm?docketID=5998...

dragonwriter6• 8 years ago

Law enforcement already conducts traffic accident investigations where the involved drivers are biased parties of inconsistent honesty and imperfect memory that can and do both accidentally and intentionally misrepresent the facts.

I don't think self-driving cars and their sensor data, even if they rely on the operator to explain what the car “remembers”, fundamentally shift the landscape.

lsmarigo• 8 years ago

We need law enforcement to be able to keep pace with advances in technology

Agree, Kumail Nanjiani (comedian) has a great rant on twitter about exactly this, ethical implications of tech-

As a cast member on a show about tech, our job entails visiting tech companies/conferences etc. We meet ppl eager to show off new tech. Often we'll see tech that is scary. I don't mean weapons etc. I mean altering video, tech that violates privacy, stuff w obv ethical issues. And we'll bring up our concerns to them. We are realizing that ZERO consideration seems to be given to the ethical implications of tech. They don't even have a pat rehearsed answer. They are shocked at being asked. Which means nobody is asking those questions. "We're not making it for that reason but the way ppl choose to use it isn't our fault. Safeguard will develop." But tech is moving so fast. That there is no way humanity or laws can keep up. We don't even know how to deal with open death threats online. Only "Can we do this?" Never "should we do this? We've seen that same blasé attitude in how Twitter or Facebook deal w abuse/fake news. Tech has the capacity to destroy us. We see the negative effect of social media. & no ethical considerations are going into dev of tech.You can't put this stuff back in the box. Once it's out there, it's out there. And there are no guardians. It's terrifying. The end. https://twitter.com/kumailn/status/925828976882282496?lang=e...

It is scary, big tech orgs have no incentive or motivation to even consider ethical implications, whats worse is the American consumer has shown repeatedly that you're OK to do really shady stuff and as long as it means a lower price product/service for the consumer. We're in a kind of dark age of tech regulation and he's right it is terrifying.

adrianmonk7• 8 years ago

Maybe something like the IIHS ( https://en.wikipedia.org/wiki/Insurance_Institute_for_Highwa... ), which is funded by insurance companies?

Though that would of course depend on how insurance even looks for SDCs. Maybe big companies like Uber will self-insure.

DangerousPie• 8 years ago

I know this is a slightly insensitive question to ask right now, but assuming that what happened was that the car tried to turn right and the cyclist came from behind and crossed into its path[0]: Who would actually have been in the wrong?

I cycle along paths like this from time to time myself, and I always assume that I should let a turning car pass in front of me. As soon as I see the turn signal I will either fall back or even pass into the car lane to overtake on the other side.

Of course that's in part because I know I would lose the fight anyway, but also because I think I am not actually supposed to be "overtaking" them on the right side, so they have the right of way before me.

What do the rules say in this case?

[0] This is based on this link posted elsewhere: https://www.google.com/maps/@33.4370667,-111.9430321,3a,75y,...

ggggtez4• 8 years ago

These things tend to be based on the factual evidence of the case, as well as local and state laws. Talk to your lawyer.

phyzome5• 8 years ago

Depending on the state law, the right-turning car probably is supposed to yield to a cyclist in the bike lane going straight. But of course, every experienced cyclist knows that what is right is not what is safe; you should aim to travel just behind the car's rear bumper to give yourself time to stop if they swerve right.

This is the insanity of (most) bike lanes, that you have a lane to the right of a right-turn lane that can go straight. And that's why I don't use most bike lanes. They're trouble.

linkregister4• 8 years ago

That move is against the law, at least in San Francisco. The vehicle should enter the bike lane to turn right.

propman• 8 years ago

This is purely an emotional initial reaction to hearing this without knowing most facts, but I just have to vent and for once be part of the outrage culture and make emotionally charged statements before more info comes out.

Uber ruining self driving cars that can save 20k lives each year and improve our economy and get millions out of poverty and improve everyone's quality of life by being short sighted, financial driven, and bringing the worst aspects of Silicon Valley to the extreme.

I am so sorry to everyone at Google and Waymo who have tried so hard to not let anything like this happen by being extremely cautious and safe. Ughhhh I feel so frustrated, this is going to cause so much backlash.

Travis, Anthony and whoever else pushed for this to go ahead without caring about safety is evil. This action doesn't have a direct correlation of evil like directly murdering someone. But these highly intelligent individuals ignored their conscience and the easy deduction that actions to compromise safety, laws, decency for quick profit are immoral. Whether they admit it or not they KNEW that this was a high possibility and refused to take precautions creating this awful culture. 20k lives a year, and all the benifits that will be delayed for years are on their head.

They won't face any consequences, just convince themselves of absolution of guilt and do nothing to remedy the situation, but I have no respect for them.

Obi_Juan_Kenobi• 8 years ago

20k lives each year

Road deaths are ~40k each year in the US alone.

I agree about Uber; smells like shit, looks like shit, tastes like shit, you better believe it's shit to the core.

Tharkun• 8 years ago

If only there was this much outrage and discussion every time a human driver killed a pedestrian.

4rgento• 8 years ago

I wonder how many different "cultures" do drivers have?

People from different localities drive differently. I hope they use drivers from say, NY, LA, and various small towns to teach the AI.

People that lives in towns with ~ 10k people drive differently that a new yorker on the same town.

EDIT: They might be training with a biased sample

make3• 8 years ago

Why am I not surprised Uber is the first company to kill someone. With their top notch ethics, ah. This actually makes me really angry, more than I had anticipated.

jehlakj• 8 years ago

Just curious: Do tech companies “pay” the state so they can try out their latest and greatest? Or do they look for states with the least amount of restrictions?

Zarath2• 8 years ago

Isn't this just lobbying?

apeace5• 8 years ago

I'm really shocked at the trend in comments here.

Yes, we get it. Fatalities will happen sometimes, they are unavoidable sometimes, and what matters in the long run is if we can achieve a significant overall reduction in fatalities.

But my god, a person was just killed by a computer. Can't we have some compassion and humility?

Let's, as a community, set the standard for how we will react to these events. Let's make sure Uber releases detailed data on what happened, whether they were at fault or not. Let's hold the media accountable for their reporting. Let's mourn the loss of life and think about how we can solve these problems.

But for christ's sake, please stop posting the same thing everybody already knows, which gets posted on HN whenever a self-driving article comes out.

Can't we do better?

---

EDIT: I clarified what I meant here [1]

Wasn't trying to say this thread should be all about mourning. This is HN so we should talk about technology, even when a person died. I'm pointing out one specific argument that gets repeated over and over in place of a substantive discussion, and I think we can do better.

[1] https://news.ycombinator.com/item?id=16621589

davesque2• 8 years ago

What trend in the comments is bothering you? It's a little unclear from your comment.

ryanwaggoner• 8 years ago

Fatalities will happen sometimes, they are unavoidable sometimes, and what matters in the long run is if we can achieve a significant overall reduction in fatalities.

I haven't even read any of the comments on this story yet, and this type of pro-self-driving-car response is what I expect to be in the majority. That and anything anti-Uber :)

apeace5• 8 years ago

The trend I'm seeing is to immediately defend self-driving cars based on "but if you look at it statistically I'm willing to bet this is better than human drivers".

It is as if HN users expect the news headline to be: "an event occurs which is statistically speaking non-anomalous".

If one doesn't feel we have enough data to draw conclusions yet, then sure, fine. But a person just died. This is a newsworthy story and it deserves our best attention and discussion, not repetition of something obvious ad nauseam.

alyx• 8 years ago

I find what might help those that quote statistics is to self-reflect and ask the question, how would you react if your father/mother/sister/son/daughter/etc was the victim?

Would you care that statistically self-driving cars have killed less people than human drivers?

Y_Y• 8 years ago

Yes. I'm in favour of safer drivers, robot or human.

Anderkent• 8 years ago

Well, just because people react badly when confronted with personal tragedy doesn't mean you have to do the same when it's not applied to you.

For example it's clearly excusable to be angry / depressed / looking for someone to blame when a personal tragedy happens to you. But that doesn't mean you should replicate that behaviour when a tragedy happens to someone else. We should distance ourselves from subjects where we're emotionally compromised, not guide our decision by the example of emotionally compromised people.

arcticfox2• 8 years ago

I think that's a strange thought experiment, since it's basically requesting that you think emotionally and as irrationally as possible to draw conclusions.

wyager• 8 years ago

how would you react if your father/mother/sister/son/daughter/etc was the victim?

How you would react when emotionally distraught shouldn’t be the model for how you act all the time.

pera• 8 years ago

Would you care that statistically self-driving cars have killed less people than human drivers?

I don't think there are enough data points right now to make such claim, but please correct me if I am wrong.

slavik81• 8 years ago

Would you care that seatbelts are statistically likely to save lives if your loved ones died drowning in their car due to them?

What do we do differently based on the answer to that question?

wlll2• 8 years ago

…statistically self-driving cars have killed less people than human drivers?

I realise that the point you're making is about emotional turmoil and not really statistics, but it looks like so far Uber aren't doing so well in terms of driving safety:

https://news.ycombinator.com/item?id=16620736

So yeah, if it turned out that Uber were sending cars out into the world without sufficient engineering or testing* and they killed someone (especially someone I know) then I would be understandably furious.

* In order to get to market sooner? I don't trust Uber and I wouldn't put this past them. Nothing I have seen of the Uber self-driving car program makes me want to be anywhere near one on a road.

lucb1e2• 8 years ago

how would you react if your father/mother/sister/son/daughter/etc was the victim?

Shit, of course. But I'd like to believe that I'll realize that banning self-driving cars is not going to make things any better, just like I'd never propose banning cars in general. I'm not sure how it would help here to imagine it's my daughter; it's not as if we imagine it being our daughter in every traffic accident we hear of, since that would be crippling given the sheer number of them...

jdietrich2• 8 years ago

All deaths are a tragedy to someone. A statistic is an aggregate account of untold suffering and misery. To be truly compassionate, we must learn to see the humanity in statistics. We must recognise that our emotional reaction to the death of a loved one and our indifference to the death of a stranger is not a sign of compassion, but egoism. As enlightened citizens, it is our duty to use all our ingenuity and effort to reduce the sum total of suffering.

This death is a tragedy. There were 37,461 such tragedies in 2016. Every single one of them matters equally. It is only by studying and analysing them in aggregate that we can prevent future tragedies. Seat belts, air bags, crumple zones, anti-lock brakes, median dividers and rumble strips are tremendous acts of compassion. Statisticians, scientists and engineers have made an incalculable contribution to human welfare.

thereare5lights2• 8 years ago

Can't we do better?

no. there's a reason our community has the reputation of being heartless. it's because it's true.

dang20• 8 years ago

It's not at all true. People here care very much. It would be weird if a large population sample (4M a month or so) had a different human nature than humans in general.

What is true is that large internet forums suffer from systemic effects that compound into problems that no one has yet figured out how to deal with.

newnewpdro2• 8 years ago

What's noteworthy of this news event is the involvement of an autonomous vehicle.

People are being killed in accidents constantly. We aren't going to be in a constant state of mourning over it.

This is implicitly a discussion about autonomous vehicles, and the very currently relevant and heated topic of wether we want these things on our streets.

acover• 8 years ago

40100 deaths in 2017. Averaging 110 per day.

akerro• 8 years ago

Fatalities will happen sometimes

How many self-driving cars are there on the roads right now? And what's fatality rate? I' quite sure it's higher now than compared to human-only driver.

they are unavoidable sometimes

Were autonomous cars not designed to prevent such situations?

Let's, as a community, set the standard for how we will react to these events

Yes and let's punish non-conformists!

which gets posted on HN whenever a self-driving article comes out.

I honestly have no idea what that comment is.

Thaxll• 8 years ago

Probably not since Google drove millions of KM without a single fatality.

Frondo• 8 years ago

Yes, absolutely, let's ostracize people who react to this situation (a machine killing some random person in a fatal beta test) with anything other than compassion and humility.

Those are not people I want in my engineering community; just impossible to trust they'd make the right calls and have the right priorities in developing cool new stuff.

tree_of_item3• 8 years ago

Do better than what? This is a technology news forum, the relevant question is how this affects the technology. People are going to talk about the potential long term trends in self-driving technology. What exactly were you expecting?

emilsedgh• 8 years ago

I think the relevant question must be how the technology is going to affect lives.

whyleyc2• 8 years ago

"The relevant question is how this affects the technology"

Is arguably the whole problem with our industry in a nutshell. The more relevant and largely ignored question should be "how this affects human beings".

wongarsu3• 8 years ago

The news isn't about the fact that a human died, unless I'm somehow missing the constant flood of ~50 news items a day about other deadly car accidents. The news is very much about self-driving cars and our relation to them. That's what people react to. The human death is tragic, but not really the topic.

carussell2• 8 years ago

While you're admonishing people for their comments and employing stealth implicature like, "let's make sure Uber releases detailed data", do you think that maybe you should include a prominent notice disclosing your (and your former colleagues') ties to the industry that Uber is operating in—most importantly as a competitor?

dang20• 8 years ago

Yikes - bringing someone's personal details into threads as ammunition in an argument is offside. It crosses into personal attack, so please don't.

https://news.ycombinator.com/newsguidelines.html

danso30• 8 years ago

I'm far from being an Uber-supporter here but I think you're overreacting. As others have pointed out, this story is only relevant to HN (and to the national media at large) because it involves Uber and AV, not because it involved a pedestrian dying. It makes sense that the discussion centers around policy and systems since few people here can claim to have even known the victim. Nor does participating in this discussion preclude anyone from going to the victim's Facebook page/funeral and expressing condolences.

m3rc• 8 years ago

What is that supposed to mean? If the only way it directly impacts a forum is because of the company doing the killing, the killing is irrelevant?

nonbel2• 8 years ago

Reminds me about this comment a few weeks ago (which made an impression on me) :

'How low has the SW development bar gone, if "it's okay" now means "at least it's not directly killing people"?' https://news.ycombinator.com/item?id=16541235

We now see the standard has slipped to: "Is the death of one person relatively so bad? People die every day."

Next a large crowd of people will be killed and the standard will become: "It's not so bad compared to creating skynet or starting thermonuclear war."

I'm not sure what comes after that.

dwighttk2• 8 years ago

"It's not so bad compared to the alternative"

I mean, we two are still capable of discussing it, at least!

milankovic• 8 years ago

Interesting, isn't it?

But if you think about it, that's not a pattern anyone should be unfamiliar with. History is full of it.

I don't know, I really have no strong opinion here. But I think stuff like this is truly fascinating. There seems no end to where rationalization can go.

Welcome to the ambivalence of reality...

jerbs3• 8 years ago

I don't think the people arguing for a different standard are saying that it is ok for one person to die because people die every day. They're asking for observers to look at self-driving cars in comparison to human-driven cars. Because people dying is such a bad thing that we want to prevent, the relevant question should always be, "will this change make it more or less likely that somebody will die." And that is why it is relevant to compare pedestrian deaths from self-driving cars to pedestrian deaths from human-driven cars, but not relevant to compare it to skynet or thermonuclear war.

I don't know, however, how that comparison turns out. From the other comments, it looks like Uber's self-driving cars might be less safe than human-driven cars--in which case you and I would end up on the same side of the argument.

intopieces• 8 years ago

A person was not killed by a computer. There was a person behind the wheel responsible for taking control of the car in a situation like this. If this is a computer killing someone, then every instance of a person being killed while cruise control was engaged also counts.

usrusr• 8 years ago

Yeah, someone put that poor person there despite all warnings that humans would perform terribly as standby drivers once the AI becomes good enough to not require interventions all the time.

mrtksn• 8 years ago

Lately, I'm quite disappointed with the tech community's approach to the society.

Lot's of people are talking as if human beings are a resource which exploitation should be secured for the businesses and technical curiosities.

No, people, humans are what matters and it just happens that there are business opportunities and scientific curiosities. No, it not O.K. to kill few people to speed up the development of self-driving vehicles. No, it is not O.K. to have total surveillance so that we improve AI.

portofcall2• 8 years ago

It’s not just you, it’s a lot of the country that’s rapidly going from concerned to pissed. If they read more threads like this, they’d probably already be coming for us with torches and bad intentions. They’re worried that tech is full of disconnected, vaguely autistic (and they don’t know what that means beyond iRobot), too-smart-for-their-own-good people who care more about making money than poeple. In my experience that’s incredibly unfair, but it only takes a small percentage of loud and callous voices to make us all look bad.

We need to start cleaning our own house, or it’s going to be cleaned for us by people with agendas and little understanding of what they’re cleaning!

zzzcpan• 8 years ago

But my god, a person was just killed by a computer. Can't we have some compassion and humility?

You have to recognize that your culture and your moral standards are not universal and you are in no position to force them on everyone else. It is also foolish to expect everyone across the globe to react in a certain way to certain things.

On the other hand, death topics is the side of HN that is heavily biased towards what mods feel about it and their culture. They even ban accounts and give out warnings if you happen to disagree with them and react differently. So there is not much sense to even discuss this.

danso30• 8 years ago

re: your edit. I agree with the sentiment, but I didn't quite get your comment because before you posted it, the most upvoted comment/discussion on this story was about Uber failing to meet safety standards [0]. People have been posting about "the greater good" but seems far from a consensus.

[0] https://news.ycombinator.com/item?id=16620042

lhorie4• 8 years ago

What bothers me is that people are jumping at all sorts of narratives and conclusions when the article clearly pointed out that the incident is still under investigation.

bargl2• 8 years ago

I think the argument needs to be made for ETHICAL development of mission critical software. The Aerospace industry has been writing software that has peoples lives in their hands for years. NASA has been doing the same.

We need to see accountability for any death caused by a car being operated by a computer. We need to see integrity in testing and metrics that show that these are actually safer for humanity. One more death than expected due to software vs human is too many.

I don't remember where I saw it but the process for creating mission critical software at NASA is insane. They analyze everything. It's a completely different and slower process than the one we use for building websites. SpaceX has probably increased the productivity of said systems, but I'd like to see how they develop mission critical software to keep things safe.

The point of self driving cars is to reduce death, if you can monetize that great, but if you can't hit that baseline in some sort of testable way, then you shouldn't be able to put your car on the road.

Bluestrike22• 8 years ago

I understand your point. But people are capable of recognizing the tragic loss of a life while also trying to understand both the cause of it and the implications.

The victim's life had meaning. To herself, to her friends and loved ones, and all of those affected by her untimely loss. We don't know her story, nor the stories of the over 6,000 pedestrians (a number that's been increasing) killed annually in the US alone as of 2016.[0] Nor the 37,461 killed in US motor vehicle crashes in the same year.[1]

We talk about the technology, because it was involved in this woman's death and because the technology has the potential to drastically reduce the overall number of such accidents. Understanding why she died, and how similar accidents can be prevented in the future, can give society back something out of an accident that only took. If we can learn from her death, anything at all, it behooves us to do so.

Yes, it's cold. It's extremely cold. But it's something doable. Even if these discussions offer little impact on the actual development of self-driving cars, they can still impact the political debate that's only now just starting. For better or worse, there will be those who point to this woman's death as an argument against the technology. Recognizing that her death is not simply one death, but an example of a type of incident that claims thousands each year without self-driving car involvement, is critically important. At times, statistics can dehumanize and cause us to overlook the very real pain depicted as a small part of a number. But at the same time, they also permit us to recognize problems and measures ways in which we can mitigate and even solve them.

0. https://www.npr.org/2017/03/30/522085503/2016-saw-a-record-i...

1. https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...

Jach• 8 years ago

1.75 people die per second. Calling for compassion and mourning of one stranger over the others seems fake. Personally I'm pleased over the circling of wagons, the very real danger is emotional kneejerks to this will set back the entire self-driving industry by at least 5 years. The trend of comments is a helpful loyalty signal even if it's not very substantive.

abhirag• 8 years ago

You are seeing the Trolley Problem Dilemma[https://en.wikipedia.org/wiki/Trolley_problem] in action. Majority of respondents when asked that problem would choose to let a few people die to save a larger number of people. Emphasis being on "let a few people die" and not "kill a few people", machines (i.e. lever in that problem and autonomous car in this scenario) somehow evoke a more logical and less emotional response, I guess machines create an emotional distance between the people who would die and the respondent. If the same respondents would somehow feel more responsible for the death, i.e. the machine is removed from the scenario the response would be much different.

Not implying that the lack of empathy is in anyway acceptable, but people should know that when machines are involved in such scenarios it is much easier for them to distance themselves emotionally, this is not new, weapons are a prime example of this. Maybe by knowing this dilemma people can react better to such tragedies.

I hope people show more empathy knowing this.

skookumchuck6• 8 years ago

If human driven cars kill 1000 people a year, and autonomous cars kill 100, will it be politically unacceptable to switch to autonomous? I suspect it will be.

awakeasleep2• 8 years ago

A portion of that depends on the funding on either side of the issue.

If the autonomous cars kill 1100 per year, but autonomous car companies fund a bunch of studies saying the benefits outweigh the drawbacks, it'll be politically acceptable.

skookumchuck6• 8 years ago

People think emotionally, not rationally. People have all sorts of greatly distorted views on the risks of various activities.

JepZ• 8 years ago

Why am I not surprised that this happend with an Uber self driving car?

In fact, their culture of ignoring rules and common sense might be okay for business development, but with respect to human safety it is just irresponsible and inhuman when it comes to self driving cars.

maxyme• 8 years ago

If you're familiar with that part of ASU campus you honestly could have seen this coming. There are a few different self driving cars in Tempe (Waymo, Uber, GM...) and Uber drives by far the most aggressive. They drive on a pedestrian heavy road and drive faster than most of traffic. They accelerate rapidly when a light changes and can brake hard. There are always tons of pedestrians in the area and it isn't uncommon to almost get run offer when even crossing during the day.

vamin• 8 years ago

I wonder if it's because Uber algorithms are trained using Uber driver data (taxi drivers tend to be very aggressive drivers).

ClassyJacket2• 8 years ago

I don't find that at all, I feel like they're happy to take their time and make more money.

fudged713• 8 years ago

This matches my experience driving elsewhere. You need to adapt to the context, the car should not drive the same way everywhere.

IgorPartola10• 8 years ago

This is what's going to happen. If you've ever seen a machine learning algorithm in action, this isn't surprising at all. Basically, they'll behave as expected some well known percentage of the time. But when they don't, the result will not be just a slight deviation from the normal algorithm, but a very unexpected one.

So we will have overall a much smaller number of deaths caused by self driving cars, but ones that do happen will be completely unexpected and scary and shitty. You can't really get away from this without putting these cars on rails.

Moreover, the human brain won't like processing these freak accidents. People die in car crashes every damn day. But we have become really accustomed to rationalizing that: "they were struck by a drunk driver", "they were texting", "they didn't see the red light", etc. These are "normal" reasons for bad accidents and we can not only rationalize them, but also rationalize how it wouldn't happen to us: "I don't drive near colleges where young kids are likely to drive drunk", "I don't text (much) while I drive", "I pay attention".

But these algorithms will not fail like that. Each accident will be unique and weird and scary. I won't be surprised if someone at some point wears a stripy outfit, and the car thinks they are a part of the road, and tries to explicitly chase them down until they are under the wheels. Or if the car suddenly decides that the road continues at a 90 degree angle off a bridge. Or that the splashes from a puddle in front is actually an oncoming car and it must swerve into the school kids crossing the perpendicular road. It'll always be tragic, unpredictable and one-off.

orbitur2• 8 years ago

I won't be surprised if someone at some point wears a stripy outfit, and the car thinks they are a part of the road, and tries to explicitly chase them down until they are under the wheels. Or if the car suddenly decides that the road continues at a 90 degree angle off a bridge. Or that the splashes from a puddle in front is actually an oncoming car and it must swerve into the school kids crossing the perpendicular road.

Are you working on the next season of Black Mirror?

In all seriousness, my fear (and maybe not fear, maybe it's happy expectation in light of the nightmare scenarios) is that if a couple of the "weird and terrifying" accidents happen, the gov't would shut down self-driving car usage immediately.

jessaustin17• 8 years ago

If that's bad, what happens when the robocars get hacked?

IgorPartola10• 8 years ago

I am definitely not. Their version of the future is too damn bleak for me.

Your fear is very much grounded in reality. US lawmakers tend to be very reactionary, except in rare cases like gun laws. So it won't take much to have restrictions imposed like this. Granted, I believe some regulation is good; after all the reason today's cars are safer than those built 20 years ago isn't because the free market decided so, but because of regulation. But self driving cars are so new and our lawmakers are by and large so ignorant, that I wouldn't trust them to create good regulation from the get go.

pps43• 8 years ago

Are you working on the next season of Black Mirror?

In Black Mirror, all cars in the world will simultaneously swerve into nearby pedestrians, buildings, or other cars.

basura0454785• 8 years ago

will never happen. least not with any explicit depth sensors. any car with lidar has depth perception orders of magnitude better than yours and would never chase an object merely because it resembles road markings

kevinwang• 8 years ago

To be fair, all the examples you gave could also happen to a human driver.

beedogs2• 8 years ago

I mean, maybe the self-driving car shouldn't exist if it's just going to run people over.

ttoinou• 8 years ago

    I won't be surprised if someone at some point wears a stripy outfit, and the car thinks they are a part of the road

Shouldn't be an issue with 3D cameras

iliis• 8 years ago

Well, in that case a non-textured or reflective clothing could have a similar effect.

IncRnd2• 8 years ago

You can't prove a negative and should be careful about promising what may turn out to be false. There is potentially quite a bit of money to be made by people with the auto version of slipping on a pickle jar. When there is money to be made, talented but otherwise misguided people apply their efforts.

bertomartin• 8 years ago

Seems like you're jumping to conclusions here. Let's wait to see what exactly happened. I highly doubt that any of these companies just use "straight" ML. For complex applications, there's generally a combination of rules-based algorithm and statistical ML based ones applied to solve problems. So to simply: highly suspect predictions aren't just blinded followed.

obastani• 8 years ago

Very little of what goes into a current-generation self-driving car is based on machine learning [1]. The reason is exactly your point -- algorithmic approaches to self-driving are much safer and more predictable than a machine learning algorithms.

Instead, LIDAR should exactly identify potential obstacles to the self-driving car on the road. The extent to which machine learning is used is to classify whether each obstacle is a pedestrian, bicyclist, another car, or something else. By doing so, the self-driving car can improve its ability to plan, e.g., if it predicts that an obstacle is a pedestrian, it can plan for the event that the pedestrian is considering crossing the road, and can reduce speed accordingly.

However, the only purpose of this reliance on the machine learning classification should be to improve the comfort of the drive (e.g., avoid abrupt braking). I believe we can reasonably expect that within reason, the self-driving car nevertheless maintains an absolute safety guarantee (i.e., it doesn't run into an obstacle). I say "within reason", because of course if a person jumps in front of a fast moving car, there is no way the car can react. I think it is highly unlikely that this is what happened in the accident -- pedestrians typically exercise reasonable precautions when causing the road.

[1] https://www.cs.cmu.edu/~zkolter/pubs/levinson-iv2011.pdf

falcolas9• 8 years ago

Actually, because there's a severe shortage of LIDAR sensors (much like video cards & crypto currencies, self driving efforts have outstripped supply by a long shot), machine learning is being used quite broadly in concert with cameras to provide the model of the road ahead of the vehicle.

IgorPartola10• 8 years ago

But that's the issue: identifying a pedestrian vs a snowman or a mailbox or a cardboard cutout is important when deciding whether to swerve left or right. It's an asymptotic problem: you'll never get 100% identification, and based on that, even the rigid algorithms will make mistakes.

LIDAR is also not perfect when the road is covered in 5 inches of snow and you can't tell where the lanes are. Or at predicting a driver that's going to swerve into your lane because they spilled coffee on their lap or had a stroke.

With erratic input, you will get erratic output. Even the best ML vision algorithm will sometimes produce shit output, which will become input to the actual driving algorithm.

ape42• 8 years ago

Exactly, with LIDAR the logic isn't very tricky. if something in front, stop.

gfodor• 8 years ago

Do you have any experience developing autonomous driving algorithms? Because you are making a lot of broad claims about their characteristics that only someone with a fairly deep level of expertise could speculate about.

Gudin• 8 years ago

Or take a slow approach like Google Waymo which was almost perfect so far on the roads. Uber is rushing it, and this could cost whole industry.

utopcell3• 8 years ago

exactly. I would personally never trust an Uber self-driving car, specifically because I've lost trust in the company itself.

legohead• 8 years ago

Your comment reminded me of the One Pixel Attack [1] and my joke about wearing a giant yellow square costume...

[1] https://github.com/Hyperparticle/one-pixel-attack-keras

dalbasal• 8 years ago

Another quirk of people, particularly acting via "People in Positions of Authority" is that they will need to do something to prevent next time.

Why did this happen? What steps have we make sure it will never happen again? These are both methods of analysing & fixing problems and methods of preserving a decision making authority. Sometimes this degrades into a cynical "something must be done" for the sake of doing, but... it's not all (or even mostly) cynical. It just feels wrong going forward without correction, and we won't tolerate this from our decision makers. EVen if we will, they will assume (out of habit) that we won't

We can't know how this happened. There is nothing to do. ..and.. this will happen again, but at a rate lower than human driver's more less opaque accidents.... I'm not sure how that works as an alternative finding out what went wrong and doing something.

Your comment is easily translated into "you knew there was a glitch in the software, but you let this happen anyway." Something will need to be done.

jakobegger• 8 years ago

It‘s amazing how quickly things can happen after an accident.

In a nearby town, people have petitioned for a speed limit for a long time. Nothing happened until a 6 year old boy was killed. Within a few weeks a speed limit was in place.

grumdan2• 8 years ago

Even if we assume that we wanted to address this for real, I fear that it will be next to impossible to actually assess whether whatever mistake caused this has actually been addressed when all the technology behind it is proprietary. I can easily see people being swayed by a well-written PR speech about how "human safety" is their "top priority" without anything substantial actually being done behind the scenes.

I think any attempts to address such issues have to come with far-ranging transparency regulations on companies, possibly including open-sourcing (most of) their code. I don't think regulatory agencies alone would have the right incentives to actually check up on this properly.

anonu• 8 years ago

Totally agree on your premise that we can rationalize humans killing humans - but we cannot do so with machines killing humans.

If self-driving cars really are safer in the long-run for drivers and pedestrians - maybe what people need is a better grasp on probability and statistics? And self-driving car companies need to show and publicize the data that backs this claim up to win the trust of the population.

falcolas9• 8 years ago

If a human kills a human, we have someone in the direct chain of command that we can punish. If an algorithm kills a person... who do we punish? How do we punish them in a severe enough way to encourage making things better?

Perhaps, similar to airline crashes, we should expect Uber to pay out to the family, plus a penalty fine. 1m per death? 2? What price to we put on a life?

rebuilder• 8 years ago

I'd be surprised if you could educate this problem away just by publishing statistics. Generally, people don't seem to integrate statistics well on an emotional level, but do make decisions based on emotional considerations.

I mean, people play the lottery. That's a guaranteed loss, statistically speaking. In fact, it's my understanding that, where I live, you're more likely to get hit by a (human-operated) car on your way to get your lottery ticket than you are to win any significant amount of money. But still people brave death for a barely-existent chance at winning money!

JoeAltmaier2• 8 years ago

Hm, people are fine with folks getting mauled in an industrial accident, or killed misusing power equipment. So its not purely a machine thing.

Maybe because its unexpected - the victim is not involved until they are dead?

tjoff• 8 years ago

It's a sense of control. I as a pedestrian (or driver) can understand and take precautions against human drivers. If I'm alert I can instantly see in my peripheral vision if a car is behaving oddly. That way I can seriously reduce the risk of an accident and reduce the consequences in the very most cases.

If the road was filled with self-driving cars there would be less accidents but I wouldn't understand them and with that comes distrust.

Freak accidents without explanations are not going to cut it.

Also, my gut feeling says this was a preventable accident that only happened because of many layers of poor judgement. I hope I'm wrong but that is seriously what I think of self-driving attempts in public so far. Irresponsible.

RoyTyrell• 8 years ago

maybe what people need is a better grasp on probability and statistics

Definitely, though my interpretation of your statement is "self driving cars have only killed a couple people ever but human cars have killed hundreds of thousands". If that's correct, that's not going to win anyone over nor is it necessarily correct.

While the state of AZ definitely has some responsibility for allowing testing of the cars on their roads, Uber needs (imo) to be able to prove the bug that caused the accident was so much of an edge case that they couldn't easily have been able to foresee it.

Are they even testing this shit on private tracks as much as possible before releasing anything on public roads? How much are they ensuring a human driver is paying attention?

avip• 8 years ago

If you ask me which one is coming first, Quantum computing or "better grasp of probability and statistics" among the general public - I take the first with 99% confidence.

thearn4• 8 years ago

Moreover, the human brain won't like processing these freak accidents.

I think this is really key. The ability to put the blame on something tangible, like the mistakes of another person, somehow allows for more closure than if it was a random technical failure.

derefr3• 8 years ago

These are "normal" reasons for bad accidents and we can not only rationalize them, but also rationalize how it wouldn't happen to us: "I don't drive near colleges where young kids are likely to drive drunk", "I don't text (much) while I drive", "I pay attention".

Tangent: is there a land vehicle designed for redundant control, the way planes are? I've always wondered how many accidents would have been prevented if there were classes of vehicles (e.g. large trucks) that required two drivers, where control could be transferred (either by push or pull) between the "pilot" and "copilot" of the vehicle. Like a driving-school car, but where both drivers are assumed equally fallible.

greedo2• 8 years ago

Pilots don't share control of an aircraft; the copilot may help with some tasks, but unless the captain relinquishes control of the yoke (etc.) he's flying it. So you'd still have issues where a cars "pilot" gets distracted, or makes a poor decision.

pvelagal• 8 years ago

May be for now, autonomous driving should be limited to freeways and roads that dont have pedestrian crossings/pavements.

jusonchan• 8 years ago

This is a great point. Solve it one step at a time.

But the problem is Uber's business plan is to replace drivers with autonomous vehicles ferrying passengers. i.e. take the driver cost out of the equation. Same goes for Waymo and others trying to enter/play in this game. It's always about monetization which kills/slows innovation.

Just highway-mode is not going to make a lot of money except in the trucking business and I bet they will succeed soon enough and reduce transportation costs. But passenger vehicles, not so much. May help in reducing fatigue related accidents but not a money making business for a multi-billion dollar company.

That being said, really sad for the victim in this incident.

coldtea• 8 years ago

So we will have overall a much smaller number of deaths caused by self driving cars

Why? This is what the self-driving cars industry insists on, but has nowhere near been proven (only BS stats, under ideal conditions, no rain, no snow, selected roads, etc -- and those as reported by the companies itself).

I can very well imagine a greater than average human driving AI. But I can also imagine being able to write it anytime soon not being a law of nature.

It might take decades or centuries to get out of some local maxima.

General AI research had also promised the moon once again in the 60s and 70s, and it all died with little to show of in the 80s. It was always "a few years down the line".

I'm not so certain that we're gonna get this good car AI anytime soon.

naasking7• 8 years ago

This is what the self-driving cars industry insists on, but has nowhere near been proven

Because machines have orders of magnitude fewer failure modes than humans, but with greater efficiency. It's why so much human labour has been automated. There's little reason to think driving will be any different.

You can insist all you like that the existing evidence is under "ideal conditions", but a) that's how humans pass their driving tests too, and b) we've gone from self-driving vehicles being a gleam in someone's eye to actual self-driving vehicles on public roads in less than 10 years. Inclement weather won't take another 10 years.

It's like you're completely ignoring the clear evidence of rapid advancement just because you think it's a hard problem, while the experts actually building these systems expect fully automated transportation fleets within 15 years.

icc9711• 8 years ago

If self-driving cars 1. don't read texts whilst driving, 2. don't drink alcohol, 3. stick to the speed limit, 4. keep a 3-4s distance to the car in front, 5. don't drive whilst tired 6. don't jump stop signs / red lights it will solve a majority of crashes and deaths. [0]

The solutions to not killing people whilst driving aren't rocket science but too many humans seem to be incapable of respecting the rules.

[0] http://www.slate.com/articles/technology/future_tense/2017/1...

adrianmonk7• 8 years ago

Well, one answer is either it will be positively demonstrated to be statistically safer, or the industry won't exist. So once you start talking about what the industry is going to look like, you can assume average safety higher than manual driving.

Splines2• 8 years ago

Interesting insight. While self-driving cars should reduce the number of accidents, there are going to be some subset of people who are excellent drivers for which self-driving cars will increase their accident rates (for example, the kind of person who stays home when its icy, but decides that their new self-driving car can cope with the conditions).

emodendroket14• 8 years ago

So we will have overall a much smaller number of deaths caused by self driving cars, but ones that do happen will be completely unexpected and scary and shitty. You can't really get away from this without putting these cars on rails.

One of the big questions I have about autonomous driving is if it's really a better solution to the problems it's meant to solve than more public transportation.

IanCal• 8 years ago

This is what's going to happen. If you've ever seen a machine learning algorithm in action, this isn't surprising at all. Basically, they'll behave as expected some well known percentage of the time. But when they don't, the result will not be just a slight deviation from the normal algorithm, but a very unexpected one.

Do we even know yet what's happened?

It seems rather in bad taste to take someones death, not know the circumstances then wax lyrical about how it matches what you'd expect.

yelloweyes2• 8 years ago

Very well put.

It boggles my mind that a forum full of computer programmers can look at autonomous cars and think "this is a good idea".

They are either delusional and think their code is a gift to humanity or they haven't put much thought into it.

web007• 8 years ago

I don't believe I'm delusional, my code is certainly medium-to-okay. I've put a lot of thought into this. I think autonomous cars are a very good idea, I want to work on building them and I want to own one as soon as safely possible.

Autonomous cars, as they exist right now, are not up to the task at hand.

That's why they should still have safety drivers and other safeguards in place. I don't know enough to understand their reasoning, but I was very surprised when Waymo removed safety drivers in some cases. This accident is doubly surprising, since there WAS a safety driver in the car in this case. I'll be interested to see the analysis of what happened and what failures occurred to let this happen.

Saying that future accidents will be "unexpected" and therefore scary is FUD in its purest form, fear based on uncertainty and doubt. It will be very clear exactly what happened and what the failure case was. Even as the parent stated, "it saw a person with stripes and thought they were road" - that's incredibly stupid, but very simple and explainable. It will also be explainable (and expect-able) the other failures that had to occur for that failure to cause a death.

What set of systems (multiple cameras, LIDAR, RADAR, accelerometers, maps, GPS, etc.) had to fail in what combined way for such a failure? Which one of N different individual failures could have prevented the entire failure cascade? What change needs to take place to prevent future failures of this sort - even down to equally stupid reactions to failure as "ban striped clothing"? Obviously any changes should take place in the car itself, either via software or hardware modifications, or operational changes i.e. maximum speed, minimum tolerances / safe zones, even physical modifications to configuration of redundant systems. After that should any laws or norms be changed, should roads be designed with better marking or wider lanes? Should humans have to press a button to continue driving when stopped at a crosswalk, even if they don't have to otherwise operate the car?

Lots of people have put a lot of thought into these scenarios. There is even an entire discipline around these questions and answers, functional safety. There's no one answer, but autonomy engineers are not unthinking and delusional.

myrryr2• 8 years ago

We look at the alternative which is our co-workers, and people giving us our specs, and our marketing teams and think 'putting these people in charge of a large metal box travelling at 100kmh interacting with people just like them - that is a good idea'...

It is not that we think that software is particularly good, it is that we have a VERY dim view of humanity's ability to do better.

newnewpdro2• 8 years ago

Having these massive robots on our public streets better at least result in unfettered, transparent public access to all the data informing the thing in the time surrounding the crash.

This shift towards autonomous vehicles is utterly unnecessary. They could be simply improving the safety systems of the vehicles to intervene when the driver is about to cause an accident.

Instead they're doing the opposite - expecting the human to intervene when the robot is about to cause an accident.

danso30• 8 years ago

The Washington Post has an updated story with the victim's name:

https://www.washingtonpost.com/news/dr-gridlock/wp/2018/03/1...

Police said the vehicle was northbound on Curry Road when a woman, identified as 49-year-old Elaine Herzberg, crossing from the west side of street, was struck. She died at a hospital, the department said.

No mention of her being a bicyclist. My gut instinct is to accept that the victim was indeed a pedestrian -- with the assumption that they'd have that detail cleared up by the time they have her identity ready to release to the public.

TillE2• 8 years ago

the vehicle was northbound...crossing from the west side of street

So, she approached from the left. That would make it significantly less likely that she just suddenly appeared in front of the car, right? It's not a narrow street.

scoggs4• 8 years ago

What I'm saying is unlikely as I'd most likely assume she was traveling along the sidewalk for some distance before being hit but there is always the chance she was walking the bike across the grass and came out from that direction (where those bushes are, they seem tall enough from what I can see)? This is all assuming when you say 'left' you meant on the left side of the Google Street View posted above, though.

abalone8• 8 years ago

I'd think so too, but there is literally a crumpled bicycle in the photo of the scene. Also the Uber has a dent on the right side, which faces east.

It's possible she was walking her bicycle across the street, but that still looks pretty bad for Uber. That street is a straight shot with clear visibility.

phyzome5• 8 years ago

And if she was walking a bicycle, it was extremely unlikely that she "darted out". It takes some skill to do that.

(Still need more info, though.)

yohui3• 8 years ago

Yeah, the police clarified that she was walking her bike across the street: https://twitter.com/AngieKoehle/status/975824484409077760

t0mbstone2• 8 years ago

The location of the crash (from what I can tell): https://www.google.com/maps/place/N+Mill+Ave+%26+E+Curry+Rd,...

danso30• 8 years ago

Close. Going by the article photo [0], the location was in front of these double-left turn signs, and the Uber vehicle would have been facing the other direction, i.e. northbound:

https://www.google.com/maps/place/N+Mill+Ave+%26+E+Curry+Rd,...

[0] The caption for this photo says the crack in the road represents "burned out flares at the location where a woman pedestrian was struck and killed". Not sure if that is exactly where she was hit, or if that's where her body landed, or if the flares were just near the location.

http://s3.reutersmedia.net/resources/r/?m=02&d=20180319&t=2&...

joeevans1000• 8 years ago

Hype cycles are beginning to cause loss of life. Self driving cars are being allowed to operate at a far faster rate than other transportation innovations would have. This is because of the enthusiasm around technology. We need to realize the difference between social media apps and steel.

xingi538• 8 years ago

My understanding, from talking to someone who works for the NTSB, is that the software controlling autonomous vehicles is currently completely unregulated, and that companies like Uber, Tesla, etc. have refused to provide access to their software for external review.

Recently I've seen a lot of comments defending autonomous vehicles from a statistical standpoint, but even if these cars have the potential to be safer on average, it scares me to think that the software driving a car is less regulated than its airbags or seatbelts. Especially considering the auto-industry's tenancy to ignore safety issues without external pressure [0].

Is my information outdated? Have there been efforts to review or regulate autonomous driving software in the last year?

[0] https://en.wikipedia.org/wiki/Unsafe_at_Any_Speed

ggg9990• 8 years ago

Of course it was Uber. I'm sure they don't even give a fuck any farther than the cost to their brand either.

jonthepirate• 8 years ago

Former Lyft eng here. From my vantage point, as an industry, we're nowhere near where we should be on the safety side. The tech companies are developing driving tech privately instead of openly. Why can a private for profit company "test" their systems on the public roads? The public is at serious risk of getting run over by hacked and buggy guidance / decision system. Even when a human operator has his hands hovering 1 inch off the steering wheel and his foot on the brake, if the car decides to gas it and swerve into a person, it is probably too late for the human crash test driver to overtake. This is going to keep happening. The counterargument FOR this is that it is overall a good idea for the transportation system if the number of crashes & deaths is statistically less than human operated cars. I see this as the collision of what's possible with what's feasible and that we are years away from any of this being close to a good idea. :( Very sad for the family and friends.

decacorn• 8 years ago

The counterargument you propose is invalid because it would be DECADES until >90% of the entire current automotive market is converted to fully autonomous. MEANING cars currently on the road either being retrofitted or completely removed from road and replaced by an AV.

So we are all in a lose-lose. I'm anticipating severe backlash from Congress this week, if not today.

Shivetya• 8 years ago

because it does not require handing over control to the car to get a good test. the cars can simply record their decision making process and immediately flag all exceptions, which includes hitting anything. It would be very interesting to see all exceptions because it is to be expected all these systems must fail.

we don't need active testing on our streets. automakers would have dared to test ABS or "safety" systems in such a manner for the simple reason of liability.

dang20• 8 years ago

Could you please not use allcaps for emphasis in HN comments? This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.

floatrock• 8 years ago

Why can a private for profit company "test" their systems on the public roads? The public is at serious risk of getting run over by hacked and buggy guidance / decision system.

Interesting way of putting it. Kinda echos the "privatize the profits, socialize the losses" sentiment from the financial crisis.

bduerst2• 8 years ago

Theoretically the reputation cost would discourage for-profit organizations from releasing unsafe self-driving vehicles for testing on public roads.

The ironic part here is that Uber already has a strong reputation for skirting the law and not caring. Their response and the following audits will be worth following.

treis5• 8 years ago

Self driving cars could potentially save tens of thousands of lives a year not to mention all of the other social benefits and quality of life improvements. No doubt whoever gets it working first will make a ton of money, but society will see a massive benefit as well.

abalone8• 8 years ago

Really it's "socialize the costs" as well, considering that autonomous vehicle development was funded by taxpayers.[1] Like most high tech.

[1] https://en.wikipedia.org/wiki/DARPA_Grand_Challenge

jessriedel10• 8 years ago

We already have a system for privatizing negative externalities in traffic accidents (and in 95% of the rest of life). It's called tort law, and it will certainly be used in this case if Uber was at fault.

There are of course situations were externalities fail to be privatized (e.g., traffic congestion), but I don't think auto accidents is one of them.

amelius4• 8 years ago

Perhaps companies need to test their safety devices first. I.e., first prove that their LiDAR correctly identifies pedestrians, cyclists, etc. From there, build test-vehicles with redundancy, e.g. with multiple LiDAR devices. Then prove that the vehicles actually stop in case of emergency. And only then actually hit the road.

Of course, the US department of transportation should have set up proper certification for all of this. They could have easily done so because they can arbitrarily choose the certification costs.

scdc• 8 years ago

if we wait for the gov to set up a certification for this, we'll delay the whole industry 10 years.

jastanton• 8 years ago

What you're describing is a driving test that every human needs to go through before they're allowed to drive on public roads. Something that can be revoked temporarily or permanently.

I would be very interested in a 3rd party (government or private) create a rigorous test (obstacles, weather conditions, etc...) for self driving vehicles. Becoming "XXX safety Certified with a Y Score" for all major updates to the AI could help restore confidence in the system and eliminate bad actors.

morogop• 8 years ago

You've hit upon one of the most obvious ways to improve the safety of these systems. Deploy the systems more broadly, without giving them active control.

Then, you can start to identify situations where the driver's actions were outside of a predicted acceptable range, and investigate what happened.

Additionally, if you have a large pool of equipped vehicles you can identify every crash (or even more minor events, like hitting potholes or road debris) and see what the self-driving system would have done.

The realistic problem is that Uber doesn't give a shit. As such, deployment will never be optimized for public safety. It will be optimized for Uber's speed to market.

dkonofalski9• 8 years ago

the US department of transportation should have set up proper certification for all of this

I think you're severely underestimating the path that something like this would have to take. The certification itself would be under so much scrutiny and oversight that it would take years for that to get done. Unfortunately, the technology is far more readily available and easy to get working than the political capital required to create a certification for this.

dgregd2• 8 years ago

My old comment from https://news.ycombinator.com/item?id=15076613

So much talk about LIDAR and other sensors. Why nobody talks about obvious idea of Road Object Message Bus? ROMB is a protocol where each road object (a traffic light, a sign, a car, a bicycle, etc.) transmits info about itself. A car could broadcast its direction vector, intention to turn, any non ROMB moving object it sees. A traffic light could broadcast current state and when it is going to change. That information would greatly enhance overall security, especially during rain and snow conditions, when even LIDAR fails.

Self-driving is such important (just after eliminating combustion engines) that we could upgrade existing cars with cheap ROMB boxes. Vehicle GPS tracking system costs about $30. ROMB box would cost about $60. Let's say that from 2027 all cars have to have a ROMB box to enter a downtown ...

Let's say your car ROMB received info about the white truck, while your car cameras and vision recognition systems see just a cloud and any truck in 100 m range.

ROMB purpose is not to replace cameras or LIDARs, but to extend gathered info.

dang20• 8 years ago

Please don't paste old comments into new conversations. Especially not twice in the same thread!

spyder• 8 years ago

Even worse: There are people who test their DIY self-driving hacks on public roads:

https://youtu.be/GzrHNI6eCHo?t=100

Using https://github.com/commaai/openpilot , which is cool but not on public roads.

icc9711• 8 years ago

That is indeed reckless, but that guy is testing open source self-driving technology.

Given that all the main car companies are keeping their technology private I don't see how open-source systems are supposed to keep up without people doing this.

I think you can also contrast this to the many more people who decide to drink alcohol and then drive.

merb• 8 years ago

in germany you would get punished for doing something like that. (well somebody would need to catch you first, of course)

chimeracoder8• 8 years ago

Why can a private for profit company "test" their systems on the public roads?

Because at some point, it's literally not possible to test something that's ultimately intended for use on public roads anywhere but public roads. Or, to put it in terms that would be more familiar to developers, "you can test all you want in QA, but prod is the ultimate test of whether your code really works".

As mentioned downthread, the problem here is that Arizona basically gave companies full leeway, without having to demonstrate that they had done sufficient tests internally before putting the cards on public roads. Apparently they're not even required to report data on accidents to the state, which is absurd.

I wouldn't be surprised if Uber were ultimately found to be cutting corners, but Arizona is also responsible for rushing to allow self-driving cars on their roads without doing their own due diligence first.

davewritescode2• 8 years ago

This is what it means when politicians say they’re cutting red tape. They’re letting industry do whatever the fuck they want at the expense of everyone.

fudged713• 8 years ago

When it is deemed necessary to test the vehicles on public roads after they have shown to be mostly safe, ALL of these vehicles should have flashing lights like any emergency vehicle.

I'm stunned that when I drive in Texas, there don't seem to be regulations over which vehicles can have flashing lights and with which colors. Yet in these other cities you have experimental robots that are designed to blend in with other cars as much as possible. NO! Make them stick out.

dkonofalski9• 8 years ago

Which is hilarious to me because AZ is the bastion of those "Don't treat on me" stickers and all this anti-regulation sentiment...until someone gets killed by a lack of regulations and then everyone wants to lock everything down and blame the government for not doing their due diligence.

cup-of-tea2• 8 years ago

Why can a private for profit company "test" their systems on the public roads?

My thoughts exactly. It is wrong. We have a chance to do this properly, openly and together, but instead our governments are letting private companies put our lives in danger.

debt• 8 years ago

That's why many unnamed companies in the self-driving space are testing solely on enormous closed-off tracks in the South Bay.

ivoras• 8 years ago

When regular cars first appeared, were they not tested "live" immediately? And yes, there was FUD even back then, and in parts of the world a backslash which resulted in sometimes silly safety rules.

The largest real difference seems to be the speed of communication nowadays, the rate at which public opinion can be manufactured and globally spread.

soperj2• 8 years ago

And a countless number of people have been killed.

CAT0• 8 years ago

When regular cars first appeard vehicles had to be led by a pedestrian waving a red flag or carrying a lantern to warn bystanders of the vehicle's approach.

https://en.wikipedia.org/wiki/Red_flag_traffic_laws

davewritescode2• 8 years ago

Real cars were an evolution of an existing technology (the carriage). We replaced a horse with an engine and a steering wheel.

In this case have a machine replacing a human and using non deterministic algorithms to do so.

Automated driving ONLY makes sense on specific, well mapped, sensor enabled, pedestrian free roads. Let’s focus on that use case before we let these things take over our towns and cities.

osrec2• 8 years ago

You are right on the money. Also, why must self driving cars be on the roads with human drivers at all? We're an unpredictable bunch, and lives are at stake! We should simplify the problem domain and keep self driving cars away from other human drivers and pedestrians (even once the technology reaches a very advanced stage). When they are actually ready to go mainstream, then think about putting them on roads with humans (and even then, dedicated routes and infrastructure specifically for self driving cars may be the safer way to go).

andrepd• 8 years ago

Why can a private for profit company "test" their systems on the public roads? The public is at serious risk of getting run over by hacked and buggy guidance / decision system.

Because the for profit company has a say in government and lawmaking, the person getting run over doesn't.

SilasX4• 8 years ago

Why can a private for profit company "test" their systems on the public roads?

Some phase of testing will necessarily be of public roads. The fact that they started that phase prematurely doesn't somehow mean that "private for profit companies" should never test on public roads.

Boeing tests new airplanes in the public airspace too, over public areas.

rpmcb• 8 years ago

Boeing certainly tests in public airspace, but only as part of a standardized certification process overseen by the US government - a process and set of regulations that are written in blood.

Uber abides by no such testing standardization.

arkh2• 8 years ago

I'm for a middle-of-the-road way of doing things at first: make some highway lanes for autonomous cars only. Separate those lane physically. Remove the speed limit there to encourage people to get cars which can use those lanes.

Implement a way for the highway operator to affect the cars on it: make way for emergency vehicles, adapt to infos the operator can aggregate and transmit, change the speed of cars to the slowest one around etc.

Once the majority of people have self driven cars and those have been proven in this controled environment you can think about allowing them on open roads. Or not. Or adapt those open roads to what self driven cars need.

EADGBE• 8 years ago

make some highway lanes for autonomous cars only. Separate those lane physically

sure, makes sense to me.

But God, that would cost a fortune.

smogcutter• 8 years ago

GM's "super cruise" fills the exact role you're describing aside from removing the speed limit, and aside from the billions of dollars in public infrastructure spending and increased traffic for everyone else.

megy8• 8 years ago

Why can a private for profit company "test" their systems on the public roads?

I mean, this is the way cars have always operated. It is somehow any different to let them "test" there brake system, there acceleration systems? Car companies have always been able to do whatever they want, who cares about the deaths.

For some reason, when it comes to roads, we all just accept a certain amount of deaths each year. Over a million people are killed every year around the world, up to 40,000 in the US alone. That is acceptable according to every car driver.

hkmurakami• 8 years ago

Car companies have (multiple) private tracks that they stress test their vehicles on. Car companies are extremely risk averse. Public road testing for something safety related is going to be very late stage in development/testing, if at all.

noir_lord• 8 years ago

That is acceptable according to every car driver.

All of them, I mean you took a poll? that must have taken a while.

My understanding (as a non-US resident) is that parts of the US are built around the assumption you drive.

belorn• 8 years ago

The FLOSS side has argued for openness in critical systems for ages. Medical devices such as pacemakers, air plane systems, automatic breaks on cars, voting machines, and the list just goes on. If the device get hacked or has any bugs then people will likely die (except for the voting machine), but the code is kept private even for those running in devices which literally gets operated into your chest and which a person has to trust with their own life. Governments all over the world seems to be very much against the idea to require that such critical code is made openly for the sake of safety, as that would leak the oh so important trade secrets.

wuliwong• 8 years ago

I can't imagine you would want any major new transportation technologies released without being tested on public roads?

wang_li2• 8 years ago

There shouldn't be any testing on public roads until the executives and engineers are willing to put on blindfolds and spend six hours running back and forth across a track on which a hundred of their autonomous cars are maintaining 60 MPH in both directions.

overgard• 8 years ago

To be fair though, it's not exactly like we're where we need to be at on the safety side from a human standpoint. I'd take my chances with a robo-car over a 16 year old texting any day.

c2210• 8 years ago

I strongly agree with this sentiment. If this technology is going to be tested on the public then all the datasets from these vehicles should be openly shared and development should be done cooperatively. Let the market decide winners based on metrics like cabin comfort and leasing terms rather than odds of causing grievous injury.

oh_sigh3• 8 years ago

It is simple: Tech companies are allowed to test their systems on public roads because the communities/states have given them permission to do so(with restrictions like roads allowable, max/min speed allowable, safety driver present behind wheel, etc).

wruwew8uu9• 8 years ago

Why is Uber allowed to test its software for free on public roads ? Why doesn't it use a purpose-built test-track of its own, where it must prove that its software won't kill people (using robotic crash-test dummies to avoid) before it is allowed to use public roads ?

wlll2• 8 years ago

I just don't trust Uber and its engineers enough to trust them as a self-driving car company. Even before this incident.

Their organisation has been shown to be deceitful, their CEO to be devious. I can completely believe they would cut corners or claim safety where they know there is risk. To get to market ahead of the competition knowing that the fate of road users lay in a dice roll.

Technically they had to resort to stealing Waymo's IP to get ahead, and their self-driving cars have previously been shown to behave recklessly.

I do, tentatively, trust Waymo. I've seen enough about the resources they've put into this, the extensive testing they've done, and their safety record so far to at least give them a shot.

Mtntk• 8 years ago

there is a standard in motor industry called misra, because the target user is human you have to met some certain safety level. Otherwise bad things happen like this.

Maybe self driving industry needs to have some safety rules too to prevent when the unexpected happens.

paul7986• 8 years ago

When will this company just go away ... after everything it's done ... it's now killing innocent people!

UncleEntity• 8 years ago

A while back I was riding my bicycle home from work and saw one of the Uber robocars coming the other direction while I was making a left turn so decided to test their reaction algorithm by starting my turn while they were still approaching -- car didn't even flinch while a human driver would've at the very least honked at me.

As an aside, I've also been hit in a bike lane in Tempe by a human driver unfortunately, if it were a robocar I'd be living on the beach somewhere off the settlement check.

nopriorarrests2• 8 years ago

I wonder how this will be treated legally.

Police can't just write "killed by python script" as a cause of death, that would be crazy, and there is no legal framework for cases like that. My guess is that A.I. could be equated to cruise-control, meaning person behind the wheel is responsible.

Maybe someone knows better?

drcode• 8 years ago

People are injured by malfeasance of a corporation involving many employees all the time, such as when a bridge collapses- There is nothing that says a police person needs to immediately find a single guilty person 10 minutes after an accident like this.

nopriorarrests2• 8 years ago

I agree, for large engineering projects, such as bridges and buildings, obviously many employees are involved and in many cases you can't even single out someone who is responsible.

But, to my best knowledge, there are no "corporations" on the road. No matter what company this person is driving for and under what type of contract, this person is responsible for all kinds of incidents.

hugh4life• 8 years ago

The self driving car craze baffles me. I would only trust it if it were on a road made for it and the cars communicated to one another.

jpao792• 8 years ago

Or at least have the self driving cars driving only along specific routes well marked, well maintained autonomous driving routes similar to light rail/bus lanes so pedestrians know to be extra careful.

And for long haul trucking applications, freeway lanes at specific times of late night/early morning could be dedicated for autonomous driving.

ChuckMcM• 8 years ago

Something of a self-driver car Rorschach test. On the one hand there is the tragedy (car hits person, person dies), on the other there is the technology (computer doing the driving), and then there is the fog of reporting where actual data is hard to come by because people are people and report on the things they saw/heard that were important to them.

Last night there was a story on CBS about the first 'self driving bus' going live in San Ramon California. [1] Where the reporter steps out in front of it to see if it will stop (spoiler alert, it does).

And while it was a tragedy, its unfortunate that because it was a 'self driving' car this fatality gets more coverage than the thousands who are killed by 'human driven' vehicles. Bicycle advocacy groups have been arguing for years that better, separated, bike lanes would save lives. Perhaps the companies behind self driving can get behind that effort to protect bike riders from humans and make it easier on their cars.

[1] https://www.cbsnews.com/live/video/20180319105443-california...

zepn• 8 years ago

This paradigm-shift is a chance to address one of the leading causes of preventable death in the world.

It doesn't surprise me for a moment that this was Uber (though it might have been a Tesla). From all I've seen and read, these companies are racing unscrupulously, and some have inferior technology compared with others.

Waymo seem to be approaching the problem from a safety direction rather than a pure race for profit.

We need to hold SDs to a higher standard than we hold human drivers. Not 1.5x the standard; 1000. 10,000. 100,000.

And every death must be treated as a serious failure of engineering. These are preventable deaths.

djsumdog2• 8 years ago

You can't really get away from this without putting these cars on rails.

We should just really build rails. Self driving cars are dumping billions of dollars onto a problem better solved by rails. Several cities already have self driving trains including London, Singapore and Kuala Lumpar, and they transport millions of people every day without incident:

http://penguindreams.org/blog/self-driving-cars-will-not-sol...

Self driving car research is asinine. It won't even being to solve the transport problem in America and it's round-peg/square-hole tech. It has the cool factor but there are existing tech that are insanely more useful for a fraction of the cost. American hate public transportation (thanks GM) that we'll never see real/good transport implemented in America.

cle• 8 years ago

Rail isn't something you can solve solely with tech, though. It requires building new infrastructure, which requires enormous political cooperation, which is an intractable problem in our current political climate. Self-driving cars do require some political cooperation, but it's much less because it purportedly doesn't require new infrastructure.

ramphastidae• 8 years ago

From my experience, relative to the rest of the first world, American highways and streets are dilapidated and neglected. I don’t see how that doesn’t get worse when self-driving cars start using those streets more efficiently than humans.

IgorPartola10• 8 years ago

Right. Which is why tech companies are the wrong people to solve transportation. When you only own a hammer, everything is a nail.

TN412• 8 years ago

We Should really just build rails

The beauty of cars vs rails is that cars can go more than just a set in stone angle or path defined by a rail. Cars can go in and out of parking lots and dirt roads. Rails would be more automated, yes, but not as versatile.

mozumder2• 8 years ago

Wherever there is a road, there can be rail.

Autonomous cars aren't a viable tech until the roads are reengineered with rails built in. It's just way too big of a technical problem to solve with computers alone.

Right now the death rate of autonomous cars is far higher than human drivers and it is never going to catch up given that it hasn't advanced technologically recently.

rags2riches• 8 years ago

Why not rails for the long distance high speed travel and then low speeds off the rails for that final distance?

jbigelow76• 8 years ago

Rails empty out into areas that are parking lots, and maybe you don't get the irony of rail but the more of it you have the less parking lots you need.

And as far as dirt roads go, if the self driving cars can't make it through well defined, paved roads what are the odds they are going to accurately navigate some dirt road?

test6554• 8 years ago

Due to geofencing, we are building a sort of rails. We're creating cars that are only useful in a limited, well-mapped area.

rocqua8• 8 years ago

Rails don't work for the last mile. The great thing about cars is that they don't suffer from the last mile problem.

It seems like a much better plan is to separate drivers from other people more rigorously, and maybe mandate people drive the last-mile themselves (with AI help, obviously).

wastedhours• 8 years ago

Rails definitely do work well for the last mile in very developed cities (i.e. London), but it's probably an unsustainable infrastructure cost for most places. That being said, it's easy to just change where that last mile is in the future (i.e. mixed zoning, flexible working environments, increasing mega-cities etc...).

CalRobert6• 8 years ago

For what it's worth, it doesn't really need to. Feet work well for the last half mile or so, and a well-designed city can get stops this close.

rhizome3• 8 years ago

San Francisco used to have rail streetcars all over the city, many more than the 6 lines now.

jpao792• 8 years ago

What happens if the 'rail' was actually a specific, demarcated line that autonomous self-driving cars used on specifically selected roads that augmented existing public transportation options? That would alert human drivers/pedestrians/bicyclists to be a bit more careful - similar to the brightly colored green lanes that are starting to be provided for bicyclists.

Similarly for long haul trucking applications - could the far left lane/carpool lane be safely dedicated to autonomous trucks during the hours of midnight-3AM with some bright blinking signs paid for by the autonomous trucking industry trade group?

radicalbyte2• 8 years ago

Here in The Netherlands we use bikes. They're great; so much faster than driving for short distances. It helps that housing is for the vast majority of people within 10-15 mins cycle and the land being flat.

Spooky23• 8 years ago

Roads aren’t free! With the technology available today, I bet a rail based solution could be more economical.

liquidcool• 8 years ago

I would love public transportation like I've experienced it in Europe, but train tracks have exceptionally tight tolerances and high costs to build/maintain vs. roads. I hope Musk will fix this, but probably not before "good enough" self-driving cars arrive. Also, trains get into accidents pretty regularly. First Google hit:

According to the US Department of Transportation, there are about 5,800 train-car crashes each year in the United States, most of which occur at railroad crossings. These accidents cause 600 deaths and injure about 2,300.

DanielleMolloy2• 8 years ago

I don’t oppose public transport at all and the US has a lot to improve in this area. But e.g. East Asia and Europe have plenty of rails, and people still want to drive their own cars, and there are still an unbelievable number of accidents.

We should require that every traffic participant gets a sensor, and connect all in a (local) network, instead of only relying on machine learning and its vulnerability to the unpredictable situations where accidents occur. Pure autonomous self driving won’t be the final answer here.

Please correct me if I’m not up to date here, but to my last knowledge self driving car machine learning lacks the most important data, that of dangerous situations. I’m surprised that Uber considered it ready for roll out.

jsjohnst2• 8 years ago

We can’t even standardize on a format across browsers without severe inconsistencies and bugs, yet you expect vehicles of all types and pedestrians to cooperate in a network? That’s not even taking into account the outliers who refuse to conform (or worse, those who try to hack it).

TraffigDoge2• 8 years ago

Germany here, we don't actually have "plenty" of rails for public transport, we love our Autobahn (no general speed limit!) and our cars, yet our vehicle fatality rates are among the lowest in the world (half that of the US).

The reason that fatalities are 10x higher in Asia is that people drive (and walk!) like lunatics and vehicle fatalities are just shrugged off.

Just be more orderly is basically what I am saying. Also eat more sauerkraut - it's healthy!

daphneokeefe• 8 years ago

My great-grandfather was killed by a streetcar (on rails) on Market Street in downtown San Francisco.

dfee• 8 years ago

I couldn’t put my finger on it at first, but I then realized why your comment doesn’t make any sense - you’re effectively arguing “if I could master-plan society and the economy for a while, this is how I’d decree it to be.”

Maybe some people don’t want to live in dense urban areas, maybe they don’t want to ride on a rail, maybe they want privacy. But looking out the window, I can say American society has soundly rejected the rail as the ultimate solution.

Short of understanding your perspective as a longing personal desire for trains, your conclusion is so far removed from reality that it’s nonsense.

dnautics• 8 years ago

I wouldn't say society has rejected the rail in America, but rather that the political structure has rejected the rail. There's a dominant paradigm in public transport that it must be run by the government (which ironically is somewhat counter to the broader american ethos). Yet there are places like Singapore where the public transport is run via (in the past fully, but currently partially) for-profit, publically traded companies, where the QOS is far higher than the US, and the service is markedly cheaper, and I would say, generally more accessible than it is in the US.

There would be considerable US pushback against such a system, sadly.

zanny2• 8 years ago

How can you reject something you never had or used? Almost nobody alive in the US today has ever engaged with general commuter rail. Trains as a common mode of transit died off with the rise of cars and planes post WW1 and had them near-eradicated post WW2.

You can't say people reject something they don't even know is a possibility.

derefr3• 8 years ago

maybe they want privacy

Every sci-fi movie of the last 50 years has solved that problem: you don't use trains; you put the cars on the rails (by grouping them together into roller-coaster-looking setups.)

maybe they don’t want to ride on a rail

Same here: cars on rails can, in most conceptions, signal to the rail system to switch them onto an offramp that dumps them back onto a road.

Basically, here, "rails" are serving as safer replacements for highways. Every road that isn't a highway would still be a road. You're just enabling people to, effectively, "have their car take the train", rather than having to leave their car at the train's parking lot.

pizza• 8 years ago

Maybe American developers have rejected the rail, but I don't think American passengers have.

gt_3• 8 years ago

But looking out the window, I can say American society has soundly rejected the rail as the ultimate solution.

“Soundly”? If you mean in the way a child “soundly” rejects vegetables, maybe. But, the American public has never had this conversation. Capitalism and it’s ensuing cultural affects had it for them.

lisper• 8 years ago

We should just really build rails.

How will that help? The reason that the trains in London and Singapore don't kill people is not that they are on rails, it's because they're underground.

TraffigDoge2• 8 years ago

Jawohl! Our Straßenbahnen kill people all the time, because they take part in regular traffic. If you walk in front of one, it can't steer away!

You need to put the trains below ground, or well above!

Aunche2• 8 years ago

Most people don't live in a dense enough city to justify rails.

gscott• 8 years ago

It would be easier to put a small transmitter in the middle of each lane run on solar power. This would keep the cars centered in the lane except for when they are executing some action.

Bluestrike22• 8 years ago

You're right. There's a lot of research and experience that suggests a well-developed rail network would have all sorts of benefits. We could take it a few big steps further and point to other problems like suburban sprawl in America, dying small towns, etc.

But there are a couple of really big, really massive problems. We have trillions in existing infrastructure and trillions more in existing property that's been developed over the course of decades. A lot of that development has come with unintended consequences. Just look at Houston after Hurricane Harvey and how their development and zoning practices took the insane rains Harvey deposited and magnified their negative consequences.[0] Decades of short-sighted decision-making took a hurricane and made it worse. Some changes will likely be made, but they won't solve nor likely keep up with a problem that's only going to get worse. That'll happen because the alternative, demolishing huge portions of the sprawling concrete city, isn't economically viable. The city can't be moved.

It's baggage. And there's so much more. Tens of trillions of dollars worth. Then there are existing roads, bridges, and other infrastructure connecting all of it. Designed for cars, and much of it legacy infrastructure that doesn't take into account modern infrastructure design principles. Add cities and towns, some of which are (charitably) in less-than-optimal positions. Towns, strip malls, and other structures that wouldn't exist if it weren't for suburban sprawl. Baggage. And then there's the people. People with ideas on what their lives ought to be like and where they want to live. There's a town, Centralia, Pennsylvania, that is literally on top of a massive underground fire that's been burning for over half a century. There was a massive relocation effort with federal and state funds. The town has no services, its zip code was discontinued in 2002, and it's still burning with all of the potential health risks that involves. There are still five residents there today, who have fought every attempt to relocate them. They will continue to live there until they die, at which point their homes will finally be taken and demolished through eminent domain.[1] If we can't bulldoze a town like that, we sure as hell can't take a bulldozer to normal ones.

Baggage. What it means is that we can't simply replace everything. It can only be done piecemeal, which means new infrastructure must take into account the need to accommodate everything that was developed under the old. High speed rail can be built out in the United States, and I genuinely hope that it is. But it literally can't replace the car in its entirety.

Self driving car research is--compared to the cost of major infrastructure replacement, let alone an alternative that simply isn't feasible--practically a rounding error. It doesn't hurt us, because the option was never a binary choice: self-driving cars OR rail. Self-driving cars give us the ability to deal within existing legacy constraints as they're slowly adapted, modified, and/or replaced. They also offer some long-term possibilities that will allow us to make radical changes in urban design. And yes, those changes are ones can accommodate rail, pedestrian-friendly city design, and other options that can drastically improve quality of life.

0. https://www.npr.org/2017/11/09/563016223/exploring-why-hurri...

1. https://finance.yahoo.com/news/pa-residents-living-above-min...

djsumdog2• 8 years ago

It didn't happen over night. The US had more passenger rail in the 1940s than Europe has now! Northern Indiana has the same population density of Scotland, so "too spread out" isn't the issue.

Cities only spread out over the past century. You start building rail again and it will get drawn back in. But the State has to fund that mode of transportation first.

triviatise• 8 years ago

in my area, our rail proposal was 1 billion for 9 miles of trail. Good luck putting everything on rails.

Ridesharing algorithms will 100% solve the problem of underutilised mass transit. Self driving isnt necessary.

Imagine shuttles that hold 10 people using uber pool algorithms and funded by the city.

People will be able to make their request and a nearby shuttle will pick all the people up in that area. The shuttles could do ad hoc transfers with other shuttles so people wouldnt even have to be going to the same area.

The algorithm would minimize the average ride time for everyone.

The #1 issue with mass transit is lack of convenience. Rideshare algorithms will eliminate that lack of convenience.

IgorPartola10• 8 years ago

Ride sharing works for young professionals who aren't moving around with kids. I grew up in a public transit city. I then got a car in the US. The car is a strictly better experience, and only marginally more expensive. Ride sharing can reduce the problem, but "100% solve the problem" is off by about 95 percentage points.

dang20• 8 years ago

We detached this subthread from https://news.ycombinator.com/item?id=16621507 since it has become its own separate discussion.

saurik• 8 years ago

There was an episode of Darkwing Duck (which shared a couple characters and I presume a lot of writers with DuckTales) that serves as an amazing parable for why vehicles are usually better off adapting to the environment than the other way around. The episode is called "Extinct Possibility", and might (or might not) be watchable at the following URL.

https://m.youtube.com/watch?v=uAqZ_jMmGAk

Steeeve• 8 years ago

Why not test these things on a private track? You have a multi-billion dollar company attacking this problem and it feels like all they tested before they hit the public roads was lefty and righty.

melicerte• 8 years ago

If you skip the emotional part of the news, the interesting question is: who is responsible when a self driving car made by a company kills someone?

blauditore• 8 years ago

Wow, this is sad.

I always imagine I would feel uncomfortable when developing software in certain areas such as this, or e.g. for space rockets containing humans. Errors have a much higher impact than when only developing web applications where the worst case scenario is a tempoarily unreachable website, or maybe some data loss.

ada1981• 8 years ago

Could this help Uber?

American's seem to enjoy the "X doesn't kill people. If only more people had X, less people would have to die." argument.

cwbrixey• 8 years ago

The problem is likely a failed ABS Brakes and on all cars. When the brake is applied and then it fails, there is less than 5 seconds to a crash and likely fatality. No time to react. Experiencing the brake failure on two cars, I have tried to get the attention of NHTSA for a decade. NHTSA has a lock on faulty research to blame an innocent driver for all accidents instead of finding the correct root cause of the crash. My estimate is 50% to 70% of all fatal crashes is caused by a defective ABS Brake.. Finally with no driver to blame, NHTSA will have to solve the correct fail mode for these crashes. The Congressional Sudden Acceleration investigation found nothing to make the roads safer and Toyotas have SA crashes today and the certified accident investigator only logs items to blame the innocent driver. Unless the braking technology is totally changed in self driving cars, I expect daily crashes with no one to blame. I have sent Elon Musk letters that the Tesla has ABS Brake issues causing crashes, but their engineers join in blaming the driver and do not fix the problem Accident investigators are clueless Lawyers and DAs are clueless News Reporters are clueless NHTSA is clueless NTSB is clueless Carnage on the roads goes on unabated and we convict an innocent driver.

downandout3• 8 years ago

The title here should be amended to “...hits, kills jaywalking pedestrian...”. Let’s get the facts straight. This person was walking outside the crosswalk. The fact that a computer was behind the wheel is irrelevant. A human broke a law designed to prevent this exact scenario, and the result was both predictable and tragic. It’s terrible, but ultimately the person’s fault.

danso30• 8 years ago

It's "predictable" that jaywalking will lead to death? That a killed/injured pedestrian was jaywalking can be used in defending the motorist in court. But the vehicle code and road infrastructure does not assume that `jaywalking == death sentence`. Drivers deal with jaywalking every single day -- if death was to be expected, driving and jaywalking laws and regulations would be much, much different than they are now.

The fact that a computer was behind the wheel is the most important factor in this discussion, because the argument for self-driving vehicles is that they, unlike humans, can react to abnormal situations such as jaywalking.

downandout3• 8 years ago

It’s predictable that when you (illegally) put yourself in harm’s way, you might be harmed or even killed. This was a terrible outcome, but it’s no more the fault of the computer that was driving the car than it would have been if it were a human driver. If I go walking into traffic, as a logical person, I fully expect to be hit and injured or killed. The most important laws where jaywalking is concerned are not traffic laws, but rather the laws of physics.

Thankfully this is not the outcome of each and every jaywalking incident, but it is a possibility with all of them. That’s the risk you consciously accept when you walk in front of moving vehicles that weigh thousands of pounds.

mo1ok2• 8 years ago

The law is that pedestrians always have right of way. Jaywalking is a fine, but not a jail sentence. Not coming to a full stop for jaywalking pedestrians, however, is how I failed my first driver's test.

downandout3• 8 years ago

The law is that pedestrians always have right of way

That actually depends on the state. “Arizona law requires pedestrians to cross the street at corners and/or designated crosswalks...[if] it seems unlikely that a driver should have avoided---as in a case where the pedestrian darts right out into traffic, then it may be that the pedestrian will be deemed 100% at fault for his/her injuries.” [1]

I’m sure that Uber will be sued over this, and they’ll probably settle quickly and quietly to avoid mass hysteria resulting from misleading headlines such as this one. But that doesn’t mean the jaywalking pedestrian isn’t at fault. Walk in traffic, die in traffic. It’s a pretty straightforward physics problem.

[1] http://www.zacharlawblog.com/2015/11/whos-at-fault-in-an-acc...

marcell• 8 years ago

Forgive my ignorance, but isn't this not supposed to happen with LIDAR equipped cars?

My understanding is that a LIDAR provides a completely accurate 3D map of the surrounding area around a self driving car. This is in contrast to Tesla's image recognition approach which makes a 3D map from 2D images. So is seems like a pretty giant bug if a self driving car ever crashes into another object with LIDAR, since it should always know the location of external objects.

Please correct me if I am wrong!

grmarcil• 8 years ago

You're not wrong, but it is more complicated than that. Lidar gives you an array of 3D points and intensity, corresponding to where the lasers bounced back, and how strong the reflection was. Roughly speaking, from there you have to decide 1) which sets of points belong to the same object, 2) what those objects are, and 3) what those objects intend to do in the future.

So yes, a lidar-equipped AV completely not sensing a pedestrian would be surprising, but you can see how it might have incorrectly classified the pedestrian, or misunderstood the pedestrian's intent.

wiz21c5• 8 years ago

I once read somewhere that privacy issues (which are very important to us, IT people) were not treated enough because they didn't have enough blood on their hands.

At least with automated cars, there will be blood.

thecombjelly• 8 years ago

This seems like a good time to question if we really should have cars at all. Autonomous cars might be better in some ways but they are still tons of metal flying at high rates of speed. Accidents will happen and people and other animals will die on a regular basis. Cars also require tons of infrastructure that walking and biking don't. You also have the environmental destruction of GHG emissions and from the mining of rare metals needed for batteries, you have roads and parking lots destroying habitat and using up valuable space, air pollution and smog from tailpipes or energy generation, noise pollution, and more harmful effects. Autonomous cars might mitigate some of those things but most of it will still be there (or we'll just use cars even more and so we'll have just as many deaths total along with the rest of it?). What about doing research and testing a world without any cars? It will probably be a better one.

Nomentatus• 8 years ago

Paris has greatly reduced traffic, thinking along these lines.

Zigurd• 8 years ago

The article says almost nothing about it other than that the pedestrian was "outside the crosswalk." But the details are what matters. Without the details it is not possible to say if striking a pedestrian was avoidable. There is also a large grey area: Could the collision been mitigated? What other risks were involved?

The bad outcome would be that it turns out that pedestrian safety is too underdeveloped in this system to really be safe.

ptr_void• 8 years ago

My non-expert opinion is that we will not see self driving cars become reality in our lifetime. I don't understand why anyone believes training on some videos/pictures to calculate some probabilities would ever be enough to handle all the complexities of the real world.

lettergram• 8 years ago

Given I've seen (just last year) an Uber self driving car run a red light in SF, nealy hitting several people - this is not at all supprising.

I do think Uber should be charged if the car is found at fault. Not a civil case, criminal. We know machine learning algorithms fail all the time (like this). It shouldn't be alright to just ignore this.

darkstar999• 8 years ago

Not a civil case, criminal.

Has there ever been a case where a car manufacturer is charged with a crime for a defect that results in a death?

linkregister4• 8 years ago

I don’t know about cars, but there is precedent for company officers for being criminally liable for negligent homocide. Even in non-homocide cases, individuals sometimes face criminal charges, as the Volkswagen engineer behind the emissions cheating scandal found out.

wiz21c5• 8 years ago

There are roughly 3e8 cars in the US, they kill 3e4 people a year => each car kills 1e-4 person

otalp• 8 years ago

Very Relevant video on what the discussions about 'the ethics of self driving cars' miss: https://www.youtube.com/watch?v=ozcaLnTuidU

Summary: We spend too much time talking about what decision the car should make in these situations, not enough on how allowing a corporation to make those life and death decisions changes our society.

wiz21c5• 8 years ago

There are roughly 3e8 cars in the US, they kill 3e4 people a year => each car kills 1e-4 person a year. If we have say 100 AV right now in US, each car kills 1e-2 person a year. So that's 100 times more dangerous than humans...

Aunche2• 8 years ago

That's not a good comparison because self driving cars are on the road a lot more often than regular cars.

wiz21c5• 8 years ago

yep, sure. I did that computation to have a rough estimate. Also, with on or two fatalities, I guess the confidence in the stats is not very high.

osrec2• 8 years ago

I really don't understand the need to put a self driving car around pedestrians or other human drivers. As with any machine learning/control system, wild instabilities can and will crop up. Why don't we simplify the problem domain and keep well suited, designated infrastructure just for self driving vehicles, and solve the last mile problem with a slow 15mph creep to the ultimate destination. I mean, it's impressive that cars can drive themselves through a busy intersection alongside other human drivers, but it is by no means necessary, especially when adding humans into the mix adds a degree of uncertainty that is very difficult to account for...

nkrisc3• 8 years ago

And now the costs of developing autonomous cars has been socialized to those unfortunate enough to be walking in front of them while they learn.

Perhaps her estate should receive a perpetual royalty.

schintan• 8 years ago

I think the test cars should be covered in layers upon layers of foam or another kind of shock absorbing material. This should reduce the collision impact at least a little and can be the difference between someone getting killed and escaping with injuries in some cases.

justonepost• 8 years ago

There were an estimated 40,100 motor vehicle deaths last year, or a drop of 1 percent from the prior year.

waraey• 8 years ago

Ubye

basura0454785• 8 years ago

I see a lot of detailed analysis in this thread, which is impressive given that literally nobody has read anything more than the half dozen factoids that have been released. At best, some of you are looking at maps of the incident site to figure out potential failure cases.

Knowing that you don't know something is as important as having the facts. To harken back to Cheney, these aren't even unknown unknowns, you're basing opinions on known unknowns.

danso30• 8 years ago

Cheney? I think you mean Don Rumsfeld: https://www.youtube.com/watch?v=GiPe1OiKQuk

basura0454785• 8 years ago

you got me

kumarski• 8 years ago

"Uber is now the target of at least three potential class action lawsuits, at least five state attorney general investigations, and an inquiry by the FTC because of Sullivan’s decision to pay off hackers and the cover ups."

via twitter.com/adamscrabble

smsm423• 8 years ago

I think it would be insane to expect 100% safety from any technology, let alone new technology. People do crazy things on the road, I am witnessing it almost daily. And if you look up dashcam videos on the internet, you'll be scared to ever drive again. Expecting that some magic technology can deal with all this craziness with 100% safety is impossible. And then we'll add bugs and malfunctions, which absolutely every technology has, especially - new one.

So, brace for the incoming wave of similar news. These things will happen, and these things will be heavily publicized. Hopefully it won't cause stupid overreaction to the tune of banning the technology altogether or surrounding it with a bubble of regulations so think as to make in infeasible.

jostmey• 8 years ago

Both the computer and the driver failed to avoid the pedestrian. It's possible that the backup driver wasn't paying attention and the technology for autonomous vehicles is not ready. It's also possible that the situation was challenging for both the computer and human and that both failed.

hackpert• 8 years ago

While incredibly sad, this incident is not terribly surprising because the technology is nowhere near safe enough.

Unfortunately, this will inevitably set the field back by at least 10 years with stricter regulations. We've seen the same with pharma and aviation, where innovation was slowed drastically and is only realistic for big companies. This is especially true here because if self-driving vehicles get banned from public roads, companies like Waymo will be forced to develop bigger private "test tracks" that approximate cities to have any hope of building deployable technology, and they will face the same fundamental limitation – the data will come from an approximation/model and not the actual location with real-life scenarios.

Although there's no doubt that regulations are crucial to prevent such mishaps, I do hope lawmakers can find the right equilibrium on the tradeoff between safety and innovation.

cbhl• 8 years ago

The last time I visited Phoenix, I recall being shocked at far apart traffic lights were in the city. Between the Wal-marts and the giant four-to-six bedroom homes, it might be normal to see a major street have a mile between traffic lights (and presumably, crosswalks). It was very different from, say, driving in New York, San Francisco or Seattle.

I'd be curious whether "walking outside of the crosswalk" turns out to be material in the analysis by NHTSA/NTSB. Were the crosswalks big enough and close enough that a pedestrian would have normally walked inside one? Was the interchange a regular 90-degree crossing of two roads, or was it something more irregularly shaped? What is the annual rate at which pedestrians get hit at this intersection by human drivers? Do self-driving cars need more examples of "pedestrians walking outside the crosswalk" in their training sets?

robterrell2• 8 years ago

I hope they aren't expecting people to only appear at intersections:

"The California Vehicle Code says you can actually cross any street as long as you aren't a hazard to vehicles. It is also legal to cross mid-block when you're not between two intersections with signals." [1]

The article points out that LA has a different rule. Uber's going to need to geofence locations and use different rulesets for different cities.

[1] https://www.scpr.org/news/2015/04/14/50992/how-do-you-cross-...

manmal3• 8 years ago

Sure, but it’s illegal in most countries (AFAIK also in the US) to not use a crosswalk when you are in the vicinity of one. Eg in Austria you will be fined (or worse, drivers license suspended) if you don’t use a crosswalk that is within 50m of you.

danso30• 8 years ago

Seems like the law is more vehicle-friendly in AZ:

https://www.azcentral.com/story/news/local/glendale/2014/09/...

According to Arizona law, pedestrians are supposed to cross within marked crosswalks, or at unmarked crosswalks at intersections. An unmarked crosswalk is the location where two roadways intersect but no marked crosswalk is present, for instance in a residential neighborhood. Jaywalking is technically crossing "between adjacent intersections at which traffic-control signals are in operation." In this instance, pedestrians shall not cross at any place except in a marked crosswalk.

But why would state-by-state (or city) geofencing be needed for this situation? Seems like pedestrian safety would be prioritized regardless of whether the pedestrian seems to be committing jaywalking. Is there a situation in which the AI shouldn't attempt to brake if a large object appears in the street in front of it? And how big of a factor would local laws play?

lsh123• 8 years ago

"Uber's going to need to geofence locations and use different rulesets for different cities."

May be not. Regardless of the street crossing law in a specific city, it is a bad idea to kill pedestrians.

dang20• 8 years ago

We changed the URL from https://www.wcpo.com/news/arizona-police-investigating-self-... to one that is longer and doesn't autoplay a video.

jordanpg• 8 years ago

Those who say it's not safe enough: given that no engineered system is 100% safe, when/how can you say that a system is safe enough?

pbhjpbhj2• 8 years ago

Had the car passed the driving test?

There should probably be a specific, complex and comprehensive test for autonomous vehicles. Also I'd want to see shared liability, the company are driving it (by proxy through their software) so they should be liable to some extent.

drstewart• 8 years ago

There should probably be a specific, complex and comprehensive test for autonomous vehicles

Maybe we should start with a specific, complex and comprehensive test for human drivers first.

And to build on the sentiment of this thread, every driver should finance their own test course to practice on before "subsidizing" their learning on public roads.

pbhjpbhj2• 8 years ago

In the UK, for motorbike tests one does a computer based test, then practices with a company on a private off-road test area. One must pass a test then before being able to practices under instruction on public roads. Once sufficiently practiced one can take the test.

Basically what you propose?

As for comprehensive testing - I think the UK car and motorbike tests are quite good, not comprehensive, but they demonstrate a general ability across a range of skills. Humans can be expected to act relatively consistently (in such things), the test must be more comprehensive when treating a system that you can't expect to have innate consistency.

t0mbstone2• 8 years ago

Based on landmarks from the photo from the news article, and the fact that the accident supposedly happened near Mill and Curry, I'm assuming that this was the location of the accident:

https://www.google.com/maps/place/N+Mill+Ave+%26+E+Curry+Rd,...

justaaron2• 8 years ago

I never understood the recent push towards self-driving cars, as it already seems perverse that the entire surface pavement has been dedicated primarily to motor vehicles and only secondarily to our actual selves, as pedestrians.

Putting further pressure on what should really be 2 discrete planes of activity (perhaps one in tunnels below ground or in the air, and I don't mean the pedestrians!)

Perhaps now the arrogance of such bad ideas becomes a little bit more clear to the tech set of folks, that in my view were excessively pushing such strange goals upon a public that wasn't clamoring for it.

sg0• 8 years ago

I'm paranoid when I'm on a bike. Any road with a dedicated bike lane may be classified as bike-friendly, but I can't consider it so if cars are on the same road. IMO, the only bike-friendly routes are where there are no cars. I wish pavements could be made wide enough for bikes everywhere, so that bikers could be on the road only for a limited amount of time while crossing intersections.

amclennon• 8 years ago

Elaine Herzberg, 49, was walking outside the crosswalk on a four-lane road in the Phoenix suburb of Tempe

Although I agree with your statement in general, it sounds like she was already walking her bicycle while crossing the intersection, and the fact that she had a bicycle was a moot point.

cf498• 8 years ago

Looking further then the devastating tragedy at hand, it might be necessary to get the question of responsibility publicly answered. I dont think I am the only one here who has deliberately chosen not to work in fields where human lifes are at risk.

Despite the likely outcome, that the engineers behind this took every safety percussions, I think its extremely important, to get the message across, that people with an CS background with such use cases are going to be held to the same standard as for example mechanical and electrical engineers.

icc9711• 8 years ago

One aspect that comes from this is that now car crashes can be treated more like aircraft crashes. Each self-driving car now has a black box in it with a ton of telemetary.

So it's not just "don't drink and drive", knowing that they'll probably reoffend soon anyway. Every crash and especially fatality can be thoroughly investigated and should be prevented from ever happening again.

Hopefully there's enough data in the investigation so that Tesla / Waymo and all other car companies can include the circumstances of the failure in their tests.

Maxious• 8 years ago

NTSB are investigating this incident including electronic recorders https://www.ntsb.gov/investigations/Pages/HWY18FH010.aspx and they have another investigation from 2017 still to report https://www.ntsb.gov/investigations/Pages/HWY18FH001.aspx

fudged713• 8 years ago

Although it's comforting that this exact situation shouldn't happen again in an Uber autonomous car... there is no mechanism to share that learning with the other car companies. There seriously fucking needs to be a consortium for exactly this purpose: sharing system failures.

Also my problem with this is that a human death is functionally treated as finding edge cases that are missing a unit test, and progressing the testing rate of the code... and that really bothers me somehow. We need to avoid treating deaths as progress in the pursuit of better things

piinbinary• 8 years ago

That raises an interesting question: Should there be the equivalent of the FAA for self-driving cars? (Perhaps this could be a function of DoT.)

emodendroket14• 8 years ago

Well, for one thing, I think I have misgivings in part because Uber hasn't really demonstrated an attitude that makes me think they'll be very careful to reduce accidents. (Also, the extent to which they've bet the farm on autonomous driving presents a lot of temptation to cut corners)

daenz2• 8 years ago

There seriously fucking needs to be a consortium for exactly this purpose: sharing system failures.

This implies they're using the same systems and underlying models. If one model hit a pedestrian because of a weakness in training data plus a sub-optimal model hyperparameter, and therefore was classified a pedestrian in that specific pose as some trash on the street, how do you share that conclusion with other companies models?

zipwitch• 8 years ago

It makes me uncomfortable too. I think it's because it's a real world trolley problem.

We all make similar decisions all of our lives, but nearly always at some remove. (With rare exceptions in fields like medicine and military operations.) But autonomous vehicles are a very visceral and direct implementation. The difference betweeen the trolley problem and in autonomous vehicles is in time delay and the amount of effort and skill required in execution.

Plus, we're taking what is pretty clearly a moral decision and putting it into a system that doesn't do moral decision-making.

pj_mukh• 8 years ago

Anyone know how this is handled between Boeing and Airbus? Can we mandate the same mechanism between Uber and Waymo?

mindslight2• 8 years ago

We need to avoid treating deaths as progress in the pursuit of better things

Au contraire. Go read building codes some time. There's a saying that they're "written in blood" - every bit, no matter how obvious or arbitrary seeming, was earned through some real-world failure.

The death itself isn't progress, of course. But we owe it to the person to who died to learn from what happened.

goshx2• 8 years ago

Thinking about different scenarios as unit tests it shouldn’t be hard for them to simulate all sorts of different scenarios and share those tests. Perhaps that would become part of a new standard for safety measures in addition to crash tests with dummies. In fact, I really think this will become the norm in the near future. It might even be out there already in some form.

ekianjo3• 8 years ago

We need to avoid treating deaths as progress in the pursuit of better things

Then by all means lets stay at home and avoid putting humans in rockets ever again because if you think space exploration will be done without deaths you are in for a surprise.

telchar• 8 years ago

With Uber's reputation, I wouldn't be surprised if they try to write an app to falsify black box telemetry in the event of a crash to put the liability on the victim. Maybe they'll call it "grey box" or "black mirror".

Does the NTSB have regulation on how black boxes are allowed to function?

em3rgent0rdr• 8 years ago

"Every crash and especially fatality can be thoroughly investigated"

Would be better if the code and data was open for public review.

jryle702• 8 years ago

That's knee-jerk reaction that may open a can of worm. Do you need personal details of the victim as well as the driver? Says if the victim had attempted suicide before? at the same crossroad? Or the driver had history of depression? Would that be a violation of their privacy? Would that cause a witch hunt?

baron8162• 8 years ago

I hope we see the day when every car crash makes national news and there’s a big NTSB investigation into what happened.

dotancohen4• 8 years ago

This is by far the most insightful comment in the entire thread.

The real news isn't "Self-Driving Car Kills Pedestrian", the real news is "Self-Driving Car Fatalities are Rare Enough to be World News". I'm one-third a class M planet away from the incident and reading about it.

icc9711• 8 years ago

I'm hijacking my own post, but this is a very relevant MIT lecture on 'Technology, Policy and Vehicle Safety in the Age of AI' [0].

[0] https://www.youtube.com/watch?v=LDprUza7yT4

more_corn3• 8 years ago

Every lesson from aviation is earned in blood. This death wasn't necessary though. The Otto/Uber guys have been informed about their car's difficulty sensing and stopping for pedestrians. I know this because I informed them myself when one almost ran me down in a crosswalk in SF. Can't learn anything from your lessons unless you listen. Maybe they can pause and figure out how to actually listen to their reports of unsafe vehicle behavior.

woofyman• 8 years ago

Police are reporting that it never slowed down. Hitting the pedestrian at 40 mph.

cityofghosts• 8 years ago

this comment section should be read in front of congress the next time regulating the tech industry is in the table. these people literally think its ok to perform experiments that kill people.

skookumchuck6• 8 years ago

and should be prevented from ever happening again.

Take a look at the cost of airplanes, even small ones.

BucketSort• 8 years ago

Black box with a ton of telemetry being piped into black box models though.

a_d• 8 years ago

https://news.ycombinator.com/item?id=16547215

This comment by Animats in the context of self-driving trucks is quite telling. He warns precisely of this danger.

qwerty456127• 8 years ago

Just out of curiosity, how many people have been killed by human-driven cars on the same day in Arizona?

WhompingWindows2• 8 years ago

My guess is 2 or 3 deaths that day. Okay, according to the 2016 Arizona Crash Facts Summary [0], there were 952 fatalities on the road that year, including 193 pedestrian fatalities and 31 pedalcyclist fatalities.

So, on any given day in 2016, there were 2.6 fatalities, with less than 1 in the pedestrian/pedalcyclist groups.

[0] https://www.azdot.gov/docs/default-source/mvd-services/2016-...

undoware• 8 years ago

Headline should read 'woman crossing street kills electric cars'

nwellinghoff• 8 years ago

I love how everyone is throwing fuel on the fire here with no actual details as to what happened in the accident. It could simply be she ran out into oncoming traffic and got hit. Hard to say a human driver could have performed any better. The devil is in the details.

CardenB• 8 years ago

Agreed. We need a lot of time to digest this. At least anything before the NTSB investigation is jumping to conclusions.

SuperNinKenDo• 8 years ago

Given the number of fatal pedestrian hits by human drivers, I'd be interested to see how Uber's self driving cars stack up against the apes.

Having said that, one concern one has with self-driving cars is that one can't "reach an understanding" with them the same way pedestrians and drivers regularly negotiate how they're going to behave with each other.

If one doesn't know one is dealing with a self-driving car, one might consider certain situations safe, because they usually would be, but the car negotiates and notices things differently than a human.

dchichkov• 8 years ago

And yet (disclaimer - reflects only my personal view, might not correspond to the current state of affairs at Uber, it is possible that Uber cars are still safer than human drivers, but still - one who forgets the history tend to repeat it):

https://www.forbes.com/sites/samabuelsamid/2016/12/15/the-tr...

AsyncAwait• 8 years ago

As someone who cannot drive because of a disability, I am putting real hope into self-driving cars becoming a thing during my lifetime. Uber may've just set that back a decade.

vpribish• 8 years ago

I suppose speculators will speculate - but c'mon. just wait and we'll have video and telemetry and this whole conversation will be moot.

gldev• 8 years ago

So, what's the statistic of actual self driving cars killing people vs the actual people killing others or themselves.

11thEarlOfMar2• 8 years ago

It's not hard to believe that self-driving autonomous cars will make the roads safer overall.

The issue is that while fewer lives will be lost, it will be a different set of lives than would have been lost without self-driving technology.

Those lives will have their day in court.

ensignavenger• 8 years ago

The various reports on this aren't clear- did the car hit a pedestrian, or a cyclist?

nimbs• 8 years ago

Tempe Police say woman killed by self-driving Uber car was pushing bicycle across street when struck.

Peter-W• 8 years ago

She stepped out in front of the car in the middle of the road, it's not like a human driver would have done any better.

zengid• 8 years ago

Did the person riding along attempt to take control? I wonder if they will get held responsible..

arca_vorago• 8 years ago

Uber is responsible for this, and corporations are people, right? That reminds me I have an episode of opening arguments to listen to on the subject.

xbeta3• 8 years ago

Statistically speaking, I am curious to know the data on these areas: - number of miles driven per fatal accident on self-driving - number of miles driven per fatal accident on a human driver.

It is important to know whether our machine is now doing worse/better than our counter-part and this means a lot for our self-driving initiatives.

fnbr• 8 years ago

The numbers that I have seen indicate that humans have a fatal accident roughly once per hundred million miles, while we have one fatal accident for a self-driving car, with somewhere around ten million miles driven across all self-driving cars.

I've heard Uber is around 3 million self-driving miles.

So Uber would be 30x worse than humans.

emodendroket14• 8 years ago

Interesting analysis, but the data seems a little sparse to draw this conclusion.

xbeta3• 8 years ago

Good to know! But I am more curious for the data cover every self-driving vehicles, not just about Uber. It could be Uber is doing worse, but my ask is whether we (as an industry) is still on a right direction for this "bold bet".

plaguuuuuu• 8 years ago

We also don't know how many fatal crashes were avoided due to human intervention (Uber has people inside the cars). It is almost certainly a whole lot worse than 30x at this stage.

jakelazaroff3• 8 years ago

1.15 per hundred million vehicle miles traveled by humans [1]. Uber only reached a million miles in September, with Waymo a little ahead at four million [2].

Note that this is far too little data from which to draw any conclusions, since the sample size for fatalities involving self driving cars is one and the number of miles driven is many orders of magnitude less.

[1] http://www.iihs.org/iihs/topics/t/general-statistics/fatalit...

[2] https://www.theverge.com/2017/11/28/16709104/waymo-self-driv...

_xnmw4• 8 years ago

My personal hunch is that fully autonomous self-driving tech is not theoretically possible under currently known computational models, because it implies many well-known NP-Hard problems. Self-driving companies are betting on the ability to find a heuristic/approximation that works "sufficiently well". But I strongly feel that the chasm that needs to be crossed to be "sufficiently good" is not one of magnitude (i.e. we just need more testing!), but of theoretical boundaries, due to the existence of at-least two sub-problems which are not computationally solvable: 1. prediction of what pedestrians/cyclists will do next, and 2. accounting for sensor input distortion under bad weather conditions.

Humans can solve these problems due to life experience, not just driving experience. In other words, I think we're gonna need fully-conscious AI to solve self-driving.

The only way self-driving tech will reach production is if the input space is restricted, which is a significant-but-not-groundbreaking iteration on what we've been doing for decades with airplane autopilots and self-driving monorails. Sure, we can have self-driving cars on specifically designed freeways, but nothing more.

emodendroket14• 8 years ago

Sure, we can have self-driving cars on specifically designed freeways, but nothing more.

But at this point what makes it better than a train?

pinum• 8 years ago

I'm not necessarily arguing for this, but one viewpoint might be: A vehicle that can autodrive the freeway (i.e. most of the way), and reverts to manual control for the shorter sections before and after the freeway, would have a significant convenience benefit over a train.

Take the train, and you have no vehicle with which to go from the destination station to your endpoint. Also, you could travel at any time instead of being held to the schedule.

tdonaghe• 8 years ago

What if "good enough" is just reduces deaths/injuries by 99%? 90%?

People would still get hurt and die directly due to software that cannot be perfected, but the gains to society as a whole might be worth it.

jjoonathan• 8 years ago

Aren't self-driving cars already doing much better than humans on the safety front? That's the standard to beat, not perfection.

yelloweyes2• 8 years ago

1. I seriously doubt self-driving cars will be viable in irregular roads

2. Who do you blame when a faulty algorithm eventually kills a person? Who do you get mad at? When a drunk driver kills your family member, you can go to court and look at their face and look at the faces of their family members. When that happens with a self-driving car you'll just be looking at corporate lawyers who will shrug their shoulders at you and say "lol sorry our dumb machine killed your daughter".

xbeta3• 8 years ago

I think the self-driving initiative is not about beating the human counterpart at launch, but at least on par with the performance done by human. That itself has huge value in efficiency and saving time.

Slowly with many AI-driven vehicles on the road, we can "optimized" for a better performance on safety and other issues.

koala_man• 8 years ago

Do you believe human-driven tech is theoretically possible?

Current experiments and production data from human controlled vehicles have not been encouraging.

_xnmw4• 8 years ago

I added an edit to my comment: We don't have a computational/theoretical model for human consciousness. This is why it's called The Hard Problem of Consciousness.

kev009• 8 years ago

That's basically what human driving is though, no? We consciously and unconsciously take our attention off the task all the time. It's not possible to drive fully alert of surroundings all the time and at least part of the time we are simply dead reckoning in fairly safe lanes of travel at constant speed and direction with minimal pedestrian and cross traffic. I could call it "controlled chaos", "luck", "planning" but there is some amount of unknown when moving a multi-thousand pound object around, and as speed increases so do the chances, ability to optimally correct, and severity of mistakes. It is very interesting and challenging to map the morality of the decisions of moving these machines onto automation.

tomdre• 8 years ago

This is not a really strong argument against self-driving cars. The fact that a problem is NP-hard doesn't make it untractable. Every day we use apps that deal with NP problems (e.g., routing problems, packing problems, etc.). Also, note that there're P problems whose instances can be harder than (smaller) NP ones.

gugagore• 8 years ago

There's problem classes and problem instances.

What does NP-hardness look like for self-driving car tech? non-deterministic polynomial in: number of objects? number of lanes? time steps in the planning horizon? action/observation branching factor? These things are bounded in practice.

Not saying that the computational problems aren't hard. But ending the conversation at "NP Hard" throws away too many nuances.

_xnmw4• 8 years ago

NP in the number of data points received from all its sensors.

jarjoura2• 8 years ago

It's fascinating that so many brilliant people in this thread can talk so casually about any single loss of life for the tech built by the very same people in this thread. I would be devastated if I learned my code ended even one person's life.

You're in a bubble if you think a world hyper connected to social media that's in the same world scared that autonomy is about to kill millions of jobs in one jab and would NOT over-react to a single accident.

From the article it's impossible to know if the cyclist was at fault, so I won't jump to conclusions, but we can't, especially as the creators of the future of autonomous vehicles talk so dissociatively about human life. Either we go out into the public roads with very high certainty that OUR software and OUR hardware won't kill someone, or we wait till we get there.

Public opinion on human driving is doing just fine with human drivers right now, so we can and should take our time to get to the future we want or risk a backlash and a guilty conscious.

habosa• 8 years ago

Thank you for adding this perspective. Honestly I am not sure how to rationalize the dueling thoughts in my head.

On one hand, the "just one isn't so bad" argument is rational. In terms of fatalities per mile, self driving cars already seem to be safer than humans as a class. Therefore it would be wrong of me to say that this tech does not belong on the road when I believe I should be allowed to drive.

On the other hand, you said it best. If I ever wrote code running on a machine that killed somebody I don't know how I'd continue in my career. Hell, I work at Google very far from Waymo but if one of our cars struck and killed someone I'd feel some personal responsibility.

So ... I guess it's complicated.

larkeith• 8 years ago

Public opinion on human driving is doing just fine with human drivers right now

There were over 40,000 motor vehicle deaths in 2016 in the US [1] - more than a hundred daily. While loss of human life is always a tragedy, individual fatalities are absolutely worth it if we can begin to reduce that count sooner than by proceeding with excessive caution.

It would be morally unacceptable to delay development merely to avoid a guilty conscience.

[1] http://www.nsc.org/NewsDocuments/2017/Fatality-estimates-Jun...

jonathanyc19• 8 years ago

And it’d be morally unacceptable to allow public deployment of software that has not been sufficiently tested. As other comments have pointed out, self-driving cars have undergone ridiculously little testing. In fact, based on only the objective statistics it is very unlikely that they are anywhere near as good at driving as humans.

There is no reason self-driving cars can’t be tested in private. Companies can hire pedestrians to interact with the cars, and the software can go through the same certification process that buildings and vehicles currently go through.

It’s a false dichotomy to say that you can either have self driving cars or minimally safe and accountable development, but not both.

EDIT: Here is a link to a thread replying to a parent which is now auto-collapsed. https://news.ycombinator.com/item?id=16620968

jeffdavis2• 8 years ago

Why are you so sure? Such absolute certainty -- certain enough to dismiss the loss of life for the greater good -- is the cause of so many bad things.

I'd like to believe this technology will save 20000 lives per year. But today is a time for humility and reflection.

jarjoura2• 8 years ago

To clarify, no where in my comment did I say we should stop development for making our roads safer. Consumer accessible cars are already far more aware of the surroundings today and will continue to gain more awareness as time goes on.

If you think rushing to market, in some self-inflicted arms race for fully autonomous vehicles is morally acceptable to you, AND human lives are lost in the process, then I find that fascinating.

felippee2• 8 years ago

The number you pull out is not telling. There are billions of miles driven in the US alone and people get killed on average after 100 mil miles driven. That is where the bar is, and I can tell you, that is a high bar. Even including all the idiots, teenagers, DUI's etc. it is still one fatality on 100 mln miles. So sure, it is a tragedy that those 40.000 people die, but that is not a simple problem to fix. In fact obesity and sugar caused disease is probably a much simpler problem to solve, that could likely save way more people.

castis• 8 years ago

It's morally acceptable to violently end the lives of bystanders if its in the name of progress. Got it.

rptv• 8 years ago

Is it worth it if we exchange 3 drunk drivers' lives for 1 bystander's? Can we really just use a single measure here?

Karrot_Kream2• 8 years ago

https://mobile.twitter.com/BenRossTransit/status/97579157290... says:

US has 1 traffic fatality per 85 million miles driven. Uber's self-driving cars have driven ~3 million miles & killed one person
While loss of human life is always a tragedy, individual fatalities are absolutely worth it if we can begin to reduce that count sooner than by proceeding with excessive caution.

Wait, when did I agree to be a utilitarian with you? Did I vote utilitarianism into power?

InclinedPlane2• 8 years ago

Currently, Uber self-driving vehicles are clocking in at a fatality rate of about 50x the rate for human drivers.

You can bring your hypotheticals to bear all you like, but this is a serious problem.

DoreenMichele3• 8 years ago

Generally speaking, it is easier to pinpoint X thing led to Y tragedy than to quantify the tragedies that would have happened without X. It is incredibly hard to show what did not happen but "should" have. Even if you can get it straight in your own mind, good luck proving it to the world.

People are living longer and child mortality is down. We don't really celebrate that. We just complain about overpopulation and how we are destroying the environment.

When Iraq lit oil wells on fire on its way out of Kuwait, they were expected to burn for months and be a global environmental disaster. When crack teams from around the world converged and put them out far faster than expected, it was not celebrated with the same degree of fervor that it had originally been decried as a disaster.

Y2K was also predicted to be a global catastrophe. It was quietly prevented and gets remembered as "Those fools who ever thought this was a big deal!"

If self driving cars could reduce mortality compared to current rates and we were confident of that, waiting until it is perfected before releasing it means accepting ongoing deaths that don't have to happen. It's not uncommon for people to feel that new tech needs to be perfected while missing the fact that it may be a big improvement over the current status quo without being perfected.

I don't really know anything about self driving cars. I have mixed feelings about replying to your comment because I don't actually know how I feel about this particular issue. But the thought process you are putting forth is both common and not very pragmatic.

I guess a pithy rebuttal would quote some saying about the perfect being the enemy of the good. But I think that leaves out a lot of important points.

CooIHandLuke• 8 years ago

What we can't do as the creators of the future of autonomous vehicles is give in for emotional appeals when the relevant thing is statistics about car accidents. If the media does it, bash the media (and hope for decent publications to support the appeal of relevant people in the field). If politicians do it, bash politicians, though nothing really helps when it comes to politics. But don't be nice to bureaucrats and their oblivious supporters who are actually preventing thousands of lives being saved every year by slowing down development and deployment.

rhizome3• 8 years ago

As someone said on Twitter, among software developers, "ethics" is seen as a hobby at best.

nothrabannosir• 8 years ago

While pithy and edgy, this attitude is, unfortunately, also unconstructive and uninsightful. For those out there who do wish to put their money where their mouth is, do you have any advice besides “don’t work at Facebook”? (Or Uber…) Because that doesn’t seem to be cutting it.

At least doctors have the Hippocratic oath.

mipmap04• 8 years ago

My career is in software, but when I peer out of my cave at those in more traditional engineering backgrounds, I realize they have so many more checkpoints and structures for guaranteeing performance / operability under load. For 99% of us in software, if our software fails, nothing really happens of consequence other than downtime - no one dies. In this case, the software written can have mortal consequences. Maybe there's a need for increased scrutiny on software development - especially as IoT gains more popularity and people are relying more and more on the software that supports or replaces human involvement in critical and/or potentially high risk activities.

Raises the question though: will stakeholders be willing to pay the upfront financial costs to obtain the level of quality and assurance that other engineering disciplines provide? We'll see.

gus_massa• 8 years ago

I would be devastated if I learned my code ended even one person's life.

I agree, but there is already a lot of software in a car that can kill people in case of an error. In particular the airbag have multiple sensors and a complex heuristic to decide when the have to open, which ones, full or partial strength, ... https://en.wikipedia.org/wiki/Airbag#Operation An airbag that doesn't open can kill someone. An airbag that open unnecessary can kill someone. From Wikipedia:

From 1990 to 2000, the United States National Highway Traffic Safety Administration identified 175 fatalities caused by air bags. Most of these (104) have been children, while the rest were adults. About 3.3 million air bag deployments have occurred during that interval, and the agency estimates more than 6,377 lives saved and countless injuries prevented.

So, there is already a lot of programmers that have to be careful because an error can kill someone in the car (or the plane, or a medical machine, ...) I guess it's more easy to ignore the people killed by airbags because we are use to them and we imagine that they have a simple sensor (something like a light switch. So in case of a failure nobody is guilty.

With a self driving car, the first problem is that it is new and scary, and the second problem is that in an error that creates an accident we are use to blame the driver. But now we don't have anyone to blame (perhaps the programmer).

tooop• 8 years ago

Crossing a 4 lane road (where speed limit is at least 40mph) outside crosswalk? It's clear that it is cyclists fault. The only thing that is questionable - would a human driver have avoided this or at least hit the cyclist with lesser speed.

neotek• 8 years ago

Either we go out into the public roads with very high certainty that OUR software and OUR hardware won't kill someone, or we wait till we get there.

As long as it kills fewer people than would otherwise have died due to negligent human drivers I'd say it's already there.

trhway2• 8 years ago

https://www.abc15.com/news/region-southeast-valley/tempe/tem...

"The fleet of self-driving Volvos arrived in Arizona after they were banned from California roads over safety concerns."

arnaetra2• 8 years ago

I have not been following the development of self-driving car tech very closely, but I have some familiarity with the difficulty of the challenges involved and I have a feeling that we are at least two decades away from having fully autonomous tech authorized for use on public roads. Am I underestimating the progress of the tech? I have the impression that there is a tremendous amount of unjustified hype in this field.

0xffff24• 8 years ago

You're not underestimating anything. Rather, the vast majority of developers (much less everyone else) have no familiarity at all with the challenges that self-driving tech has to overcome.

lacerta• 8 years ago

It is both an over and underhyped field depending on the area you are looking at.

HUGE difference between consumer self driving car [Everywhere, at all times] and the Machines as a Service [Geo, time, weather-fenced operations][1].

Waymo appears to be at the head of the pack; Sacha Arnoud, Director of Engineering for Waymo gave a talk a few weeks ago at MIT[2] and gives a good idea. We are about 5 years out until these start rolling out as MaaS (machines as a service). Probably 10+ years for level 4 highway operations for consumer models according to Frazzoli.

[1]Emilio Frazzoli, CTO of nuTonomy https://www.youtube.com/watch?v=dWSbItd0HEA [2]https://www.youtube.com/watch?v=LSX3qdy0dFg

redm• 8 years ago

It's terrible when anyone is killed in an automobile accident. The irony is in the promise of autonomous vehicles to prevent this very thing.

I've heard the general report, and I find it interesting that a supervisor was at the wheel, and they, as well as the car, failed to prevent the pedestrian from getting hit. I read they were not in a crosswalk and I'm sure the specific details of the accident matter greatly. It makes me wonder if a human driver with no autonomous system would have faired any better since someone paid to be alert and a car designed to be alert both failed to prevent the accident.

I'll reserve final judgment until more details are out.

mr_toad3• 8 years ago

It doesn’t matter how good the driver is, AI or human, if you unexpectedly walk right out in front of a car moving at 40mph it’s not going to be able to stop.

princeb• 8 years ago

now the question is why is a car (autonomous or not) driving at 40mph on a road where people can be expected to walk right out in front of a car?

would you drive 40 mph down a city street with trucks and buses parked head to tail on both sides of the road?

raphaelj• 8 years ago

To put this accident in perspective, Uber self-driving cars totaled about 2 to 3 millions miles, while the fatality rate on US roads is approximately 1.18 deaths per 100 millions miles [1].

[1] https://www.nhtsa.gov/press-releases/usdot-releases-2016-fat...

thegabez• 8 years ago

Knowing that makes this much more infuriating.

cjdaly• 8 years ago

that doesn't sound like a fair comparison. How many human interventions were there in the 2-3 million Uber miles?

goshx2• 8 years ago

That’s not apples to apples. How many cars account for the nhtsa number and how many for Uber’s program?

utopcell3• 8 years ago

I'd say that miles driven is a good normalizer.

In fact, NHTSA statistics includes miles driven under adverse conditions (rain, snow, etc) while I'd bet that this is not the case for Uber.

nealabq2• 8 years ago

Waymo has driven 2.7 billion miles in simulation [1]. Is that enough to give them some idea of how many deaths to expect?

[1] https://waymo.com/ontheroad/

mickronome• 8 years ago

It's impossible to know. The simulation needs to be very good with thousands and thousands of all the weird situations that can occur in the real world. Without knowing how sofisticated the simulation is and if they also are using generative algorithms to try to break the autonomous system you can't even ballpark it.

ejstronge• 8 years ago

The appropriate comparison would be to ask how many pedestrians were struck by a car and killed.

Considering that humans would likely slow down if they see a pedestrian - even if one appeared suddenly - this is even more disconcerting.

lev992• 8 years ago

It's more specific, but I care about all fatalities, not just car-pedestrian fatalities.

Rambunctious• 8 years ago

This Rand study looks at impact of lives saved and recommends that "Results Suggest That More Lives Will Be Saved the Sooner HAVs Are Deployed". Any mishap, while most unfortunate and tragic for everyone concerned, should not result in kneejerk reactions!

https://www.rand.org/pubs/research_reports/RR2150.html

lifeisstillgood• 8 years ago

There are only two paths from here - an escalation of the scramble to real world driving hidden under self-serving rationalisations, ("the bigger picture") or a step back to some industry self-regulation, where self-driving just does not happen till the sensors, the maps and the algorithms are much much much improved.

We know very little about the circumstances here, apart from the obvious tragedy, but given that we learnt from Tesla that you cannot rely on cameras alone, whatever is learnt from this must be shared amoung all the industry and serve as the baseline for future.

I would prefer an outright ban on driverless cars in real world settings (until perhaps each competing car can safely navigate the worst sensor data other companies can offer) but as a minimum we need a neutral clearing house similar to the airline industry.

There are on this forum enough industry insiders, plus a sprinkling of regulators that a global self-regulated path could easily be forged.

FlyingSideKick• 8 years ago

Let’s take the extreme example where a bug in a self driving semi truck leads to it mowing down 50 people on a busy sidewalk. How are the victims to get justice? In almost all incidents besides a human truck driver having a major health incident, that driver would go to jail but it seems that there will be little repercussions beyond fines for autonomous driving companies. I think once a mass fatality occurs people will scream for autonomous driving execs and programmers to be held liable.

lopmotr3• 8 years ago

The way I see it, by using the road, you're sort of giving up the right to revenge justice. It's a game of Russian roulette. People make mistakes and kill others or themselves no matter how careful they try to be. It's like asking where's the justice if you die from a heart attack or cancer? Those aren't crimes, they're bad luck.

aurizon• 8 years ago

Uber must have a full video of this - just in case. I can hardly think they did not have a full record of this, as well as all miles traveled? It may be in these 1366+ comments, but I have not read them all to see.

rboyd2• 8 years ago

FTA "Elaine Herzberg, 49, was walking her bicycle outside the crosswalk".

The article says "as soon as she walked into the lane of traffic she was struck".

The police sergeant also "said he believed Herzberg may have been homeless".

These three things may together indicate that even a human driver might have struck and killed the lady if she entered into the roadway unsafely. I'm not sure how everyone (especially on Twitter) can be jumping all over Uber yet until the investigation is complete.

Sad of course, either way, but not yet enough evidence to assume that the algorithm+driver combo was any less safe than a human driver alone.

JimboOmega2• 8 years ago

Finding any tidbits on what actually did happen has been very difficult. I get that the investigation is of course ongoing, but that's what I want to know - did the tech actually make a mistake here?

And a mistake in a broader sense - was it an avoidable accident? If it wasn't this barely feels like news, other than as a milestone.

Also... I don't see how her being homeless has anything to do with it.

rboyd2• 8 years ago

Also... I don't see how her being homeless has anything to do with it.

well, higher than average rates of substance abuse in the homeless population. other than that I'm not sure how it could be relevant or why the article mentioned it. we wouldn't know without an autopsy but being drunk could explain walking out into traffic.

heedlessly2• 8 years ago

Homeless people tend to be in poor mental health and possibly under the influence of drugs/alcohol. All 3 of these things can contribute to bad judgement when she got hit.

It's unfortunate that this happened. But we do need take into context the situation and all factors involved.

DoreenMichele3• 8 years ago

Also... I don't see how her being homeless has anything to do with it.

I can think of a number of ways it may be relevant. In addition to what I said in this discussion, I left a comment about that here:

https://news.ycombinator.com/item?id=16625242

taneq3• 8 years ago

Exactly. Given the information available at this point, the headline and most of the article is pure sensationalism. It could equally easily have been spun as "Homeless lady walks in front of car" but that won't get the clicks.

smilekzs• 8 years ago

Anecdote: About a year ago when I was driving at night I almost hit someone, apparently homeless with dark-colored clothes, pushing a covered shopping cart across a 5-lane street with no crosswalk. Fortunately I was in one of the middle lanes so I noticed them and slammed the brakes in time (while risking getting myself rear-ended).

DoreenMichele3• 8 years ago

The article says she died from her injuries some time later. So, she didn't die instantly on impact.

I know more than average about homelessness. A high percentage of homeless people have serious health problems. If she was homeless, she may have already been in very poor health. That may have contributed to her death.

I can think of potentially other contributing factors, but I am leery of coming across like I am badmouthing homeless people. They are subjected to enough hostility and prejudice and I am usually trying to advocate on their behalf.

muxator• 8 years ago

Sorry, but what's the point of this?

Elcock said he believed Herzberg may have been homeless.

I do not think that Herzberg possibly being homeless adds any meaningful information. On the contrary, I feel this may subtly support unconscious prejudices.

cloakandswagger• 8 years ago

Spend some time in any major city and you'll see homeless people doing all sorts of reckless things around cars. Just last week I saw a homeless man jaywalk into oncoming 40+ MPH traffic without so much as looking or stopping.

In this sense, it is a possibly relevant detail of the story.

danso30• 8 years ago

He said this in response to a reporter who asked if she was homeless: https://www.facebook.com/cnn/videos/10158139292591509/

Can't speak for the reporter who asked that, but I can take a guess about why it was asked. The victim's death and identity has been known for half the day. Her being homeless might possibly figure into next-day stories about why we haven't heard much from grieving family members. But yes, you're right, it is a detail that will figure into people's prejudices.

nothrows• 8 years ago

“The pedestrian was outside of the crosswalk. As soon as she walked into the lane of traffic she was struck”

Uber has some fucking shitty programmers guaranteed. They murdered this lady. Think about it:

Smart cars basically have 360 degrees of sensors. There’s no way it didn’t detect that lady coming up on the car and the fact that it kills her says to me there was no counter measure. It probably didn’t even slow down until the moment of impact. That's some fucking elementary school shit.

Chance of Collision? 100%. Maintain Speed

In structural engineering if you designed a bridge poorly and it collapsed killing people you'd lose your ability to practice engineering. The same should be applied.

lopmotr3• 8 years ago

There's no indication that this is the fault of the car or its human driver. Unfortunately, the media won't notice that. She stepped out in front of the car and since she was struck immediately, it sounds like the car had no time to stop or swerve, let alone time for the human driver to react in any way.

EDITed to remove offensive word.

dang20• 8 years ago

rabble

Please follow the site guidelines when commenting here. Adding flamebait into a flammable topic sparks explosions, making this place worse for everyone.

https://news.ycombinator.com/newsguidelines.html

booleandilemma• 8 years ago

I feel like I should share this here.

The first person killed by a car:

http://www.guinnessworldrecords.com/world-records/first-pers...

xbmcuser• 8 years ago

Why am I not surprised the first death of a pedestrian came from a Uber driven car.

itsuart• 8 years ago

So, who is going to jail (or at least - trial) over this?

kappi• 8 years ago

1500 comments before knowing the real reasons. It could be suicide. The dead woman has history of substance abuse and arrested 6 times. http://www.dailymail.co.uk/news/article-5519433/Self-driving... http://fortune.com/2018/03/19/uber-self-driving-car-crash/

ISL• 8 years ago

It needn't be suicide; from the second link, the victim may have stepped from a shadowy spot/out of view. A car cannot avoid what it can't see.

femto3• 8 years ago

“They are going to attempt to try to find who was possibly at fault and how we can better be safe, whether it’s pedestrians or whether it’s the vehicle itself.”

Autonomous vehicles should automatically be 100% liable in every accident in which a person is injured, and it should be paid for by a levy on every vehicle.

A levy is fair because the per vehicle risk is much more uniform than with a driver behind the wheel.

100% liability is fair because the software should be written so that the vehicle predicts the path of all objects within its range and adjusts is speed/trajectory so that there is negligible risk of collision given the worst case behaviour of the object. If an object doesn't show on the car's sensors, then its still the car's fault as the sensors should be built to they do pick up all objects. There is a complete power imbalance otherwise: software cannot be injured in a collision but people can, so all people should be 100% covered to give the software incentive to be the best it can.

Summary: a car is governed by predictable physics, hence there is no such thing as an accident involving an autonomous vehicle, only mistakes.

JimboOmega2• 8 years ago

Given current road and vehicle design, that's an incredibly impractical goal. There simply isn't enough separation between pedestrians and roadways; to keep injury risk negligible you'd also have to keep speed incredibly low (way below speed limits) on all but the best limited access highways.

If the vehicle can only crawl at 5MPH because you never know when a pedestrian might just dive in front of you... that's just impractical for real vehicle operation.

It's one thing to hit the brakes because somebody decided to jaywalk without looking; another to expect there is always enough space to stop in those situations.

femto3• 8 years ago

I disagree. Pedestrian density is generally low enough that the car could generally be doing 60km/h and not hit a "worst case" pedestrian.

For example, how many people do you see walking along freeways? If a person is on a freeway then cars should be slowing down for them. It's a rare enough event that cars could be programmed to avoid all collisions with minimal impact on average speeds.

If a car is around people then it should be going at a low speed. If there are enough people around that it can't get speed up, then that car is effectively in a "shared" or "local traffic" area and should be going slowly.

I'd argue that the effect on average speeds will be similar to having low speed limits on residential streets:a slower period at journey beginning/end but the majority of the journey would be at speed on sparsely populated arterial roads.

Here's another argument: intrinsically safe programming might even increase average speeds? Why do we even need traffic lights and pedestrian crossings with autonomous vehicles? If vehicles were 100% safe, we could do away with dedicated crossings, and there would be no need for vehicles to waste time at red lights, merely slowing down on the relatively rare times when it is necessary.

curveship• 8 years ago

Here we go. We'll now have the first traffic fatality trial where it's not drivers-trying-drivers but people-trying-a-megacorp.

(Disclaimer: I'm a bike advocate, so I may have a different perspective on some of this than most.)

Our car-based transportation system is far and away the most dangerous thing any of us accept doing on a daily basis. 40,000 die a year.

But when cases come to court, everyone on the jury has in the back of their mind "that could have been me if I lost concentration at the wrong moment, or made one bad judgement, etc etc."

So penalties are comparatively light for traffic fatalities. Big punishments are only meted out if the case is so egregious -- repeated drug use, flagrantly reckless behavior -- that the jury can be convinced that the driver is different from them.

In other words, drivers don't get punished for doing something dangerous, because everybody on the road is doing something dangerous. They get punished for doing something more dangerous than the norm.

In this case, there's no question that the "driver" is different than the jury -- it's a computer. Now the symmetry that made jurors compare themselves to the accused is broken.

The result, and what self-driving car advocates don't get, is that self-driving cars don't just have to be safer than human drivers to be free of liability, they need to be safe period. In a trial, they don't benefit from the default "could have been me" defense.

That's a HUGE requirement. In fact, it's probably impossible with our current road system. It won't just take better self-driving cars, but better roads and a major cultural change in our attitudes about driving.

As a bike advocate, I welcome this shift, but I also see how deluded many of the current self-driving projects are. Software moves fast, but asphalt and mentalities move slow. We're not years away from a self-driving transportation system, we're decades.

And this trial is just the beginning of that long story.

femto3• 8 years ago

There's a current trend for people to anthropomorphise cars, with the backing of the industry (two eyes/lights and a mount/grill on the front of "him/her"). Won't the industry go into overdrive on the anthropomorphism front to try and preserve this judicial empathy?

CydeWeys3• 8 years ago

People can't be cars, so, no.

brownbat• 8 years ago

Most drivers avoid the most serious penalties, and more importantly most victims are denied restitution, not because of sympathetic juries but because of insolvency.

The chance someone will cause a grave accident and the chance someone is insolvent are not independent variables.

You get some real horror stories in law school about people trapped in burning vehicles with no remedy for the surviving family.

https://en.m.wikipedia.org/wiki/World-Wide_Volkswagen_Corp._...

One way you can solve for this is strict products liability. Normally we ask if there's a defect in design or negligence. That produces costly litigation where a lay panel reviews a bunch of technical engineering documents with no special training, with highly varying outcomes. Instead, just have the manufacturer pay some statutory compensation whenever their product causes harm. The social harm of the thing will be borne by its purchasers.

It's hard to do that with car accidents though, because driver error contributes so much to outcomes, it's often weird to punish the company.

But we could vastly simplify the auto insurance system and improve the safety of these vehicles by forcing them to just warranty against harm once drivers are out as a factor. (Insofar as a warranty is a promise not to do harm with a fixed penalty up front based on breach.)

It would be fitting if such a statutory regime were the earliest primary laws about AI, because that would officially make a requirement that ai does not harm humans into literally the first law of robotics.

namelost• 8 years ago

I agree. People wish not to avoid death, but to lessen their fear of death.

Arguments about how relatively safe self-driving cars will be are beside the point. A self-driving car is something that can kill you. The fact that death by self-driving vehicle is especially rare makes it especially scary.

A low accident rate is not a good thing, psychologically. Only when a risk is comfortably common can we comfortably begin to ignore it.

robhunter• 8 years ago

I'd be interested in hearing more about why you are a bike advocate?

Maybestring3• 8 years ago

Here we go. We'll now have the first traffic fatality trial where it's not drivers-trying-drivers but people-trying-a-megacorp.

On even the slimest cause, plaintiff's council always tries include the manufacturer in the case. Anyone with deep pockets that they can pull in.

bad_hairpiece• 8 years ago

Moreover, "Wrongful death settlements are often paid out by insurance providers who provide liability coverage for the person or entity for whom the death is being blamed. Insurance policies typically have a policy limit amount, above which the insurance company will not pay and the person is individually liable" from https://www.google.com/url?sa=t&source=web&rct=j&url=https:/...

Although, as far as I know, were the jury to become aware of this, a mistrial should be declared.

dilanj• 8 years ago

According to the latest info, it doesn't look like the system was at fault at all. Seem like the likely homeless woman abruptly stepped into traffic. Every accident still of-course is terrible, but some maybe are unavoidable. https://www.sfchronicle.com/business/article/Exclusive-Tempe...

beedogs2• 8 years ago

Funny how this story has been pushed off the front page of HN prematurely.

aanet• 8 years ago

I would hate to have been the (backup driver) in this sad situation. Regardless, the regulations couldn't come soon enough.

From SF Chronicle [1]:

<QUOTE> The self-driving Volvo SUV was outfitted with at least two video cameras, one facing forward toward the street, the other focused inside the car on the driver, Moir said in an interview.

From viewing the videos, “it’s very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway,” Moir said. The police have not released the videos.

The incident happened within perhaps 100 yards of a crosswalk, Moir said. “It is dangerous to cross roadways in the evening hour when well-illuminated, managed crosswalks are available,” she said.

...

“I suspect preliminarily it appears that the Uber would likely not be at fault in this accident, either,” Moir said.

However, if Uber is found responsible, that could open a legal quagmire.

“I won’t rule out the potential to file charges against the (backup driver) in the Uber vehicle,” Moir said.

But if the robot car itself were found at fault? “This is really new ground we’re venturing into,” she said. </QUOTE>

[1] https://www.sfchronicle.com/business/article/Exclusive-Tempe...

chriskanan• 8 years ago

>The pedestrian was outside of the crosswalk. As soon as she walked into the lane of traffic she was struck.
>Tempe Police Chief Sylvia Moir said that from viewing videos taken from the vehicle “it’s very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway.

A lot of comments on here are faulting machine learning for this or algorithmic flaws, but unless we see the video it is really hard to blame the algorithms. Based on the quote from the story, it sounds like someone stepped out into the street, not in the crosswalk, at night, directly in front of a car going 40 miles per hour. How could a human or any algorithm cope with that scenario?

msie• 8 years ago

Everyday I see crazy, drunk or angry people just walk across the road recklessly. Some seem to walk in defiance of the cars.

keypusher• 8 years ago

Elaine Herzberg, 49, was walking her bicycle outside the crosswalk on a four-lane road in the Phoenix suburb of Tempe about 10 p.m.
Tempe Police Chief Sylvia Moir said that from viewing videos taken from the vehicle “it’s very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway."

This seems like an unfortunate accident, but it's not at all clear that the car was at fault or that it could have done anything to prevent it. There was even a human behind the wheel and they didn't react in time either.

gkya2• 8 years ago

I can't describe how shocking it is watching many HN commented blame a person killed by a machine and defend the thing instead. There are even those who are saying that such deaths are necessary and useful for the advancement of technology.

tim3334• 8 years ago

The self-driving Volvo SUV was outfitted with at least two video cameras, one facing forward toward the street, the other focused inside the car on the driver, Moir said in an interview.
From viewing the videos, “it’s very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway,” Moir said. ( https://www.sfchronicle.com/business/article/Exclusive-Tempe... )

Not saying the software wasn't partly at fault but it doesn't look that clear cut. It sounds like better sensors could have helped.

scottmcdot• 8 years ago

Shouldn't '[Coming] from the shadows' be irrelevant when the sensors are LIDAR?

ACow_Adonis2• 8 years ago

One also needs to consider the relative dynamic range of the camera/video in terms of both sensors and playback.

If the hardware is not capable of capturing sufficient dynamic range to be able to see into the shadows while driving in daytime/nighttime in this location, one must ask why not? That seems obviously negligent.

If the car can see into the shadows (say either with lidar or sufficiently sensitive hardware) then a self driving car either just failed to identify and predict the object and subsequently killed a pedestrian, or it detected them and killed them anyway.

If it killed them anyway, it may be that it was physically impossible for them to avoid the detected object, or forensics (if properly performed) would show that the collision was avoidable.

All of these scenarios seem pretty inconsistent with the hypothesis of the police chief knowing what the duck he's taking about, and I question what the hell he's doing releasing such a statement before the evidence is in...

drieddust4• 8 years ago

Is it normal for police to give these kind of statements right away? I thought they need to present evidence to the court of law instead of giving verdict themselves.

kwhitefoot• 8 years ago

Why is everyone concentrating on the software? There was a human in charge and the car was exceeding the speed limit. That sounds like either driving without due care and attention or reckless driving. After all the driver's reaction time is almost certain to be much longer when supervising an autonomous car than when driving directly making it much more important to adhere to the rules.

And if it can be shown that it was company policy to allow the cars to exceed the speed limit then heads should roll all the way to the top of the company.

kawfey• 8 years ago

It was exceeding the speed limit by 3mph, I don't believe this far exceeds safe margins.

the_cat_kittles3• 8 years ago

could there be some way of emitting a signal on your person so cars never get confused? seems like a pain in the ass, but maybe it would be worth it? cars can know for sure there are people wherever the signal is emitted from? dunno if its feasible, curious

HGGLLJNLJHU• 8 years ago

http://www.autonocast.com/blog/2018/1/18/46-part-two-of-a-co...

bambax2• 8 years ago

If the car is deemed non-responsible it makes it the perfect crime...

How long before someones tries to program a car to "hunt" a victim and hit her when she makes a mistake and it will appear it was a pure accident with no-one at fault?

hndamien3• 8 years ago

I don't see why this lesson could not be learned on a test track or simulator.

Self-driving Uber car kills Arizona woman crossing street

Related Stories