Our Self-Driving Cars Will Save Lives, but They Will Kill Some of You First

https://www.mcsweeneys.net/articles/our-self-driving-cars-will-save-countless-lives-but-they-will-kill-some-of-you-first

by sundaeofshock • 3 years ago

66 86 3 years ago

mcsweeneys.net

inamberclad• 3 years ago

I know this article is satire, but it certainly strikes a chord. Putting big metal robots capable of moving at highway speed in close proximity to squishy humans is going to cause issues. It might be the human's fault. It might be the vehicle's fault. It might be completely unrelated.

However, there are industries like aviation that take these issues seriously. They're investigated, reported, and learned from. We need to do the same for autonomous vehicles if they're actually going to be safe. This means we need an open, independent body outside of industry control, like the NTSB or NHTSA that can produce in depth, transparent, and publicly available reports that manufacturers can't bury or buy away.

On the other side, we, the (future) passengers need to be conscious of these safety issues. We need to hold companies accountable for fixing flaws in autonomous vehicles. Airplane accidents can be spectacularly big, which captivates the public. I'm worried that car accidents will just be too small for enough people to care, and therefore we will continue to have a trickle of fatalities in every city and we will never be free from cars.

JumpinJack_Cash5• 3 years ago

> However, there are industries like aviation that take these issues seriously. They're investigated, reported, and learned from

If everything were to be as safe as aviation the whole world would come to a screeching halt standstill.

And besides they do it to protect the 200M plane not the passengers because those same passengers enter in the subway or a mall or a stadium or a train 15 mins after they disembark from the 200M plane, and they could bring (or face someone with) an assault rifle without facing any screening or x-rays or even an ID check

I don’t like self driving cars but if the bar to plow ahead with deployment of a new tech is commercial aviation post 9/11 then we can call the end of technological improvement already.

woodruffw10• 3 years ago

There's a lot of speculation here: aviation manages to be simultaneously extremely safe and a cornerstone of the global economy. It's unclear why other modes of transportation can't do the same; more precisely, it's unclear why private cars get put in the "acceptably dangerous" bucket when just about everything else is astoundingly safe (trains, planes, public & private buses, etc.).

(Independently: the bigger reason the FAA and airlines take security seriously is because consumers demand that they do, not just because the planes themselves are expensive. Humans are notoriously bad at estimating risk.)

arcticbull4• 3 years ago

If everything were to be as safe as aviation the whole world would come to a screeching halt standstill.

Trains are just about as safe as planes. Busses too. It's really just cars that are a major outlier. [1]

[1] https://injuryfacts.nsc.org/home-and-community/safety-topics...

Veserv3• 3 years ago

Not everything, just things that can kill hundreds to thousands of people if they go wrong. That is safety engineering 101; identify the important and critical parts and do those parts better and more carefully.

You do not need to uniformly improve all elements to the same high standards as long as you can isolate them from the safety-critical path. This is a necessity because, as you point out, doing everything to the high standards is economically infeasible. But doing everything to economical standards is mass murder. No one standard works for the entire system, so you must isolate and apply the correct amount of effort for each component.

This is a very weird concept for most software developers because most software is a big mushy ball of spaghetti where it is impossible to isolate or even identify the critical path. Luckily, those individuals largely do not develop safety-critical systems, so the lack of knowledge of basic safety processes does not hinder their deployments or result in a danger to public safety.

What is problematic is when those people start doing safety critical systems. That is how you get the ideas being satirized by the article. Acting as if there is nothing in between total recklessness and impossibly perfect safety, so there is no choice but criminal recklessness. Basic industry-standard processes are, in fact, like 6 orders of magnitude safer during development and increase development velocity toward a safe, working product.

Any viewpoint as insane as normalizing deaths during the development process is totally incompatible with successfully deploying a safety critical product. Deaths may be unavoidable due to the sheer weight of statistics, but deaths are always a failure of immense gravity. A flippant disregard for human life has no place on a safety critical product.

jakeinspace• 3 years ago

The commenter you're responding to is talking about aeronautical engineering safety practices (things like DO-178), not TSA-style security. If you haven't worked in the aerospace sector, it's hard to appreciate just how slow, methodical, and bureaucratic things are.

toomuchtodo2• 3 years ago

~40k people die in the US from human caused car accidents every year (~12.9 deaths per 100,000 people, ~1.37 deaths per 100 million miles traveled), and we're getting squeamish about the robot that never gets tired, drunk, or inattentive?

If you want to be free from cars, build walkable and carless cities people can afford and want to move to. We can then discuss sunsetting mass automobile infra (which is very unlikely, although infra spending can be weighted to more efficient urban development, mass transit, bicycle infra, etc vs automobile centric systems). Until then, the need for autonomous vehicles is an inconvenient truth.

This means we need an open, independent body outside of industry control, like the NTSB or NHTSA that can produce in depth, transparent, and publicly available reports that manufacturers can't bury or buy away.

Strongly agree.

add-sub-mul-div2• 3 years ago

40k people

You're jumping to a quantitative comparison because that's simpler and more obvious, but the right way to think about it is qualitatively.

We have over half a century of understanding of modern human driving patterns. We know the ceiling on the number of accidents/deaths that humans will cause. We know that if it changes, it will change slowly.

AI works differently. An update to code or models wipes out all expectations we've learned from prior versions.

In short, the entire fleet of autonomous vehicles can change behavior overnight with a software update and become murder tanks.

Nothing will ever happen to all human drivers at once. Some small percentage will be drunk at any given time. We'll never all wake up with a new drunk version of ourselves at the same time.

I don't mean to say that autonomous technology could never work in principal. But we know that this SV in practice has too much hubris and too little oversight to deploy sensitive products responsibly, or with attention to any values above profit and growth.

epistasis3• 3 years ago

If you want to be free from cars, build walkable and carless cities people can afford and want to move to.

Oh man would I love that, but our legal system makes this nearly impossible, especially when combined with car culture.

It's a guaranteed way to reduce pollution, improve happiness and health, save lives, etc etc etc. But it is easier to solve computer vision than go get cities to ban cars.

Animats• 3 years ago

If you want to be free from cars, build walkable and carless cities people can afford and want to move to.

"It will only encourage the lower classes to move about, and this is certainly too big of a step." -- The Duke of Wellington, on railroads.

mjan22640• 3 years ago

robot that never gets tired, drunk, or inattentive?

it also never gets bug free

1vuio0pswjnm7• 3 years ago

"~40k people die in the US from human caused car accidents every year (~12.9 deaths per 100,000 people, ~1.37 deaths per 100 million miles traveled), and we're getting squeamish about the robot that never gets tired, drunk, or inattentive?"

And never is held responsible, never compensates its victims and never gets punished.

It's anybody's guess. No one knows who will be held responsible. I find it difficult to imagine a so-called "tech" company accepting responsibility for damages, injury or death caused by software, but who knows.

In the one case so far in AZ, Uber was not held criminally liable. Instead an Uber driver plead guilty to negligent homicide. Uber quickly settled the civil case with the victim's family. The amount of the settlment is not public information.

https://www.reuters.com/business/autos-transportation/backup...

https://money.cnn.com/2018/03/29/technology/uber-fatal-crash...

Guvante• 3 years ago

"Human drivers kill a lot of people" is true "AI will probably kill fewer people" is true but probably is doing a lot of work when you turn that into "no reason to slow down AI full speed ahead".

In fact that is exactly the point of the article, we shouldn't fall into the trap of not worrying about deaths for AI just because we have deaths from humans.

After all part of the problem with human deaths is we consider it normal. If we can avoid that trap with AI we can ensure we do better.

adfkganio• 3 years ago

The "legacy automakers" that took the meticulous approach, gradually rolling out boring sensor suites with boring software in controlled conditions, are running circles around the reckless upstarts who kill people on a regular basis.

We do not need to sacrifice safety for progress. We don't need mad scientists in the real world. Sanity leads to better results. It doesn't pump the stock price all that well, though.

Buttons840• 3 years ago

You say we need to investigate autonomous vehicle crashes, I agree.

We haven't done this with people because there's not much to investigate; someone drinks, drives, and crashes, there's not much to gain from serious scientific investigation here. Autonomous vehicles are different and we would benefit from more focused scientific investigation of autonomous vehicle accidents.

bobbylarrybobby• 3 years ago

The idea that there's not much to investigate after an accident — that every accident is idiosyncratic — is one of the great lies of the auto industry (and their buddies, transportation planners).

seanmcdirmid• 3 years ago

We need to hold companies accountable for fixing flaws in autonomous vehicles.

Don't we even just start getting a chance at doing that with autonomous vehicles? When a car crash happens and driver is found at fault, we can say "let's fix driver training to prevent this from happening again!" but humans are diverse, many don't pay attention in driver's ed, and they retain learnings at different rates, so this isn't going to work very well. With a self driving car, at least, since the software is replicated in many vehicles (while humans aren't), we have an actual chance improving on the situation over time.

SCAQTony2• 3 years ago

Perhaps better rapid transit could be quicker and safer than driving in amy form. Perhaps designers should design cities around rapid transit rather than building rapid transit after the cities have been built.

thereisnospork2• 3 years ago

It won't, at least not in the near term:

Cars are on the precipice of both becoming immensely greener (electric) and immensely more practical (self-driving) than the status-quo. The transit-equilibrium is going to shortly and rapidly shift towards cars, not away from them.

arcticbull4• 3 years ago

They're better, but I'm not ready to call them immensely greener. They're the same order of magnitude of impact, all things considered.

Creating an electric car battery emits as much CO2 as creating a whole new gas vehicle from scratch. Depending on the size of the battery and on the green-ness of the grid it's attached to, it takes anywhere from 20,000 to 90,000 miles of driving an EV to reach break-even against a gasoline car. Then it has to be replaced after 100-200,000 miles. Then there's the environmental impact of lithium, copper, etc, extraction.

Of course you have to consider that private ownership requirements and mandating parking minimums, free parking, etc, are a massive regressive burden on the poor. Between fuel/power, maintenance, insurance, tickets, tolls, parking, interest, payments, etc, you're liable to spend a half million dollars over your life on your car.

Why? Why not retire sooner, or leave more to your kids?

Freedom is your ability to go anywhere you want any time you want, it shouldn't be defined by your ownership of a very expensive metal box.

The environmental benefits are quite likely to be wiped out by the growth of the car market in for instance India and China.

JambalayaJim3• 3 years ago

Electric and self driving vehicles do not solve the fundamental problem with personal vehicles; they cause congestion.

woodruffw10• 3 years ago

Perhaps designers should design cities around rapid transit rather than building rapid transit after the cities have been built.

This is already the case, at least in the US. Most of our car-centric cities didn't start out that way; they were systematically converted into car-centric environments by dismantling public transit systems and replacing them with highways and surface-level parking.

Examples: Sioux City[1]; Albany[2]; Boise[3].

[1] https://www.siouxcitymuseum.org/history-website/transportati...

[2] https://www.timesunion.com/living/article/The-evolution-of-t...

[3] https://www.boisestate.edu/sps-urban/urban-resources/trolley...

grandmczeb7• 3 years ago

Sounds great, so when do you think those new cities will be ready?

SCAQTony2• 3 years ago

One is being planned outside San Francisco. They have invested nearly a billion-large so far. REF: https://fortune.com/2023/08/26/silicon-valley-elites-want-ne...

patrick451• 3 years ago

Our streets and highways are dangerous, and it’s clear that the current system isn’t working.

In 1970, there were about 26 traffic fatalities per 100,000. Today, that number is down over 50% to 12. We are doing something right.

paulryanrogers• 3 years ago

That stat doesn't capture all the life altering injuries or financial losses. Cars are incredibly powerful and dangerous compared to lesser modes like walking, bicycling, or even scooters.

Aloha4• 3 years ago

If you did look at life altering injuries and financial losses, you'd find fewer of those too - cars today are measurably safer, from tires, to better seatbelts, to airbags, to ABS - cars in virtually every way are safer. I'm actually reasonably certain that bicycles and scooters are more dangerous than cars.

The one exception to this is danger to others, modern cars are probably more dangerous to pedestrians than cars in 1976.

woodruffw10• 3 years ago

The first mandatory seatbelt laws weren't passed until the 1980s[1]; that and frontal airbags (not required until 1998[2]) probably cover a large amount of the overall decline in deaths on our roads.

That's to say that we have done some things right, but that it's premature to assume that we've done all we can. We can, and should, go further (including but not limited to traffic calming, disincentivizing driving where it makes sense, and making the legal and educational requirements for driving similar to those in European countries.)

[1] https://en.wikipedia.org/wiki/Seat_belt_laws_in_the_United_S...

[2] https://www.autonews.com/cars-concepts-history/us-frontal-ai...

Aloha4• 3 years ago

For the uninitiated, how do differing european rules make people safer?

grandmczeb7• 3 years ago

Worth stating that “something right” in this case is adopting life-saving technologies like seatbelts and airbags, exactly the kind of thing the original article is arguing against.

PopAlongKid• 3 years ago

Leaving aside the satire, over ten years ago, this article pointed out that "The Ethics of Saving Lives With Autonomous Cars Is Far Murkier Than You Think".[0]

"Let’s say that autonomous cars slash overall traffic-fatality rates by half. So instead of 32,000 drivers, passengers, and pedestrians killed every year, robotic vehicles save 16,000 lives per year and prevent many more injuries.
"But here’s the thing. Those 16,000 lives are unlikely to all be the same ones lost in an alternate world without robot cars. When we say autonomous cars can slash fatality rates by half, we really mean that they can save a net total of 16,000 lives a year: for example, saving 20,000 people but still being implicated in 4,000 new deaths."

[0]https://www.wired.com/2013/07/the-surprising-ethics-of-robot...

omarforgotpwd• 3 years ago

Funny article, but as someone who has access to Waymo, Cruise, and Tesla Full Self-Driving Beta I have never seen any of the above have trouble differentiating between a green light and a green shirt. Again, very funny but from a technical perspective the challenges he's describing — rain, a dog running into the road, are things that I see many different companies already solving reliably.

woodruffw10• 3 years ago

What about blocking ambulances[1]?

[1] https://www.nytimes.com/2023/09/02/technology/driverless-car...

SR2Z2• 3 years ago

The NYT, in this article, admits to seeing a video that proves that Cruise did not block that ambulance.

This is a garbage headline meant to garner clicks. The city has been complaining about these cars for a while. They've never had any data and now are trying to complain about "x incidents where an AV interfered with emergency vehicles." They're banking on the headlines, and then they'll bring up whatever number they could passably claim in the next letter/meeting they have with the DMV.

sagarm• 3 years ago

So much discussion about whether Cruise slightly delayed the ambulance from leaving the scene. Hardly any discussion of the fact that it was a human driver that ran over the pedestrian, "critically injuring" (brutally crushing parts of their body, so they bled out after the driver fled I assume) them in the first place.

Was the driver charged? Did the street design contribute? Nobody cares, because apparently a few dozen San Franciscans' lives per year are a worthwhile sacrifice to save suburbanites a few minutes on their commute.

foxes• 3 years ago

Self driving / electric etc is just the same scam.

Remove car dependence and invest more in public infrastructure.

matthewfelgate• 3 years ago

We will always need some cars.

There will always be cars.

robbbed• 3 years ago

"Like a space monkey, ready to be shot into space. Ready to sacrifice himself for the greater good."

ryanjmo• 3 years ago

I really didn't like this line of the article. "This stuff isn’t easy, and before we get to a world with zero traffic deaths, we’ll—briefly, I hope—have to increase the number of traffic deaths just a little. We feel really bad about that."

Reason being, while satire the rest could hold true. Mistakes will happen from cars, but mistakes happen from people anyway. Now this line quoted above is just incorrect. The average automated driver is going to cause many less deaths and the average human driver. That is the thing that is so frustrating about this type of talk putting down automated driving. Automated driving (from my understanding) is already safer than human driving, we just don't accept deaths from automated driving but we do from human driving.

The line seems more correct if it was written: "This stuff isn’t easy, and before we get to a world with zero traffic deaths, we’ll—briefly, decrease the number of traffic deaths, but not to zero and mistakes will be made. We feel really bad about that."

That is certainly a more true statement than the one in the article.

Our Self-Driving Cars Will Save Lives, but They Will Kill Some of You First

Related Stories