FBI: Google's driverless cars could be lethal weapons

http://www.bbc.com/news/technology-28344219

by schrofer • 12 years ago

20 56 12 years ago

ANYTHING can be a lethal weapon if used improperly. SHUDDUP Gubmint.

dllthomas• 12 years ago

That's a good reason not to use this as justification for regulation or nonsense. It's not a good reason for them not to pay attention to developments and figure out possible responses to various situations. I don't know to what degree "warns" was flavor from the reporter versus content of the statements, and I don't know the context in which these statements were made.

DamnYuppie• 12 years ago

LOL, and how many tens of thousands die a year in accidents caused by vehicles with human drivers? Apparently over 30k according to the NHTSA.

sp332• 12 years ago

The FBI didn't say it was a bad tradeoff.

xster• 12 years ago

lol, in 100 years, it'll look like us saying penicillin should be illegal because if you fired a bucket of it through an uber powerful potato cannon, it can act as a buckshot

nickthemagicman• 12 years ago

What about a driverless car filled up with murderous clowns?

That's something we should all be afraid of and why we should stop progress immediately.

krapp4• 12 years ago

Because any suggestion that driverless cars might present any hint of increased risk in any possible scenario amounts to mindless Luddite paranoia and should be dismissed out of hand?

Who's talking about "stopping progress immediately?" The FBI doesn't appear to be saying that in this article, rather they seem to be speculating on how driverless cars might tactically change the dynamics of a vehicular pursuit, which is a perfectly valid thing for them to do. Because they're right - you can shoot at the cops a lot better when you don't have to worry about a steering wheel.

fixermark• 12 years ago

Then the headline is misleading. The car isn't the lethal weapon; the car is enabling the occupant to do dangerous and illegal things.

IanDrake2• 12 years ago

Also, the Pope is upset because unmarried people could have sex in the back seat while the car drives itself. There is really no end to the infinite possibilities.

any hint of increased risk in any possible scenario amounts to mindless Luddite paranoia

Not any hint, but this particular one...yes. Dismissed out of hand. A get away driver (an idea that has been around for a very long time) would be far better than a computer that will stop for literally anything in front of it.

The FBI pushing such stupidity is embarrassing to me as an American. Next they'll report that if banks didn't have money they'd never get robbed.

mikeyouse2• 12 years ago

It's easy to ridicule the FBI for statements like this, but I'm glad they're considering the downside to some of these technologies. The pie-in-the-sky version of a driverless car society is tremendous, and I hope we get as close as possible to that, but it does open up some other problems.

1. How would you go about stopping an autonomous semi truck that was programmed to drive through crowds?

2. Will terrorists use autonomous cars as delivery mechanisms for what were formerly suicide bombs?

3. If locations with pedestrians are sandboxed in some way, could a sufficiently skilled 'hacker' intentionally break that sandbox without a trace only to blame it on bad technology?

4. How much privacy / 'liberty' would we be willing to give up to enable such a society?

Someone should be considering the worst-case scenarios..

Someone12342• 12 years ago

1. How would you go about stopping an autonomous semi truck that was programmed to drive through crowds?

In my opinion that question is no longer important. Back in the 80s and 90s the prevalent thinking was that "bad guys" won't die for their cause. This mandated things like removing the baggage of passengers who aren't flying (which wasn't the case before).

I think we learned quite starkly in the 2000s that suicide attackers are a legitimate threat and therefore we cannot take for granted any longer than a "bad guy" won't put themselves in harms way in order to hurt others.

So if we apply that to your autonomous semi-truck example, how do you stop that NOW? Since presumably someone who wanted to accomplish that could go out right now, get in a truck and then do it. Automation doesn't really change too much.

As far as an answer: Concrete bollards supported by steel mesh (both internally and in the foundation) in pedestrian areas.

mikeyouse2• 12 years ago

Surely the universe of "bad guys willing to kill scores of people" is magnitudes larger than the universe of "bad guys willing to die to kill scores of people"?

sailfast2• 12 years ago

This is true, but humans scale a lot worse than automation. Now instead of convincing somebody to kill themselves, all I've gotta do is wire up 30 trucks to do what I need them to do. Didn't work? Do it again. Cars are cheap.

Discussing this keeps reminding me of Suarez's Daemon novels. Swarming changes many, many things.

sailfast2• 12 years ago

Agreed. A law enforcement kill switch requirement is probably something that will be discussed as agencies talk through compromises that enable this to happen. (Can't think through all of the pros/cons to this)

That said, if someone is skillful enough to program an autonomous vehicle, they'll probably figure out a way to get around a kill switch as well - like any technology, it can be used for both good and bad. Risk can't be completely removed, but can be sufficiently mitigated. I'm glad the report addressed both sides of the argument, and I'm a bit disappointed that the BBC chose to sensationalize one small conclusion to such an extent.

dm27• 12 years ago

An autonomous semi-truck that drives through crowds?

I understand that there are maniacs out there, but the technical skill required to do this combined with the cost and the level of insanity would make this scenario pretty unreasonable to consider a valid threat. Not saying it couldn't happen, but there are easier ways to cause a similar amount of damage.

How would I stop it?

- Box it in with dump-trucks, other semis, or armored vehicles.

- Use a tank or other armored vehicle from one of the many armories located everywhere.

- Use one of those MRAPs that the military has been giving law-enforcement recently.

- Shoot out tires.

- Armor piercing or 50 caliber rounds to the engine.

- Remote shutoff that is impossible to disengage without also disabling the GPS.

- I'm very close to a major air-force base, they have plenty of ways to stop a civilian semi-truck.

- Use one of the many armed drones, a military helicopter, or even an armed aircraft. I'd have no problem with the US military using an air-to-ground missile to take out a autonomous vehicle that was programmed to do harm, as long as they were sure that there would be zero civilian casualties.

IanDrake2• 12 years ago

How would you go about stopping an autonomous semi truck that was programmed to drive through crowds?

Same way you stop an non-autonomous one

Will terrorists use autonomous cars as delivery mechanisms for what were formerly suicide bombs?

No. If they wanted to the could use a $200 remote control car to get a good sized bomb into a crowd from a distance.

For those people, suicide is part of the point.

could a sufficiently skilled 'hacker' intentionally break that sandbox without a trace only to blame it on bad technology?

It doesn't take a genius to cut a few brake lines of your target, but we don't see that happening.

Throwaway823• 12 years ago

You can remote control a vehicle or truck at the moment with a few electronics and supplies. I don't see anyone using them for weapons, so why would autonomous vehicles be used that way?

As for suicide bombers, a lot of them appear to have other options for achieving the same goal, yet they choose the method that takes their life. They can buy a grenade for $10, and toss it into a crowd, yet they choose to strap a bomb to their chest. I don't think they're doing so because it's the only option, and because autonomous vehicles are not available, I think they're doing it because they want to give their life to their cause.

What about quadcopters? Couldn't anyone just strap a bomb to a quadcopter, fly it over a stadium, and down into the crowd? I think that's even more scary than autonomous vehicles, because they can fly over barriers.

People have countless ways they can cause damage or mass casualties available to them. However, I can walk around a city surrounded by millions of people, and feel completely safe. Most people are quite nice, they're not sitting at home unable to achieve their terrorist goals because Google autonomous vehicles are not available to them.

Nothing will change. Sure, someone will toss explosives in an autonomous vehicle, instead of putting them in a parked car. It's unfortunate, but not a reason to hold back technology that would positively benefit the other 99.99999% of the population.

Sir_Cmpwn• 12 years ago

Wow, the comments here are of an unusually low quality. Let's step it up.

I've personally been thinking of the negative implications of driverless cars as well. I also thought that a car with a driver could take advantage of a highway full of conservative automated cars by driving like a maniac to get to his or her destination faster while all the autonomous cars moved out of the way.

Of course, I still strongly support moving to driverless cars, but I can acknowledge that there are some unanswered questions.

antr• 12 years ago

The report noted criminals using automated cars would have both hands free and be able to take their eyes off the road during a car chase.

... and guns aren't the problem?

stanmancan• 12 years ago

Would a driverless car really be ideal to use as a get away vehicle? You would be stuck in a car that obey's the speed limits and rules of the road. Hardly useful to dodge the cops.

rtkwe• 12 years ago

Clearly that section is based on the question 'assuming car software could be modified in a way that made them violate the Laws of Robotic Cars, what new dangers would that pose.' Just like the bit that mentions the trailing car assumes they would have a way to know where a car is going in the future, suggesting a way to query a given car for it's direction.

The first isn't an outlandish question, modifying locked down software isn't exactly a new art. It wouldn't happen for a while likely just due to the small reward to the difficulty of execution.

dm27• 12 years ago

Autonomous vehicles offer far more benefits than they have drawbacks.

This article is mainly focused on the situation where the criminals will have both hands free. It ignores the fact that the vehicles would just stop if anything was blocking the path. Plus, the vehicles could certainly be turned off remotely during an emergency. And nothing stops law-enforcement from having autonomous vehicles that could be set to follow another vehicle with near perfect accuracy.

This is similar to the Tesla stolen vehicle. WHY would the police engage in a high-speed pursuit, risking the lives of numerous people, to catch a vehicle that has GPS in the car and can probably be shut down remotely. Law-enforcement should be partially responsible for the people injured in that situation, in my opinion.

eslaught2• 12 years ago

Your post seems to ignore the very real possibility that criminals will prepare their cars by customizing the programming prior to the crime. Does your car have a remote disable switch? Hack the car and turn it off. If you have arbitrary physical access to the car, then it will be very difficult to stop you from doing this.

As others have commented, you really do have to take into account the worst-case scenarios. Even if automated cars are a win in 99.9% of the cases, the 0.1% of failures (intentional or not) may be devastating.

dm27• 12 years ago

Answered here: https://news.ycombinator.com/item?id=8049671

Also, what's to stop it from happening now?

Remember the guy that stole the tank? https://www.youtube.com/watch?v=0AcVSEY2DP0

Remember the guy that built an armored vehicle out of a bulldozer? https://www.youtube.com/watch?v=C4nlcmmt8X0

Semi-trucks:

https://www.youtube.com/watch?v=n3NKlzZmgl0

https://www.youtube.com/watch?v=loUWmQdj70w

You'd have to equip it with run-flat tires to stand a chance.

Even an armed drone or helicopter would probably be publicly acceptable to stop it.

altcognito• 12 years ago

Even if automated cars are a win in 99.9% of the cases, the 0.1% of failures (intentional or not) may be devastating.

9/11 proves that we are incapable of handling the 0.01% failure case without going to extremes. Very disheartening.

syntern• 12 years ago

You are ignoring the very low-tech possibility that criminals may just hire a good driver. Cheaper and more reliable than hacking a very complex system.

bryanlarsen• 12 years ago

All the police probably have to do to stop an autonomous vehicle is to turn on their lights. Autonomous vehicles have to follow all traffic laws, which would include stopping for emergency vehicles.

killertypo• 12 years ago

Not if you override that bit of logic in the system. Cars are reprogrammed all the freaking time - for performance reasons.

It's not very far fetched to do the same for other nefarious reasons.

Zikes3• 12 years ago

That's actually a grey area. Following incidences of police impersonation, it was recommended that concerned people should turn on their blinker to acknowledge the police and safely drive to the nearest populated area before pulling over.

Having your car automatically and uncontrollably pull over every time red & blue lights are observed could lead to traps being set up on any backwoods road.

Edit: I really wish the news article would link to the actual source report so I could confirm it, but I bet that the above scenario is included in the FBI's hypotheticals.

dm27• 12 years ago

Is it required that you stop for emergency vehicles? I have always been under the impression that moving out of the way was sufficient. Cops would have no reason to pull over autonomous vehicles unless they were stolen or unsafely damaged.

The vehicles could limit themselves to only drive X miles on a public highway after the registration expires before it alerts the DMV and you receive a ticket.

I just realized something, when the majority of vehicles become autonomous, we will need less police patrolling the streets. I'm not sure if that's a good thing or a bad thing.

DanBC2• 12 years ago

They can possibly be remotely controlled to lock the doors and drive to a police compound.

The BBC article fails to link to the report. The few details in the BBC report make me want to say "so much derp", because it really does sound dumb. But then maybe the full report is a lot better?

clavalle• 12 years ago

The report noted criminals using automated cars would have both hands free and be able to take their eyes off the road during a car chase.

They would also be remarkably predictable compared to a human driver.

jqm• 12 years ago

These are very valid points.

It doesn't negate the potential use of driverless cars nor is it a good argument against continued development....just things that are worth considering...

My guess? Law enforcement will try to get a kill switch they control installed in all driverless vehicles. I don't know that they should have this, but I bet they try.

dm27• 12 years ago

The police being able to turn off vehicles should be heavily restricted and logged to help prevent abuse.

Maybe even require them to contact the manufacturer to get this done.

OnStar can do it currently can't they?

Zikes3• 12 years ago

Some people apparently don't realize that this is a part of the FBI's job: to dream up and evaluate all the possible scenarios, weigh risks, and plan for as many contingencies as possible. As the article notes, the report is not entirely negative about driverless cars, apparently mentioning as many pros as there are cons.

The point is, when the first mobster/hacker manages to turn a driverless car into an autonomous hit-and-run machine, or the first terrorist uses one to deliver a car bomb, the responsible agencies will be able to react quickly because they've already made plans for that scenario.

wavefunction2• 12 years ago

I thought they were supposed to investigate crimes?

Not decide what cars can and cannot do because of their speculative thinking...

krapp4• 12 years ago

Why would it not also be their job to speculate on the way new technologies might affect their ability to investigate crimes?

Zikes3• 12 years ago

I thought they were supposed to investigate crimes?

When someone manages to figure out how to send a driverless car through a crowd of people in a mall somewhere, do you want the lead investigator to show up and say "What do you mean there was no driver?!"

Some of investigatory science comes from analyzing crimes that have taken place, and some of it comes from imagining potential new crimes. By having imagined the above scenario and being familiar with the type of car in question, they might know to look for computer logs on the car that could lead them to the perpetrator, or that autonomous vehicles rely on GPS which might have left a trail.

The purpose of the research and report is not to determine whether or not driverless cars should be legal, rather to determine what the potential risks are and how they might be mitigated with regulations and/or cooperation with the manufacturers.

Someone12342• 12 years ago

I always just assumed that driverless cars would come with a remote kill switch that law enforcement and public safety officials could use.

Even if we ignore this particular threat scenario, what happens if someone has a heart attack, is passed out in an automatic car (as seen by other cars on the road), the cops/ambulance turn up but are unable to get to the person to give them aid?

Or if an automatic car somehow malfunctions and starts driving the wrong way down the freeway/motorway.

Nanzikambe• 12 years ago

Article summary: Driver-less car leaves more hands free for evil people to do evil things.

Someone FOI the FBI's report on hands-free sets and prosthetic limbs please.

NoMoreNicksLeft• 12 years ago

Hint to FBI: Driverful cars can be lethal too.

rurban• 12 years ago

I call this bullshit.

As far as I know, criminals had always a designated driver for exactly those reasons. "bad actors will be able to conduct tasks that require use of both hands or taking one's eyes off the road which would be impossible today"

Looks like the FBI is just pushing to get a real-time interface into the car control. "Algorithms can control the distance that the patrol car is behind the target to avoid detection or intentionally have a patrol car make opposite turns at intersections, yet successfully meet up at later points with the target"

dudus• 12 years ago

Allow Law Enforcement Officials to remotely control driverless vehicles in emergency cases. There I fixed it.

ChuckMcM• 12 years ago

Pretty typical right? And your bank robbery getaway car could be orbiting the bank waiting for you to come out, etc etc. The report boils down to "gee a self driving car should would be useful to crooks." which is sort of obvious.

That said the benefits for non-crooks are a lot higher. And while roads in general benefit crooks getting away in cars, most of the people on them are just trying to get somewhere.

Zigurd• 12 years ago

This is scare-mongering, likely for the purpose of reducing privacy and personal rights in using autonomous vehicles for personal transportation.

There is enough open information regarding autonomous vehicles that anyone with disregard for safety could build an autonomous car bomb now. As with other supposed terror threats, the only thing preventing them is that nobody is doing it. But that doesn't sell anti-terrorism brainstorming consulting.

DanBC2• 12 years ago

Stories like this are useful. Not for the details they contain, which are scant and make me think a lot of people in olved are stupid, but because they give "us" a chance to prepare for the inevitable accidents or criminal uses of this good tech. One day a child is going to be oilled by a self-driving car. That will be very sad for that one child, but we need to be able to point to the thousands of children who were not killed or maimed. Getting that message right seems important.

ianstallings• 12 years ago

Passenger: "Uh, car, the bridge is out."

Car: "I know Luke" speeds up to jump it

Passenger: "Who's Luke"?

Car: "YEEHAW!!" hits the rebel horn

But seriously, FBI analysts get paid to imagine all types of future scenarios let's not hold it against them too much. It wouldn't surprise me though to see more anti-electronic gear, such as EMP, in the field as we go forward. They already have devices that can shut down a car, I imagine they'll work on these new ones just as well and the FBI simply wants to prepare for it.

FBI: Google's driverless cars could be lethal weapons

Related Stories