Tesla's Autopilot 'tricked' to operate without driver

https://www.bbc.co.uk/news/technology-56854417

by Black101 • 5 years ago

21 27 5 years ago

bbc.co.uk

f6v• 5 years ago

Let me guess: was it a mannequin with its hands taped to the wheel?

thu21112• 5 years ago

Yeah. This is a ridiculous story by the BBC - it says the car can easily be tricked and then doesn't explain how. Surely that is the primary set of facts that are needed to evaluate the story?

capableweb• 5 years ago

If Tesla hasn't implemented a fix for the problem yet, it's possible that BBC is choosing not to share the trick, as then more people would do it. Similarly to how we have "responsible disclosure" of security issues, or not writing too much about suicide in media as it'll increase overall suicides.

throws235773• 5 years ago

BBC just copied the consumerreports story. It has all the detail on how they did it. It's linked from the BBC article.

https://www.consumerreports.org/autonomous-driving/cr-engine...

throws235773• 5 years ago

No mannequins.

https://www.consumerreports.org/autonomous-driving/cr-engine...

Black101• 5 years ago

most cars that I drove had a weight sensor in the seat, but I guess it is just as easy to defeat...

gibspaulding• 5 years ago

This was what I was picturing:

https://xkcd.com/1559/

renewiltord2• 5 years ago

Guys, isn't this obvious? I can also "trick" a knife into stabbing me instead of cutting this carrot.

This is like the basic thing of giving your users power over the product they purchased.

What next? All cars can be tricked into driving into other cars with merely some steering input? "Researchers concluded that with a mild flick of the wrist, any driver could flatten 40 children in a preschool yard. Activists have called for wrist flick regulation over the years but their appeals have been ignored by all manufacturers except Daewoo - whose cars do not respond to steering input at all"

weare1382• 5 years ago

If my Honda can tell when I'm sitting in a seat but don't have my seat belt on then it seems like this is an easily solvable problem.

londons_explore3• 5 years ago

Tesla's already do this, but it's easy enough to fool - just put a bag on the seat and put the seatbelt round it.

renewiltord2• 5 years ago

A basketball player waiting for the draft literally died yesterday because he fooled his car into thinking he had the seat belt on.

msrenee• 5 years ago

You just brought back memories of the Subaru I had that thought my backpack was a passenger. I'd throw it on the passenger seat after class sometimes without thinking about it. Then about 10 minutes down the road, it would shift and wind up triggering the sensor. So I'd be having to reach and finagle the seat belt to shut off the distracting beeping while navigating a busy stretch of road where everyone likes to drive like idiots. Very safe. Luckily the previous owners of my current vehicle were kind enough to disconnect the seat belt chime and a few other safety features that I sure haven't missed. And yes, I always wear my seat belt and so do my passengers.

Ah, the old days of buckling up back packs and fish tanks, and pretty much anything else I didn't want to throw in the back that weighed over 30 lbs.

londons_explore3• 5 years ago

A standard car will also drive with nobody at the wheel. Just put it in drive and hop out...

If you want more danger, put a brick on the accelerator.

kgin• 5 years ago

Totally, and if anyone was selling bricks called “Full Self Driving Bricks”, they’d probably be in trouble.

fgonzag• 5 years ago

Ever heard of cruise control?

londons_explore3• 5 years ago

Could the real issue here be that this technology is "too alluring".

In the same way some medicines are regulated to stop people self medicating, perhaps cars need to be made less tempting to be stupid with...

For example, perhaps a $1000 deposit that you lose if you are ever found to not be paying attention at the wheel?

MomoXenosaga• 5 years ago

There are more accidents happening with smartphones behind the wheel.

Drunk driving and not wearing a seatbelt became socially unacceptable after decades of horrific accidents (I recall a story of a drunk driver in Belgium plowing through an entire class of school children).

gruez• 5 years ago

For example, perhaps a $1000 deposit that you lose if you are ever found to not be paying attention at the wheel?

...or a $1000 fine?

loa_in_• 5 years ago

> if you are ever found to not be paying attention at the wheel?
...or a $1000 fine?

It's a great idea except the fact that it's unenforceable and will eventually be abused. Whether or not someone has been paying attention is a transient property you can't prove or disprove after the moment has passed.

throws235773• 5 years ago

Hi dang , please change the URL to [1]. The current BBC one has no detail at all.

[1] https://www.consumerreports.org/autonomous-driving/cr-engine...

tmptmp217• 5 years ago

If it's in beta then wouldn't always-on cameras be beneficial; the data from facial expressions before a potential crash might overcome the lack of users due to opt-out? "By comparison, BMW, Ford, GM, Subaru, and others use camera-based systems that can track the movements of a driver’s eyes and/or head position to ensure that they’re looking at the road."

savanaly• 5 years ago

Fisher engaged Autopilot while the car was in motion on the track, then set the speed dial (on the right spoke of the steering wheel) to 0, which brought the car to a complete stop. Fisher next placed a small, weighted chain on the steering wheel, to simulate the weight of a driver’s hand, and slid over into the front passenger seat without opening any of the vehicle’s doors, because that would disengage Autopilot. Using the same steering wheel dial, which controls multiple functions in addition to Autopilot’s speed, Fisher reached over and was able to accelerate the vehicle from a full stop. He stopped the vehicle by dialing the speed back down to zero.

If anyone is curious how to do it. From https://www.consumerreports.org/autonomous-driving/cr-engine.... Personally I wouldn't call it too easy to do; That summation makes me think it would be possible on a whim with no forethought (surely a wealth of the scenarios where someone would choose to do this), but looking at the picture in the article they needed a setup with a large weight (but not too large) tied around a length of rope (also something I don't have available on demand in my car and would have to go somewhere and buy).

I'm not saying a drunk idiot at home with some unusual materials couldn't do it but it's not as simple as something you can do anytime you feel like it.

On the topic of better systems to get the driver to look at the road: "By comparison, BMW, Ford, GM, Subaru, and others use camera-based systems that can track the movements of a driver’s eyes and/or head position to ensure that they’re looking at the road. Some vehicles—including those equipped with GM’s Super Cruise—can automatically slow to a stop if they detect that drivers have ignored repeated warnings to look at the road."

We shouldn't forget that there's a lot of prior art on this area of human-machine relations. One place where we've had automated driving for ages is in train/subway and getting their conductors to pay attention even when they don't have anything to do for hours on end is something that's been struggled with. I recall that they've tried things like adding a button that they have to press periodically, but it doesn't work (the brain still can't help falling asleep and it will automatically interact with the button). Same for other more advanced things like making them input some more complicated action to show they're awake. The brain's just too good at ignoring things it realizes aren't necessary. I think one of the best solutions, maybe the only solution that's been foolproof, is https://en.wikipedia.org/wiki/Pointing_and_calling. Not sure how we would utilize that idea for self driving cars though.

djanogo• 5 years ago

I am still sticking with my previous guess...

"I am guessing after hitting the tree the doors got jammed, so the driver got in the back seat to try to force open the back door or trunk access. They burned alive."*

This explains why front passenger was occupied (airbag and no hands to brace for impact - this person was probably was still in stock), the driver braced using steering and got in back seat to open doors, but the smoke/fire killed them both soon.

*https://news.ycombinator.com/item?id=26871493

Tesla's Autopilot 'tricked' to operate without driver

Related Stories