Self-driving cars turned out to be harder than expected

https://www.vox.com/future-perfect/2020/2/14/21063487/self-driving-cars-autonomous-vehicles-waymo-cruise-uber

by apsec112 • 6 years ago

204 277 6 years ago

bravoetch• 6 years ago

This meme of shock and surprise that self-driving cars haven't appeared overnight is just media BS. Yes, all technology development is a gradual process and we may not know how long it takes. Companies working on it have pitch decks and investors so they give their best or most optimistic estimates.

ghaff13• 6 years ago

Go back a couple of years and look on just about any thread about self-driving on a forum such as this one and you'll find no shortage of people arguing that they're just around the corner. Because, after all, that's what the SV hypesters were saying and they'd never lie.

seanmcdirmid6• 6 years ago

you'll find no shortage of people arguing that they're just around the corner

Ya, but around the corner no one thought that would be 2020. The strawman is always to place around the corner as tomorrow, but many people just mean in a decade or two.

heynk• 6 years ago

When I was starting my engineering career, it was right around when the first DARPA challenges had started. The hype was beginning, and my optimism towards technology was strong. I thought the predictions and timelines would be correct, and I still feel strongly that self-driving will be safer than humans in the long term.

Recently, I bought a newer Subaru, with EyeSight. It has adaptive cruise and lane keep assist. The LKA is fine - it'll beep if you sway outside of a lane, and automatically adjusts the steering, but it won't keep you centered. It's more of a safety thing, and it works well from that perspective.

The adaptive cruise is really good. It's camera based, and I have had zero problems with it. It works well at night and in pouring rain. It'll even stay pretty close to the car ahead of you if you turn the "tolerance" all the way down. I'm always impressed.

Since I've had this car, I've thought a lot more about the practical implementation details of actual self-driving. I more often notice situations when driving that are seriously complex.

The more I think about it while I'm driving, the more I realize how fucking hard self-driving would be.

OnlineGladiator• 6 years ago

The adaptive cruise is really good. It's camera based, and I have had zero problems with it. It works well at night and in pouring rain. It'll even stay pretty close to the car ahead of you if you turn the "tolerance" all the way down. I'm always impressed.

I checked, assuming it's actually using a radar - and you're right. They seem to use a stereo camera system. Neat.

https://www.subaru.com/engineering/eyesight.html

I'm surprised it works well in bad weather, but I've never tried it.

scotje• 6 years ago

It's not impervious to bad weather, but pretty resilient. I'd say in the 2 years we've had ours the system has shut off maybe 3-4 times due to one of: a) very low and direct sun angle, b) very heavy rain, c) dense fog. Which, to be fair, are all difficult conditions for a human to drive in as well.

But I agree with the parent, the suite of driver assistance features is very good, but a long way from "self driving".

Shivetya• 6 years ago

I see the current Tesla system in similar light, it does very well in every day common situations and some of what it does is damn good; driving down country roads with curves and such is exhilarating but still safe.

Now currently the Tesla system does give you a much more clear idea of what the car does see around it but still no option to see what it fully records; there are means to get this footage but its not something every driver can.

Now my TM3 goes in at the end of the month for the hardware 3.0 upgrade which will allow it to process more of what it sees and also relay that to me. The difference in what other's have show in just what the car relays back to the driver exposes just how much information has to be processed.

Then comes the simple fact, the real issue is that the hard decisions are ones we make all day, driving by exception. We make so many choices that are exceptions to the rule we are numb to it, it is nearly subconscious.

Then the other issue, other drivers. Not just people who drive badly but those who will go out of their way to cause self driving cars issues. with the number of people on the road you will find them with too much regularity. More might pop up if regulation comes down which demands self driving cars or semi autonomous cars obey all traffic laws, especially speed limits. On some roads I drive just obeying the limit is enough to impart rage on other drivers.

learc83• 6 years ago

Recently, I bought a newer Subaru, with EyeSight. It has adaptive cruise and lane keep assist. The LKA is fine - it'll beep if you sway outside of a lane, and automatically adjusts the steering, but it won't keep you centered. It's more of a safety thing, and it works well from that perspective.

I have 2020 Subaru and it has lane centering on top of that. On the highway, with clear lane markings it comes very close to driving itself. It won't slow down to handle curves on its on though.

close044• 6 years ago

Today saying cars have "self driving capabilities" is like saying you're fluent in 3 words of a language. They have advanced driver assists but the insistence on the "self driving" terminology tricks enthusiasts and less tech savvy people alike into a false sense of confidence in the tech. Sometimes all the way to their deaths.

skywhopper3• 6 years ago

Ultimately, the right place for most carmakers to focus on at this point would be situational awareness and safety features, gradually improving the situations where the car can prevent a crash.

Put it this way: If a driverless car would be safer than human drivers, then that would imply that all the necessary technology would already exist to allow humans to be the driver while the car still keeps them out of deadly situations. If such tech is not possible to develop, then it seems unlikely that true driverless tech (which would need to combine that safety tech with a lot of other technology) will happen.

ghaff13• 6 years ago

I'm inclined that improving systems up to full autonomy for many highways in many weather conditions is a fairly realistic maybe 5-10 year plan. Which would actually be pretty nice and potentially a big win for safety.

The problem with widespread L4/5 is that you need to get to a car that can literally drive itself between 2 points on a map with a high degree of reliability, in a wide range of weather conditions, on roads of varying conditions, with unexpected/unmapped obstacles that may require doing something technically illegal to get around, without human help. And that, as you say, seems really hard.

slavik81• 6 years ago

I tried out a relative's Subaru on a 6h drive over the holidays. I really liked the adaptive cruise control for following behind folks who were not keeping a consistent speed. I just set it to a reasonable value at the maximum following distance and stopped worrying about my speedometer.

However, at one point the guy in front of me turned off onto a small side road. It was at night, and I don't think the car realized he had moved into a turn-off lane. It slammed on the brakes. I probably went from 90kph to 40kph before it realized I was not going to hit that car.

I completely failed to react to the situation. I was worried my erratic braking would cause an accident behind me, but in the moment, I didn't know how to stop it. That was not a type of emergency I had considered or prepared for.

bdamm7• 6 years ago

Yeah this is interesting. My Tesla M3 does similar behavior, and so I often am ready to punch the accelerator. In the Tesla, this is how you solve that problem. The driver's push on the accelerator contraindicates the AI's decision to slow down, and so the car follows the driver's direction.

Where it gets dicey is the scenario where the "imminent collision" (hazards on, seatbelts tightened) detection is triggered, and the driver continues to push hard on the accelerator. Tesla has a fairly lengthy statement in the manual about this scenario. The bottom line is there are all kinds of heuristics at play that may or may not result in an override depending on the specific sequence of events.

wffurr• 6 years ago

Same thing, first time I used the Subaru EyeSight system. Now I know to pay attention for that particular failure mode, and override with the gas pedal and a little steering.

Definitely surprised the heck out of me though the first time the car slowed way down on the interstate because the car ahead of me pulled onto the off-ramp.

LoSboccacc2• 6 years ago

you haven't had a issue with it. multiple people died already.

rootusrootus5• 6 years ago

Earlier today I was driving on the highway with autopilot (I have a Model 3) and came to a section where the road is angled in such a way and the pavement is old enough that there is a fair amount of standing water. Driving manually, I steer to the right or left slightly to avoid the ruts filled with an inch or two of standing water. Autopilot, on the other hand, was perfectly happy to blast right through it.

That's the kind of weird edge case that makes me think we're farther from real self-driving than most people want to admit. I'd be hard pressed to define exactly how I'd tell the computer to avoid that. Maybe the answer is that it can't deal with that until it results in hydroplaning, and then it reacts however it can.

catalogia4• 6 years ago

That's a particularly insidious circumstance since standing water can conceal hazards from any vision system these cars or humans have. I would expect self-driving cars to refuse to drive through water in any circumstance. There could be a large pothole in the puddle that would ruin your car.

Worse than a mere car-destroying pothole, what if the flooded portion of the road no longer existed at all? That's a common enough occurrence that student drivers are generally warned about it specifically, warned to never drive across flooded sections of roadways because your car might fall into 10 feet of water without warning. If a self-driving car doesn't avoid a scenario we teach teenagers to be wary of, I don't think it deserves to be called self-driving.

Johnny5555• 6 years ago

You have local information that cars don't yet know (but they probably will someday -- cars can send detailed road conditions to a central database, or they can communicate with other cars, so the car in front of you can say "watch out, there's a big pothole 8 inches from the left lane line" and your car will try to avoid it.

What do human drivers do there when they are unfamiliar with the road? Seems like the auto pilot should be able to do at least no worse than human drivers.

Mattasher• 6 years ago

You have to wonder, when you see these stories, whether the engineers in charge of these programs spent much time driving around, trying to see the road like an AI might. In my experience, driving in the city, it's rare that I can go more than a few miles before some exception comes up that requires human judgment. Here's a few of the things I saw just last week:

* Car double parked. Do I cross the center line to go around them?

* Stop sign that's been bent and it's no longer obvious which street it refers to.

* Semi with its hazards on in a left turn lane. Do I make a left turn around it?

* Cop directing traffic by hand at an intersection.

Most city driving is a series of continual exceptions to the rules, or situations that are one-off. Who thought this would be easy?

dstroot• 6 years ago

Cop directing traffic by hand at an intersection.

That’s going to be really challenging. Good one!

bentcorner• 6 years ago

Google (Waymo?) has been thinking through these exceptions: (around the 9:45 mark) https://www.ted.com/talks/chris_urmson_how_a_driverless_car_...

mark-r4• 6 years ago

I ran into this on my commute just last night. Cop car in the left shoulder, lights flashing. Cop on the right side of the road, waving a flashlight. Oncoming traffic crossing over into my lane and blocking my way. Finally I figured out that the cop was directing traffic and I had to wait until something changed. Eventually the oncoming traffic dwindled, and the cop turned around so I had a harder time seeing the flashlight - I assumed that meant it was my turn to go. I approached very slowly.

I never figured out what the problem was, this was in the middle of the road and not at an intersection. I was more concerned with doing the right thing than trying to determine the root cause.

pasttense018• 6 years ago

The cop can carry a transmitting device which will communicate with cars telling them what to do.

PeterisP• 6 years ago

It should be possible for current technology to (a) detect a cop in the middle of intersaction; (b) stop safely before the intersection and (c) delegate control to a human - possibly someone in the car, possibly a remote one.

ghaff13• 6 years ago

At a talk a few years back MIT prof John Leonard showed video he took on his commute back and forth to Brookline to highlight all the points where it was going to be really hard to get self-driving to handle. (He was very skeptical about getting to full autonomy before it was fashionable to be so.)

It's not just cities but spend a few hours walking around Boston or Manhattan and just make a note of all the crazy things drivers do. (Sometimes by necessity.)

(Unprotected lefts in busy traffic are one challenging area. There's a certain social aspect to it and the right decision in one set of circumstances is either way too aggressive or conservative to the point where you'll never turn in another.)

andrei_says_• 6 years ago

This one? https://www.youtube.com/watch?v=GarvfAPSWwo

...How do you write the code for "Always stop at a red light unless you see a police officer waving you through?"

dhosek• 6 years ago

* Typical school pickup/dropoff situations where cars will line up, often in the parking lane, to get to the pickup/dropoff spot. Sometimes the line goes around a corner.

grecy• 6 years ago

And the amazing part is there are millions of drivers on the road right now that have absolutely no idea how to handle those cases, and do an exceedingly bad job of it.

From the very new drivers, to people who simply don't drive often, to people who got their license in another country where the rules are entirely different (me!), to the very elderly who are losing their sight, reflexes and judgement and people playing with their cell phone or otherwise seriously distracted.

It's a wonder only 35,000 people are killed on US roads every year.

jacquesm6• 6 years ago

It's a wonder only 35,000 people are killed on US roads every year.

do an exceedingly bad job of it.

No, they're doing an amazingly good job of it. Given the cumulative number of miles driven and the conditions driven in it should impress you rather than that you come to the conclusion they are doing a very bad job. If the current crop of AI would be unleashed in those same conditions the carnage would be unbelievable. Humans are very well suited to adapting to changing environments, it is the one thing that we have over the rest of the animal kingdom. From hanging from tree limbs to riding on the autobahn with the same software.

fuzzfactor• 6 years ago

Not long ago there was a company which failed to perform the relatively impossible themselves, and when they ceased operations their message was clearly _we could not automate that which we could not do that well manually_.

I think there's an abundance of drivers that are almost always unsuitable for the most challenging situations, or for whom the roadway or conditions encountered are completely unsuitable themselves.

Collisions are everywhere and are dwarfed by near-misses. Fatalities are statistically limited by hazard mitigation, safety measures, and a seemingly larger portion of miraculous good fortune.

Some drivers are completely out of control and some even like it that way. There is a much smaller amount of mechanical or health failures.

These all can be the riskiest due to unpredictability.

We will have to admit that each out-of-control driver can exhibit perhaps completely unexpected and unique behavior from each other, because that's what they've always done.

That's not a very high bar for different out-of-control AI events by comparison.

Tragedies occurring from individual human driving deficiencies have been largely attributed to social effects, because the underlying mechanical engineering performance has been so overwhelmingly predictable by comparison.

With AI different unforseen tragedies are to be expected, and it will not be due to any recognized or imagined individual human driver deficiency. Attribution will be squarely on the engineering team deficiencies when it comes to the programmable electronics that were overlaid onto the fundamentally predictable 20th-century-proven mechanical platform.

There could be deaths that can not be dealt with using natural human social constructs.

Being killed by an out-of-control robot is always going to be something that's supposed to be nearly impossible to occur, compared to being killed by an out-of-control human.

Anyway, with emerging programming technologies sometimes actually achieving highly suitable goals which were only reachable when combined with ideal support, and a sales/advertising mentality that can make up the difference when the incredible engineering effort has done all it can and the goal is not quite met nor suitable, everything's looking better than ever so what could go wrong?

derefr4• 6 years ago

IMHO cars are for getting between cities, or between places in non-cities; within a city, the economical long-term solution is to just replace the roads (save for cargo corridors) with some kind of pod rail network, i.e. self-driving four-seater trolleys. The exceptions become far fewer if there's nothing on the "road" but other rail-bound vehicles.

In such a future, if you lived in a city but owned your own real car, there'd be no road to use to drive it from your house to the city limits; and so it'd have to live at the city limits in a garage—just as people with personal planes have to keep them in a rented hanger at an airfield. You'd take this fancy pod-transit to the City Common Car Park, summon your car down from storage, and then get right onto the highway.

seanmcdirmid6• 6 years ago

some kind of pod rail network, i.e. self-driving four-seater trolleys

If you go that far, the problem is already trivial right? Just making roads specifically for self-driving vehicles would make the problem trivial. What the companies are trying to accomplish is self driving cars in current conditions.

jcranmer2• 6 years ago

just replace the roads (save for cargo corridors) with some kind of pod rail network, i.e. self-driving four-seater trolleys.

So Personal Rapid Transit? This idea has been tried before, and has never lived up to expectations.

taco_emoji• 6 years ago

I honestly can't picture a self-driving AI that doesn't just amount to straight-up General Intelligence. Not only does driving involve predicting the trajectory of various moving objects (already very hard), it involves predicting the behavior of various other intelligent agents as well as communicating with them.

Especially that last one -- nobody seems to consider how much we communicate while driving. Several types of communication require having a body: flagging someone on, waving "thank you", eye contact, LACK of eye contact ("they don't see me, I better [prepare to] brake/swerve"). Some involve subtle movements of the car: rolling forward after a stop sign ("I intend to go next"), or hugging one side of lane. Even something as apparently binary as beeping the horn can indicate probably a dozen or more different things depending on physical context and length of the beep, not to mention local culture.

clairity• 6 years ago

i would love to see mechanical eyes on the front of self-driving cars to signal emotion/attention, like from the pixar cars movies. =)

jcranmer2• 6 years ago

For a mundane situation that's surprisingly difficult: toll booths.

The highway goes from N lanes to N5 (or more!) lanes and then back into N lanes, usually with the lane markings being completely absent in the process. You have to pick a lane to get into, with your choice being potentially impacted by such things as "are you a truck?", "exact change only", or "I have an EZ-Pass", not to mention the potential for some lanes to be outright closed. The signage is likely to be somewhat unique and potentially counterintuitive to normal meanings--the lane under the big yellow light is the kind of lane I usually want to take, especially since it's usually higher speed than the lane under the big green light. Drivers are going to be more erratic than usual ("oops, I don't have EZ-pass" SWERVE*). And they usually occur smack-dab in the middle of limited-access highways where people might want to use self-driving cars.

mkl• 6 years ago

Put spaces around the * in "N * 5" to avoid the italics (or just write 5N).

tptacek• 6 years ago

I'm a self-driving skeptic but the toll booth problem doesn't seem logistically all that daunting, since there are so few of them compared to overall road miles. It seems like they could be modeled individually and directly.

pasttense018• 6 years ago

Self driving cars already use a map of the roads in the country--all you do is include the toll booths in the map.

They could also include a transmitter to vehicles.

KaoruAoiShiho• 6 years ago

I think you can reasonably expect a FSD to perform on the level of a human. Asking it to go above and beyond that is too much. When it comes to situations that give humans trouble you would rely on the same extra tools and processes that humans use.

* Car double parked. Do I cross the center line to go around them?

* Semi with its hazards on in a left turn lane. Do I make a left turn around it?

You would err on the side of caution, but ultimately you would go with the help of 5G remote assistance so that the passenger can stay napping.

* Stop sign that's been bent and it's no longer obvious which street it refers to.

How would humans handle this? They would rely on previous knowledge of the area, aka maps could help.

* Cop directing traffic by hand at an intersection.

Not challenging at all imo, this should be trainable.

sjburt• 6 years ago

Than who expected? The criminally credulous tech media?

scottlocklin2• 6 years ago

You mean the output of VC firm and Silly Con Valley marketing centipedes isn't strictly true? Perish the thought! Next thing you know, you'll be telling me the advocates of strong-AI, 3-d printing, Virtual Reality and Drones as revolutionary technology might not be exactly accurate.

chungus_khan• 6 years ago

Turns out you can't make ridiculously complicated autonomous tech that interacts with the real world in an incredibly nuanced and varied way out of pure VC hype.

ghaff13• 6 years ago

I do wonder how much of the self-driving narrative at the peak of inflated expectations was the result of

1. Wild optimism fueled by how rapid the progress of ML had been in certain domains over the course of a few years combined with a lot of hype and general SV techno-optimism. (And the fact that a certain demographic so desperately wanted a robo-chaffeur to drive them around.)

vs.

2. Hypesters and scammers who knew it was mostly smoke and mirrors but it didn't matter so long as they got their payday.

scottlocklin2• 6 years ago

In the case of 1. I'm certain a whole bunch of it is people getting high on their own supply; aka pay a marketer to hype your thing, your competitor does same, then you become afraid of all the progress your competitor has made (which is imaginary marketing hype). I've actually seen this dynamic at work in "AI" land; it's gotten me work.

I've even seen it happen within the same company: marketing dude talks to engineer, gets it all wrong and exaggerates capabilities; then CEO demands to know why the capabilities in his sales literature don't exist in the product.

I call it "human informational centipede."

Hypesters and scammers always around; excepting in Theranos type cases, they mostly don't really push the needle. Pretty sure Irene Aldridge didn't change perceptions of HFT much, for example.

spiderfarmer• 6 years ago

I have a feeling the adoption of AI will be like the early years of the internet. Massive hype, then a massive crash and then slowly it finds its into everything, fulfilling most of what was promised.

flyGuyOnTheSly• 6 years ago

So do a lot of ventures but that's no reason to throw in the towel.

csours• 6 years ago

Disclaimer up front: I work for GM, I don't work on SDC.

As I see it, self-driving is a cursed[0][1] problem. If you could choose a different problem space, or to ignore some complications, self-driving would merely be very hard. But the requirement to handle ANY external behavior with ANY external conditions and navigate to ANY destination, while also maintaining safety is impossible* satisfy.

One cursed corner of the problem space is ML itself: ML is amazing in that it enables emergent behavior [3], but ML is terrible in that it gives rise to emergent behavior. The traditional engineering mindset wants a map of inputs to outputs, but you don't get to choose your inputs in the SDC world, and you can't specify all of your outputs.

Another cursed corner is the Always/Never[2] problem. You want safety features like Automated Emergency Braking to Always kick in when there is a problem, and you Never want them to kick in when there isn't a problem.

I really don't know how any of this gets fixed. I do think that sensor fusion and advances in AI can reduce the size of some of the cursed area of the problem, but the problem is also meta-cursed in the definition of Self-Driving: the solution to normal cursed problems is to reduce or change the solution scope, but if you reduce or change the Self-Driving solution scope, then "it's not real self driving".

0. https://twvideo01.ubm-us.net/o1/vault/gdc2019/presentations/...

1. https://en.wikipedia.org/wiki/Curse_of_dimensionality#Machin...

2. https://www.newyorker.com/news/news-desk/always-never

---

3. edit: [pdf] https://core.ac.uk/download/pdf/81580604.pdf Especially see section 4.3

jacquesm6• 6 years ago

This is spot on. The Always/Never is well articulated, it is exactly why I got rid of a vehicle with auto braking. It did it twice when it wasn't supposed to and that was more dangerous than if it had never triggered at all. And yet, in an actual imminent crash it would have probably responded a lot quicker than I ever could. Still, I'm not going to be scared twice in a month by my car doing stuff 'on its own' to my possible detriment.

GeorgeTirebiter2• 6 years ago

Could you share which company's auto-braking had the false positives? What is the acceptable false-positive threshold for you to use auto-braking again? Also, did your system tell you why it decided to brake (albeit incorrectly)?

erikpukinskis• 6 years ago

Waymo and Tesla have both solved the problem, in different ways, by gating the feature.

Why would they need to force Level 5? They can operate with the promise of L5 in the future, until the moment the statistics show it’s safe to turn it on.

Between now and then they can provide a steadily increasing supply of value to their customers. And steadily ramp the data they have to improve the system.

I have no idea if it will be 2 years or 10 years or 100 years before they can turn on Level 5 without a geofence.... but what difference does that number make? It’s academic. It’s not an existential question for either company’s self driving value prop.

And consider that different jurisdictions will set different dates. All it takes is one government to OK L5 in some area, and Tesla’s ass is covered. And every jurisdiction that signs on will put more pressure on other jurisdictions to follow suit.

To me, that’s the brilliance of Tesla’s strategy. They have decoupled their forcing functions from the L4/L5 legislative question. Governments can drag their feet as long as they want but it won’t hurt Tesla’s ability to sell the tech or collect the data they need to improve the tech. They are already selling autonomy features, and they can make that value prop incrementally better year over year for as long as it takes.

GeorgeTirebiter2• 6 years ago

Sensor fusion? Absolutely required. Also, I think "AI Fusion" is needed, where different AI approaches evaluate a situation. Maybe a supervisor DNN AI which is looking at the output from the AIs below, inferring patterns.

DNNs (alone) are incapable of Level 5. OTOH, it's conceivable that in this limited domain, a "Society of AIs" might yield 'acceptable' performance ( 1K deaths per year? ).

I would also like to see a given driving system rated at maximum speed for autonomous use. If you're moving at 5 MPH and your stopping distance (reaction time + coming to a stop) is 5 feet, then your 'Zone of Despair' is the 5 feet in front of you (and some on either side). If you absolutely know this Zone is clear, you can proceed with confidence.

It's a hellishly difficult problem; but I don't think it requires AGI to solve 'adequately'.

takk3095• 6 years ago

I am a traffic engineer and one of the things that I do regularly is write 10 year traffic plans for small to medium sized cities, think 60,000 people max. When a lot of the self driving hype was really kicking off, I was told to make sure to incorporate them into the plans. This mostly consisted of a few sentences about self driving and that was it. When modeling future traffic growth, the client would always say, "these numbers a pretty high, do you thing self driving cars will lower them?" To which I would respond that it is doubtful. I am in the rural west where cars are a large part of life. Ride sharing and public transportation are rarely a thing outside of the core of towns. The idea of a self driving car that one doesn't own would be very odd to a lot of people in this area.

On top of the social side of things, the roads are not in great shape in northern climates and many of the visual cues that we use to drive can be missing or very hard to see for many miles. Striping delineating the edge of the road often gets worn away over the course of a few years and doesn't get re-painted for a few more. Some major roads connecting two towns may not even have a paved shoulder, just 24 feet of asphalt with a stripe down the middle. (For reference, 12 foot lanes with 4 foot shoulders are the general norm for this part of the US.) All of this and I have still yet to touch on weather.

I look forward to self driving cars. However, I don't think that they are going to solve many of our traffic issues outside of urban cores. For me, the incremental steps to reach self driving will result in fewer injuries and fatalities on our roadways, and that is a win.

chrisco255• 6 years ago

Yeah just collision detection and lane guidance are already helpful in modern cars. After driving in snow in Colorado I have to wonder how self driving cars perform under such conditions. I couldn't make out where the lines are and no one really does. A four lane road can easily become a two lane road under such conditions, as people cut a worn path through the snow.

I really think self driving could improve public transportation in urban cores by tracking preset paths. I could also imagine buses that are mostly autonomous, but where a remote driver could override the controls for exceptions.

throwawayhhakdl• 6 years ago

A lot of people are incredibad at driving in snow. I would not be surprised at all if self driving cars vastly reduce the accidents attributable to snow on the road. Some of that might be due to the car deciding something is too dangerous to attempt, and frankly that seems fine to me. If your problem is “the car can’t handle x condition”, there’s really no issue so long as that’s because the car refuses to try, rather than trying and failing with non trivial probability

manmal• 6 years ago

It will also restore mobility for many people who cannot drive yet or not anymore. Where I live, Taxis and Ubers are still too expensive and not ubiquitous enough to replace an owned car.

takk3095• 6 years ago

This is one of the most important use cases I can see for self driving cars. As it is, on demand transit is expensive and in rural areas not the most frequent. For example the northern portion of Montana has a very low population spread out across a large distance. For many elderly to get medical care they must travel 100-200 miles to the nearest hospital. When they have to make this trip every week, I can be a massive burden on their loved ones. On demand transit is an option but may only run a couple times a week due to limited drivers. A self driving vehicle would help this situation a lot.

Johnny5555• 6 years ago

I'm counting on self-driving cars to be viable by the time I'm too old to drive -- it was very hard for my grandmother to give up her car and it took a lot of coaxing (and a little sabotage of her car to make her think it needed expensive repairs) to get her to stop driving. But it was necessary because she was becoming a dangerous driver.

skywhopper3• 6 years ago

I'm always baffled by folks who assume self-driving cars would reduce traffic. No matter what, even if they are privately owned, truly self-driving vehicles could only increase traffic. If they can be dispatched without a human driver to pick up groceries or take-out, for example, there'll be a lot more trips. And sharing self-driving cars rather than owning them would surely be more likely to increase the number of cars on the road since they have to make a trip to pick you up and another one after they drop you off.

takk3095• 6 years ago

The rationale I have heard is that a single car could be used to make a multiple chained trips instead of multiple cars making multiple trips. This is the same idea behind ride sharing. I feel this breaks down with privately owned vehicles. I do see an increase in traffic in areas with expensive parking. Specifically I would sent my car to a cheaper lot and that would result in an additional trip. Even worse, that would be a zero occupancy trip.

ctdonath2• 6 years ago

I envision herds of driverless vehicles slowly cruising randomly, summoned when needed, avoiding the cost of parking.

seanmcdirmid6• 6 years ago

"these numbers a pretty high, do you thing self driving cars will lower them?"

My answer would be no: the opposite, if self driving cars work out, people will probably just drive more! The only traffic gains would come from 100% self driving cars that could then be optimized somewhat globally.

xiaolingxiao2• 6 years ago

hey Takk309 could I reach out to you to ask a few question? I am researching what it takes to validate the safety of these cars on the street in relation to other cars, and pedestrians, and your background would be immensely helpful! Feel free to email me at lingxiao@seas.upenn.edu

takk3095• 6 years ago

That is going to be a tough but to crack. Crash statistics are tricky because they are either underreported or very infrequent. Two good resources for just about anything traffic related are ITE [1] and TRB [2].

[1] https://www.ite.org/ [2] http://www.trb.org/

skywhopper3• 6 years ago

Billions of dollars of VC funding go a long way to convincing people of what is possible, whatever the reality is.

jacquesm6• 6 years ago

Expected by who?

bnegreve• 6 years ago

Apparently by General Motors, Google Waymo, Toyota, Honda and Tesla.

jacquesm6• 6 years ago

All of those except for Tesla have never made hard claims about being able to deliver full autonomy in the near future as far as I know.

tjpaudio• 6 years ago

To anyone with a detailed understanding of the current limits of machine learning, this should be no surprise; unsupervised learning is far from solved and ML in its current state will always be plagued by the cat and mouse game of edge cases. The reality is the industry has decided to go this way anyways because it has a good profit outlook if you can get it to work, which is the only thing funding the endeavor. Consider two ways of going about automated transportation:

1) AI. The car independently makes decisions and drives itself. 2) Networks. The car communicates with a grid to make decisions.

Why did we go for #1? Well, thats easy, capitalism. Consider:

AI: - Company gets to own the intellectual property to form a temporary monopoly. - Easier to sidestep governments involvement. - No need to build large infrastructure

Networks: - Shared, less opportunity for monopoly formation. - Will need the government to cooperate. Governments are slows.

If I had to guess, we will eventually go the network route. The research used for AI will drive safety features and failsafes, but not the meet of it. Anyways, why accelerate a line of stopped cars one at a time with autonomous vehicles when you could accelerate the entire line of cars simultaneously with a networked setup?

wilg• 6 years ago

I think #1 is easier for lots of reasons.

streetcat1• 6 years ago

So I do not think that this is an unsupervised problem. As a matter of fact, this should be one problem where labeled data is not an issue.

I.e. just let humans ride the self-driving car, and record the human actions as well as all the sensors (at time t-1).

Combined that with all the humans on road X, and you have the a model for road X.

The problem here is that each road needs its own model as well as model per time of day.

Another problem is that each company will not share its supervised data.

pasttense018• 6 years ago

And what happens if the network breaks down? Everything gridlocked. Conversely if one vehicle breaks down the rest of the cars just go on by.

krm01• 6 years ago

It’s really just a matter of time. Those that started out optimistic (when it was hyped) but are now saying that they don’t see self driving cars can do the complex situations we humans supposedly can should have some patience. As the tech improves, and more data is used to train models, it’ll surely surpass humans in driving ability.

jacquesm6• 6 years ago

If only it were that simple.

mark-r4• 6 years ago

I have this argument constantly. Humans make mistakes too, so we will eventually have tech that makes fewer mistakes than humans. The problem is that humans have a unique ability to make sense of situations they've never specifically encountered before. No amount of training data will be able to make up for that, because there will always be situations so rare that they never made it into the data.

pasttense018• 6 years ago

But what really matters is the overall statistics: The self-driving doesn't know how to handle this one in a 10,000 situation. But in the other 9,999 of 10,000 situations the self-driving vehicle is equal or better. Thus the self-driving vehicle averages out to be a lot safer.

wutbrodo2• 6 years ago

This is what makes hype cycles so dangerous. Most people are too simple-minded to understand nuance, uncertainty, or cautious optimism. So you end up with either "this is terrible and worthless and impossible" or "this is incredible and game-changing and imminent". I've seen it happen for everything from SDC to politicians that my social milieu pin their wildest dreams on that end up disappointing them by being decent, competent and non-messianic.

It's probably worth ignoring what most people think about topics like this, except to the extent that their insane thrashing affects the situation (votes, funding, etc).

anewguy9000• 6 years ago

no they didn't lol

unless you were a layman, but then, what are your expectations based on?

cletus• 6 years ago

Alternate headline: Some people overstated how simple this problem was because they didn't know what they were talking about, could promote themselves as experts, could get funding for a self-driving startup or some combination of the above.

I guess that isn't as pithy but it's closer to the truth.

When it became clear that Uber's strategy (under Kalanick anyway) was premised on replacing drivers with AIs before the cash ran out I couldn't see general self-driving vehicles coming within 20 years. I still say that's true. AI assistance? Sure. But there's an uncanny valley there too where the AI will be good enough in most circumstances that drivers lose attention and people will die. You already see this with Tesla autopilot.

Here's a simple counterexample to the idea that self-driving cars are "just around the corner": in NYC, quite a few buildings have doormen. This is great for residents. Part of this is dealing with deliveries and so forth but there's also an issue of general security. People can sneak in (and I'm sure do) but just the fact that a human is there acts a strong (but not complete) deterrent. Just like having a dog is one of the most effective burglary deterrents.

What prevents a lot of bad actions on the roads is actually fear. Fear of what other drivers might do. Fear of road rage by other drivers. That sort of thing. This is just how humans work.

Once a driver knows the car next to them isn't driven by a person it changes their behaviour. They will do things they wouldn't do if it were a human behind the wheel, particularly because they know an AI won't ram them, cut them off, yell at them and whatever. There's no fear there. Even if there's a passenger in the car, it's still (psychologically) different.

How do you program around humans changing their behaviour to take advantage of there being no driver in your car?

chipperyman573• 6 years ago

Counterpoint: personally I don't avoid doing things on the road from fear of the person behind the wheel but for fear that their reaction will cause an accident (swerving, slamming on the brakes, etc). Self driving cars have the same problem and honestly are probably less predictable.

This might be due to the fact that I don't really care much about road rage. At least where I live, the chance of someone getting out of their car and actually escalating it into a situation that matters is basically 0. When I lived in a major city, I felt the same way but for different reasons (most people in big cities Don't Have Time For That). But I can't imagine it's an isolated opinion.

bdamm7• 6 years ago

I had that opinion too, until someone decided to get out of their car and escalate a situation that really shouldn't have been such a big deal. But they were macho in a car oozing with hormones so yeah, it happened. Even so, I'm personally more motivated by financial damage, the headache of dealing with insurance, and the consequences of being found to have broken the law, than I am by the threat of road rage.

bsder2• 6 years ago

How do you program around humans changing their behaviour to take advantage of there being no driver in your car?

The AI car will have sufficient cameras to record your actions and report them directly to your insurance company if not the police.

This will be MORE effective at deterring stupid driving, not less.

eckza• 6 years ago

That sounds... Orwellian.

WalterBright• 6 years ago

That is an interesting point. I can see that human drivers will be much more willing to drive aggressively (i.e. play chicken) if they know the other driver(s) are AI and will take evasive action.

For example, I avoid doing things on the road that an inattentive other driver might be slow to react to and hit me, such as suddenly cutting across 3 lanes of traffic.

wyxuan• 6 years ago

A little bit like how some people aggressively brake in front of Tesla drivers

jeremysalwen• 6 years ago

So you're saying we need to program our driverless cars to occasionally go on road-rage rampages if they're cut off too many times?

alexpetralia• 6 years ago

No, that is reductio ad absurdum. He is simply saying that things which are too predictable are easily gamed, and this changes others' behavior in a game theoretical sense. Irrationality can have utility: see the "Ultimatum game."

salawat2• 6 years ago

Nah. You need to have an ALRU and vehicle to vehicle comm net whereby all the self-driving AI's distribute amongst themselves information on jerk behavior so the entire macropopulation of AI cars can wolfpack these drivers.

Now please laugh, put down the phone and don't even think about implementing this. It's a terrible idea.

mullingitover2• 6 years ago

Once a driver knows the car next to them isn't driven by a person it changes their behaviour.

If the car next to me isn't driven by a person, but is bristling with high resolution cameras that will immediately upload footage of my face to the authorities if I collide with it, I would indeed change my behavior.

falcolas• 6 years ago

Being excessively cautious makes you an unpredictable driver as well. Unpredictability is the antithesis of safe driving.

vorpalhex• 6 years ago

I can see you've never tried to actually get charges brought against someone using dashcam footage...

AtlasBarfed• 6 years ago

As usual the article conflates the (very difficult problem) of self-driving everywhere with various other sub-problems.

The self-driving problem on highways is much more solvable, and can further be improved by convergent infrastructure evolution (forewarning of traffic jams from central control, embedded rfid to aid pathfinding in bad weather, weather reports, wildlife nets/barriers, highly standardized signage, internetworked cars). It would solve and serve a number of functions that are extremely valuable in efficient logistics, safety automation of boring long term driving, and better utilization of infrastructure (automated overnight driving when the interstates are much less used).

I don't really care if my trip to Taco bell is automated. Sure it would be nice, but that is so insanely difficult compared to automating a 400 mile trip on an interstate.

When they can full automate a 400-800 mile interstate trip so that I can sleep in my car while it drives overnight, it will massively disrupt the airline industry in so many positive ways.

But the article is from a sensationalist news source, so what can one expect.

ghaff13• 6 years ago

I suspect that part of it is that a lot of people in the tech and tech journalism space live in and around cities. Many of them are mostly focused on being driven around and not owning a car.

Long highway drives may be something they don't do that often. So the ability to own a car that can automate that subset of their driving probably doesn't strike them as all that interesting.

Personally, as I've said elsewhere, this seems like a much more tractable problem and gives me a lot of the total benefit of self-driving.

simion314• 6 years ago

When they can full automate a 400-800 mile interstate trip so that I can sleep in my car while it drives overnight, it will massively disrupt the airline industry in so many positive ways.

This could be implemented today with train tracks and automated trains.

Someone will comment that there is not enough profit but are the roads making profit?

If you have the money you can travel A class, much more comfortable then sleeping in a car or an airplane.

I am not against cars so don't reply with some examples of why you need a cat, I understand that... my point is that for long distances transport where you could sleep,rest or do something else trains could be done today where AI driven cars would take at least 20 years.

perl4ever5• 6 years ago

"I don't really care if my trip to Taco bell is automated."

Well, it would probably be a significant benefit to someone who is blind to be able to commute to work by themselves.

pasttense018• 6 years ago

Basically the vast majority of people take very few 400-800 mile interstate trips so that doesn't matter [conversely the trucking industry does so long distance trucking would be a strong market]

As you said the Taco Bell trip doesn't matter.

The one area where it does matter is for the longer distance commuter--the unpleasant experience of spending an hour every morning and every night in stop and go traffic.

technofiend2• 6 years ago

Alternate headline: Some people overstated how simple this problem was because they didn't know what they were talking about, could promote themselves as experts, could get funding for a self-driving startup or some combination of the above.

I jokingly call it management myopia: anything I don't understand can't be that hard.

jacquesm6• 6 years ago

anything I don't understand can't be that hard.

Brilliant line, that probably goes a long way towards explaining why software in general simply sucks from a users perspective.

SilasX3• 6 years ago

When it became clear that Uber's strategy (under Kalanick anyway) was premised on replacing drivers with AIs before the cash ran out I couldn't see general self-driving vehicles coming within 20 years. I still say that's true.

Even so, this never made sense as a business strategy, unless Uber somehow has a comparative advantage in achieving self-driving cars and then can get a long monopoly on them.

If they don't -- if others have SDCs around the same time, then sure, their costs go way down, but they can also charge a lot less, because the competition has the same "advantage"!

It's like:

"Hey, man, do you really think your grocery store can make those superhuge profits when you sell at the wholesale price?"

'Oh, no, I have a plan. You know those barcodes they're putting on products now? I'm going to invent a way to integrate that with the checkouts, and boom, I don't have to pay for people to put price tags on. Labor costs go way down!'

Same problem: "No, you're not. Someone better at that will, and they'll sell the tech to all grocery stores."

There was never a reason to believe Kalanick's operation as it stood in 2010/11 had that comparative advantage, so that strategy never made sense. And today, not surprisingly, Uber is one of the worst at SDCs, and is correspondingly unlikely to have that advantage.

perl4ever5• 6 years ago

I think you mean competitive advantage, not comparative.

I got a D in my only economics class, but I think that I've seen comparative advantage used to mean that "X is better at doing Y than doing Z", whereas competitive advantage is "X is better at doing Y than Z is at doing Y".

If you're not using it as jargon, then the fact that it is jargon might be confusing.

lacker• 6 years ago

it became clear that Uber's strategy (under Kalanick anyway) was premised on replacing drivers with AIs before the cash ran out

This is kind of nitpicking, but this has always been a backup plan at best for Uber. Look at their expenditures and it is clear that their strategy has never been "all in" on self-driving.

Their main plan was and basically still is a combination of driving competitors out of the market, increasing their efficiency, raising prices, and lowering payments to drivers, before the money runs out. If self-driving cars happen, it will be a tremendous boon to Uber's bottom line, but it is not a necessity.

mLuby• 6 years ago

driving competitors out of the market,… raising prices, and lowering payments to drivers

So, textbook monopoly behavior.

beamatronic• 6 years ago

Those at the top, early, get rich either way.

hindsightbias• 6 years ago

Troll driving. Imagine testing/probing AI cars with the intent of getting them into an accident with you so you can sue for uber-millions and AI hysteria social media points.

schrodinger2• 6 years ago

Interestingly, that might actually lead to self driving cars being better in the end.

Johnny5555• 6 years ago

Self driving cars tend to have enough cameras that it might be hard to do that without it being caught on camera.

danieltillett• 6 years ago

I always thought Uber’s strategy was to get large enough for the VC and PE guys to be able to offload the whole pile of steaming garbage onto the index funds.

JKCalhoun• 6 years ago

Or simply: Self-driving cars turned out to be just as hard as we thought they would be

gvhst• 6 years ago

Aggressive driving will be captured and recorded by multiple cameras (potentially by multiple vehicles), making most court cases open-and-shut. I think that serves to put the fear in people.

rmtech4• 6 years ago

couldn't see general self-driving vehicles coming within 20 years. I still say that's true.

20 years is a long time, arguably you're already wrong because self-driving vehicles already exist. But Self-driving with the ability to replace Uber drivers will be here by 2030 at the latest.

thebruce87m2• 6 years ago

If you’re talking about level 5 then I disagree with this. None of the technologies I see in use today can work in the same weather conditions that humans currently drive in. Things like snow, fog, and even heavy rain.

If you’re talking anything below level 5 then all we will be doing is making current human drivers worse. Some of the deaths so far have been people who knew the limitations of the technology and still got complacent. If it requires you to have constant supervision then people simply won’t do that and will die.

aphextron• 6 years ago

It amazes me how Tesla can continue to just outright lie in their sales material about this. Right now you can purchase a brand new Model S with a "full self driving" package, even though no such thing exists, nor is there any timeline at all to when it will (if ever). Autopilot as it stands is nothing more than an advanced driver assist system with blindspot monitoring and automatic lane changing. Not even close to a level 3 system, let alone full autonomous.

pkaye• 6 years ago

Didn't Elon Musk said by 2020 there would be robo-taxis?

ctdonath2• 6 years ago

"Lie"? Rather a harsh phrasing.

The only thing unleashing it is the decision to turn it on, which is largely up to regulators.

rantwasp• 6 years ago

sef driving cars... by 2020... on Mars

psychometry• 6 years ago

Any software dev worth their salt knew this was going to be a 90-90 problem.

mark-r4• 6 years ago

Even in a software dev environment where you'd think this would be common sense, I get a lot of over optimism.

AnimalMuppet• 6 years ago

Oh, far worse. 90% of the effort will get you 90% of the way there. Another 90% of the effort might get you another 9% of the way there. But that last 1% of "getting there" is going to take many more 90%s of effort.

noonespecial• 6 years ago

It still feels kind of like we're in the "Apple Newton" phase of self driving with the iphone in the indeterminate future.

One day we'll look back and say "aww, they tried so hard with the limited tech they had and got so close but what they needed to make it good just didn't exist yet".

NohatCoder2• 6 years ago

I think car companies try to rely way too much on machine learning. You get some promising results fast, but it is all inside a black box, both verifying the correctness and changing what is wrong are almost impossible jobs.

Maybe machine learning can be used to tell the difference between a dog and a plastic bag, but you'll need some hard code to describe how to react to either.

ghaff13• 6 years ago

ut you'll need some hard code to describe how to react to either

My understanding is that's largely how it's done. The ML part is mostly about recognizing objects. But the car doesn't "learn" how to drive. It's told how to drive depending on what's happening in its field of view.

Which is why there's probably misperceptions about the importance of miles on the road. It uncovers un-programmed situations but it's not like the car runs over someone and reinforcement learning leads to it not doing that next time.

NohatCoder2• 6 years ago

There are some levels in between, like feeding an ML algorithm recordings of human drivers and expecting it to learn how to drive from watching those.

SemiTom• 6 years ago

It's one of the biggest engineering problems in the last 10 years due to sensory input issues, dealing with the unexpected behavior of outside influences and you have to have the mechanical and software components that the first two components give the commands to, and can then faithfully execute them https://semiengineering.com/challenges-to-building-autonomou... and https://semiengineering.com/more-data-more-problems-in-autom...

erobbins4• 6 years ago

I've been beating this drum for years (but I'm a nobody, so it's like pissing in the wind):

1. Anything less than 100% FULL automation is MORE dangerous than manual driving, because the "driver" will almost certainly lack any situational awareness. When the need for manual intervention happen, it will be at the moments where you need maximum awareness and split second reflexes.

2. There are SO many edge cases and never-seen-before situations that happen when driving "at scale" that the automation features will fail unexpectedly and in strange ways.

3. G and Cruise might be exceptions, but most of the companies in this space are cowboys with reckless disregard for public safety and terrible "iterate quickly" coding practices.

4. At some point there will be an accident that kills a photogenic "middle America" person or people and at that point the government will crush this industry with regulation, with the financial backing of automakers, UAW, and other people who benefit from the status quo.

The only way 100% fully self driving cars will ever happen is for the infrastructure itself to be built to accommodate them. Mixing regular cars, parking, trucks, bicycles, scooters, pedestrians, dog walkers, hoverboards, etc all together on the same roads ensures that the problem is unsolvable.

ghaff13• 6 years ago

I have some confidence we'll get there on limited access highways because there are a lot fewer of the things that you describe there. That doesn't really help the people who want to be driven around everywhere. But it would actually be a really nice feature for the majority of people who own and drive cars.

mumblemumble• 6 years ago

4. At some point there will be an accident that kills a photogenic "middle America" person or people

I'm ready to guess what it looks like: Car on autopilot going through a residential neighborhood, playground ball bounces out into the street from between two parked cars, car does not brake in anticipation of a 6-year-old that is not currently visible.

pasttense018• 6 years ago

It's too expensive to built the infrastructure to accommodate them--current infrastructure spending isn't even enough to eliminate potholes and other bad pavement conditions.

glenra2• 6 years ago

Anything less than 100% FULL automation is MORE dangerous than manual driving, because the "driver" will almost certainly lack any situational awareness.

With enough reliable bandwidth a decent workaround is to have the system fall back to human control by someone other than the local driver. I'm imagining a Car Traffic Control Center where your onboard robot driver sees a situation it doesn't understand and throws control to a remote driver wearing a VR rig with your car's video feeds as input. The remote human driver assesses the situation, steers you carefully past the weird obstacle/issue then returns control to your robot.

A system where robots drive automatically, say, 95% of the time while human remote drivers handle special cases 5% of the time still seems like a big improvement over the status quo - there's a market for that.

deepspace• 6 years ago

throws control to a remote driver

Throwing a remote driver into a dangerous situation with no context sounds like a terrible solution to me. See also: Cpt Dubois from AF447. Doing that deliberately and repeatedly just multiplies the chances of catastrophic error.

And the VR driver job would be so stressful that there may not be many takers. Who would take responsibility if they made a bad call and caused a crash?

woodandsteel• 6 years ago

Anything less than 100% FULL automation is MORE dangerous than manual driving, because the "driver" will almost certainly lack any situational awareness.

Something I worry about is that if SD became normal, then people would never get the experience of thousands of hours of driving a car in countless situations that is needed to develop good judgement, much less quick reflexes. And so when a rare situation arises when they need to take over, they won't be able to do it well.

erobbins4• 6 years ago

We've already seen this in aviation actually. AF447 is a good example of flying a plane into the ground because of reliance on automation and lack of experience hand flying.

There's a really great YouTube video called "Children of the magenta" that's part of a lecture the chief training pilot for AA gave about 20 years ago or so as part of their continuing education. He goes over incidents and situations and the essential conclusion is that pilots are getting too used to turning dials and flipping switches when in many situations they need to just take control and fly the plane.

testcase_delta• 6 years ago

To your first point, I'm not convinced that's true. People augment their lives in lots of ways that don't seem to reduce safety. A few examples off the top of my head: simple "dumb" cruise control hasn't lead to more accidents. Parachutes have auto-deploy features if the cord isn't pulled by a certain height. Scuba divers use dive computers that basically eliminate the need to learn dive tables (and beep at you when you're doing something dumb). Apparently passenger jets are highly automated (I'm out of my depth on that one). These are all on the spectrum towards automation and have only been helpful. Do you think the problem occurs as you approach 100%? Like an uncanny valley in the 99 to 99.99% range?

K0SM0S• 6 years ago

Not OP but to your last question I think there's proven danger in the "too familiar, too easy" zone, i.e. most car accidents for instance tend to happen in places you know pretty well — hence why you may get surprised when things happen out of the ordinary.

Whether it's an illusion of safety, a letdown of attention, the general idea is that humans should never trust that things will go well when there's real probability that they don't. I think it's not in the amount of automation, as you explained well, but rather in focusing users on the critical parts that they should watch out for — and there clearly automating helps us remove the unimportant from the equation, and also make us more responsive, more accurate for the important parts. But it's a lot of great UX, and that's one field where e.g. the military is usually great but commercial companies are abysmal if they can get away with it (read: sell enough to justify not spending a dime on more quality). That's worrying when security is involved, but it hasn't proven a moral or ethical problem for most industries absent of regulation (forced ethics, ha!), so... I think there's valid concern by OP.

As for passenger jets, the Airbus A320 (late 1980s) was the first commercial plane to have a "full" autopilot; all systems were electrical¹ (manoeuvering, thrust control, etc) which allowed the computer to integrate and manage it all. :)

It was tested a number of times by pilots for fun, from taking off to landing entirely on autopilot — ofc they're standing right there ready to take over if anything goes wrong but I've seen it first hand many times. We're talking commercial flights with passengers, it's 100% safe and actually quite "smooth" because the computer is so accurate.

Honestly, the problem is much, much, much easier for planes: a good GPS and it becomes quite the closed problem, and obviously 100% of autopiloted planes are simultaneously piloted by real humans... ready to take over. Yet a plane could technically land itself just fine if pilots were incapacitated, it really could. I suspect it did more than we know for many reasons. And when flying by instrument (means you see s__t), an autopilot is basically just a computer doing what a human would do slower by reading the same data (and maybe cross-checking with physical/manual instruments, but an autopilot doing the grunt work of stick-holding gives you more time to double, triple-check everything incidentally).

_____

[1]: Note that all systems are also doubled (even tripled) with mechanical (hydraulic etc) failovers, because obviously you can lose electricity in catastrophic situations, hence why it always seemed crazy to me that a planed requires software to fly properly instead of plain old good physics and mechanics).

erobbins4• 6 years ago

The thing with other activities (diving, flying are great examples) is that when a problem occurs, you generally have minutes to analyze what's happening and decide on a solution. If my dive computer goes on the fritz, I can decide to immediately start an ascent, or go off physical dive table, or make an extra safety stop at say 20 feet just to be sure.

When you're going 45 in a curve driving along PCH, and a sudden fog bank obscures your cameras and LIDAR and the computer says "your controls, good luck!" you have maybe 2 seconds to react, if you're lucky. It might be a lot less.

Humans make really dumb decisions sometimes, but we are also outstandingly capable of reacting to novelty.

nomel2• 6 years ago

The only way 100% fully self driving cars will ever happen is for the infrastructure itself to be built to accommodate them.

I think we'll see 100% automation for freeways within a short time. I think the only way we'll see 100% automation for arbitrary point a to point b in 50 years, that a standard human could "safely" do, is if we get flying transport.

erobbins4• 6 years ago

I agree with you. I think freeway travel can be 99% automated in the very near future. I also think that's where the MAJOR wins will come from. Long haul trucking can flow 24/7 at that point, with the drivers keeping their jobs and doing last mile delivery.

Johnny5555• 6 years ago

Airline pilots already face this -- it's hard to stay engaged in flying when the plane can fly itself. By the time something bad happens and the plane gives up and hands control back to the pilot, the pilot lacks the full situational awareness he would have had if he was flying the whole time.

axguscbklp• 6 years ago

Yes, harder than hype and deluded optimism expected. Plenty of people have seen through the hype and delusion for years now. Level 5 requires AGI. AGI is probably nowhere close to being realized - humans may well be centuries away from reaching the necessary technological level. Humanity might never get there at all. In any case, if humanity ever developed AGI, it would revolutionize the world. Self-driving cars would be one of the most boring and mundane applications.

viburnum• 6 years ago

You’re right, but this is a religious conviction for nerds, like their belief in space colonies. They flip out if you don’t play along.

nunez• 6 years ago

I have a feeling that self-driving will be one of those things that everyone will be opted into. Self-driving becomes much easier if everyone is playing by the same ground rules (with either the same set of cars or a common protocol through which cars communicate). Trying to engineer away the entire problem space of driving seems intractable (think: you have to engineer for drunk drivers doing damn well nearly anything on the road)

amanaplanacanal• 6 years ago

Even if everybody in every vehicle (including motorcycles?) is forced to opt in, you still have pedestrians, bicycles, scooters, animals, and who knows what else on the roads. This feels like an impossible problem.

Andrew_nenakhov• 6 years ago

Lived in Russia and have some knowledge about programming. Unless some breakthrough happens in general purpose artificial intelligence research, I do not believe self-driving cars that can drive here are possible at all.

Naac• 6 years ago

What I think is interesting is that none of these articles talk about one specific implication of self-driving cars.

What happens to car-insurance companies?

Seems like with self driving cars the form of car insurance we have now wouldn't be really necessary. I expect car-insurance profits to decline. Isn't there an incentive ( read: lobbying ) for car insurance companies to discourage self-driving cars?

jes5199• 6 years ago

Tesla offers car insurance for Teslas, probably trying to get ahead of this

Marsymars• 6 years ago

In competitive jurisdictions, car insurance isn't a profitable venture per customer - insurance companies make money on the float.

In other jurisdictions, auto insurance is provided by a single public auto insurer - these rarely run a notable profit.

tanilama• 6 years ago

Than expected? I am pretty sure none of the people who actually works in the related industry expect self-driving car to come out anytime soon. The hype is created by media and investors, with intention other than creating a viable product.

Driving is simple but still logical. Though it doesn't seem to involve too abstract of processing like what is required for language, that doesn't mean it is just looking the road ahead and making turns. Say if an accident happens and the road condition is messy now the one-way road is changed to switch to allow once direction then another, how would AI understand this? No it can't.

Current NN based model had huge problem guarantee Robustness, while human can be incredibly resilient against adverisal scenarios, because we are superfasr few shots learners.

WhompingWindows• 6 years ago

It's hard to fully understand the challenges of SDCs, because most of these players are extremely secretive about their approaches. The fact that we rely on disengagement data from California as some proxy, that's just not great data when so many of the players are doing 1000's of miles in other states. It's sad, I wish we knew more about the inner-workings of these systems, and I would love to see them collaborate for the benefit of society.

reggieband2• 6 years ago

I've been watching Lex Fridman's youtube podcast and there is a recent interview with Jim Keller [1]. Keller is a chip designer famous for his involvement in multiple chips at Intel, AMD, Apple and he was co-author of the x86-64 instruction set. He also worked for Tesla.

There is a point in the conversation where Lex and Jim clearly disagree about how "easy" self-driving AI should be. Lex is clearly pessimistic and Jim is clearly optimistic. I have to admit I was more swayed by Lex's points than by Jim's, but it is hard to discount someone so clearly (extraordinarily) expert and working directly in the field.

1. https://www.youtube.com/watch?v=Nb2tebYAaOA

theresistor• 6 years ago

Jim Keller went back to Intel in 2018.

reggieband2• 6 years ago

My mistake - I should have checked his bio rather than assume based on the content of the discussion. I've updated my comment to change his association to Tesla to the past tense, Thank you.

krtong• 6 years ago

If you could create instructions efficient enough for a computer to drive a car, it always seemed like you just created the best instructions for a human to drive a car too, and a human who had similar instructions would always be a better driver of the two. "Pay attention to this, prioritize these things, brake when this occurs, turn when this occurs, safest speed is this given these parameters." and so on. You would think silicon valley would be obsessed with learning how to drive really well given the amount of time so many engineers have spent on automating it. We should have our own F1/WRC team by now sponsored by these companies.

msoad• 6 years ago

The more self driving advances in lab, the more safety features for regular cars are invented which in turn makes it harder for fully self driving systems to computer with human+safety features and justify their value...

frosted-flakes• 6 years ago

That's an excellent point. Why hand over complete control to a computer when we can have both? Each has different strengths, so an almost-SDC with a human at the wheel will surely be better than a true SDC. I think that's a much more likely future.

0x8BADF00D• 6 years ago

There’s a huge bubble in the autonomous vehicle space. I foresee Waymo being able to do it, only because I know some of the crazy smart people that work there. It won’t be until the end of the decade though.

rafaelvasco• 6 years ago

As the article said, we're in that awkward transition period. The problem is that it puts people's lives at risk, even though most deaths could be avoided if the drivers hadn't been so negligent, like watching their smartphones while the car AI operated the car, sleeping etc. It's a turbulent transition, but the tech is here to stay, and it'll not only keep improving, but the advancements in AI due to self driving research will surely benefit several other areas;

ars• 6 years ago

I've said it multiple time on HN, and I'll say it again: Self Driving cars are impossible until we have Artificial general intelligence (and not the limited AI that exists now).

The only exception might be specially instrumented road tracks, with limited access (like train rails).

And yes, I fully expect such a thing on interstates. It will never happen inside cities.

Want to make money? Design a vehicle agnostic system, primarily aimed at long haul trucks. Install it in a ton of cars, but don't switch it on.

Then instrument some highways, and convince governments to only allows these cars on those lanes. By being vendor agnostic, anyone with a car could get this.

It will require deep pockets, and maybe you would need government to mandate this (some kind of open standard).

vmchale• 6 years ago

My impression (not in the field) was that Uber wanted to believe in this out of wishful thinking, and then everyone scrambled because they thought they were going to be left behind.

LoSboccacc2• 6 years ago

harder than you pundit expected. nothing in the news cycles ever warranted such optimism. been a skeptic of current tech long enough. they'll come eventually, but they haven't yet even started to define the boundaries of the domain problem, just piling heuristics and people literally died because of corner cases that weren't covered.

well unless you want to tow the line of "that wasn't true autopilot" to wich I say yeah, that's the point exactly, none of them is doing it.

ecolonsmak• 6 years ago

as long as software is based on clumsy "if then else when for" statements and computation relies on binary switches - self driving cars will remain impossible.

linksnapzz• 6 years ago

Expected by whom?

Where did they find people who thought this was going to be easy, and more importantly why were they given billions to screw around with after grotesquely underestimating the complexity of the task at hand?

jihoon796• 6 years ago

Outside the vacuum of technology (can it be done?), it seems to me that the role that policy plays when it comes to the actual rollout of self-driving cars (should it be done and how?) is vastly underrated. This seems to be the real bottleneck in mass adoption - improvements in technology will have diminishing returns after a certain point.

For example, things like getting companies to agree to a unified standard at a government/industry level & determining frameworks for liability all seem to be as important (and perhaps difficult) as eking out another 0.00001% increase in safety.

betoharres• 6 years ago

just like any other side project: "Oh, that's easy to build" fast foward 10 weeks "Shit, this is harder than I expected"

agumonkey• 6 years ago

It's only the end of a wave. 50s saw some ideas, 80s a few more, this one was a massive push [0], I honestly don't think the road to SDV is long and windy. It's just a natural bubble bursting backpush for now.

[0] I'm not even a fan of the idea anymore btw, just trying to assess the technical hurdles. Sensors have become numerous and cheaper, compute power is immense. Investment won't be higher to make new steps in 20-30 years.

readhn• 6 years ago

i think that the best solution right now is very costly: have a separate lane for "self-driving tech" equipped cars.

having a mix traffic (humans&automated cars) is a recipe for a disaster.

kumarvvr• 6 years ago

And they are just getting started with in Developed countries.

The real test for a self driving car would be on roads in third world countries.

That would be a real test of capability.

Meanwhile, I just wish I could drive my car without moving my legs or hands so often. Why can't somebody make a car such that it can be driven by, say a simple joystick like thingy.

mango7283• 6 years ago

I think if you actually tried to drive with a joystick you will appreciate the appropriateness of the steering wheel as a means to control a car.

catalogia4• 6 years ago

Saab tried joystick control back in the 90s: https://en.wikipedia.org/wiki/Saab_9000#Prometheus_(prototyp...

c3534l• 6 years ago

There are problems where we know how difficult the problem is. These problems, you can just throw money at them to make them work and the skill is in figuring out how to do it cheaply. There are other problems where we don't even know what we're missing. I see people, all the time, try to extrapolate current progress on problems where we don't know what puzzle pieces we're missing. Nobody knew what it was going to take to build self-driving cars, nor do we know what it will take today, but the hype machine that sucks in funding and grants and produces nothing pretended like this was a problem we understood and just had to throw money at it. I've found that asking "do we know what we don't know yet?" has been a surprisingly good way to cut through the bullshit over the years. I'd say "I told you so" if anyone knew who I was or had any reason to listen to me.

puranjay2• 6 years ago

I've become increasingly jaded about the tech hype cycle. We had 3D Printing, VR/AR, crypto, and self driving cars hyped to no extent in 2-4 year cycles. We were told that they were going to change our world in the next few years.

Turns out that the underlying technology was far from maturity in each case and the use cases limited to a handful of enthusiasts.

allhacks• 6 years ago

I worked at a car marketplace company. The number of time people told us back in 2014-2016 that Americans would no longer own cars in “2-3 years” and that consequently our market would vanish over night ... btw this group of people included very respectable folks, many of whom have a lot of fans here on HN :)

wutbrodo2• 6 years ago

That's trivially stupid though; if a perfect SDC came out tomorrow, it would take more than 2-4 years to turn over the vast amount of capital out there, and probably another decade or two to shake people (and thus policy, in a democracy) out of their status quo bias, no matter how terrible the status quo is.

oblib• 6 years ago

some researchers have argued we won’t have widespread self-driving cars until we’ve made major changes to our streets to make it easier to communicate information to those cars.

It feels to me like those working on the concept are expecting that if they keep adding sensors and twiddling with AI they can avoid that.

I get why. It's a vast and expensive undertaking that is out of their control and they want to sell their product asap. But if we started with major city streets and highways it could be a quicker and safer route to get it to market.

Years ago (around the late `90s) I worked on an "Intelligent Traffic Systems" project for the city of Branson, Missouri. The 3M Corporation demonstrated a magnetic tape for street lines and a snow plow truck outfitted with sensors that could detect the lines connected to vibrators on each side of the drivers seat. When the truck got too close to the line the seat would vibrate on the side they were close to. I got to ride in the truck for a demo of the tech and worked well.

We also had street cameras that detected autos and could estimate speed and traffic congestion. These sent video and data to the local 911 center. I created a "traffic congestion map" that ran on a web server using that data that worked pretty much the same as Google Maps that show congestion.

We need "smart streets" to really make this work. Without adding that to mix corporations could be banging their heads against the wall and spending billions of dollars to try and never make the last mile.

joejerryronnie• 6 years ago

Self-driving personal aircraft would probably be easier to implement than self-driving cars.

mdonahoe2• 6 years ago

Definitely. The air is a universal medium... the ground is a mess.

zelly2• 6 years ago

Self-driving stops being a (difficult) AI problem if you force all cars to be self-driving and network with each other. It converts the behavioral/theory-of-mind problem into a distributed computing problem. We could have had self-driving in the 80s with this approach. Imagine where we would be today if we had decades of 24/7 trucks and ubiquitous robotaxis. It would be a completely different world. It's a shame humans are so bad at cooperating. But still, this is not out of the question today. Obviously the only way to pull this off would be with massive government subsidies. It would be worth experimenting in a small country like Luxembourg.

Barrin922• 6 years ago

relying on networked cars is monstrously fragile. One critical error in the system is going to tank everyone, disconnecting from the system is going to drastically reduce the safety of both the individual and the system, as it now has to deal with rogue elements, and the potential for malicious attacks, like terrorism or simply some sort of natural freak event is huge problematic.

I think the direction of the thought is reasonable though, you should just take it a few steps farther. If networked infrastructure is a good idea, then maybe cars are not a good idea. We already have driverless, well defined, organised modes of transportation, they're called trains.

modern subway systems already pretty much drive themselves, they also come with the added bonus of not having everyone carry two tons of steel around.

zelly2• 6 years ago

Those concerns are legitimate. My proposal is not a purely centralized hub-and-spoke network topology where one data center drives a million cars. Yes, you can have the central brain too, but I'm thinking more of a mesh network among nearby cars on the road. The network can be fully connected so that one nefarious car cannot take down the whole mesh. There are a lot of cryptographic safeguards to prevent bad actors too. Each car is able to drive on its own, or pull over, only relying on networking for a stream of sensor output from nearby cars. The upstream servers can go down without doing anything worse than making all its client cars pull over and stop. All the steering is done on the client side.

What if terrorists spoof phantom cars or rewrite maps to send people off cliffs? Assume they stole the master signing keys, have root on the central servers, exploited 0-days on the client car software, etc. One car misbehaves somewhere, triggering sensors of nearby cars which tell every other car in an N kilometer radius to pull over en masse. If anything, it is more robust to hacking/terrorism than the independently-self-driving Tesla or Waymo approach because those do not have the benefit of the herd. One gazelle in a herd who gets tackled can yell out to save the rest of the herd.

I think the direction of the thought is reasonable though, you should just take it a few steps farther. If networked infrastructure is a good idea, then maybe cars are not a good idea. We already have driverless, well defined, organised modes of transportation, they're called trains.

How do I take a train/subway from my apartment to the front door of a McDonald's? People go from building to building. It's not practical to do this without cars or buses outside of maybe 5 cities in the world like HK or NYC.

Also there is an ungodly amount of cars in the world. It's a lot cheaper and efficient to retrofit cars with self-driving modules than to recycle all that metal into trains/subways.

m3kw92• 6 years ago

There are lots of edge cases that human can adapt to easily but would confuse machines. The risk is that when that edge case comes up if the car go full bat sht crazy and kills someone, that will bring down the hammer.

m3kw92• 6 years ago

Just look at parking lots and self summoning. Tesla is still in step 2 of 10

fnord77• 6 years ago

near human levels of AGI will be needed to achieve Level 5 AVs- change my mind.

mdonahoe2• 6 years ago

It depends on your definition of vehicle.

I bet we see a few Level 5 flying cars before we see Level 5 ground vehicles. Roads are more complicated than the airspace.

Flying cars are unlikely to be as mainstream as cars due to energy requirements though.

KKKKkkkk1• 6 years ago

In the debate over self driving, there are two schools of thought. One says that you can't solve it without machine learning because it's impossible to hand-engineer a system to cover all edge cases. The other says that you can't solve it with machine learning because any solution must have zero glitches. Both sides are correct.

_pmf_• 6 years ago

Expect massive push for goal post shifting.

vgaldikas• 6 years ago

One study attempting to estimate the effects of self-driving cars on car use behavior simulated a family having a self-driving car by paying for them to have a chauffeur for a week, and telling them to treat the chauffeur service the way they’d treat having a car that could drive itself.
The result? They went on a lot more car trips.

That's kind of pointless 'study'. Of course I will take a lot more trips for some time after getting chaffeur/self driving car, just for the novelty of it. One family for one week does not really tell anything

fourthark• 6 years ago

Puff piece with critical title.

axilmar• 6 years ago

We could make driving fully automatic by going the train route, i.e. filling roads with tracks and letting the cars move on those tracks only, fully automated.

But that's not really sexy, is it? it just doesn't sell as well as fully automated AI driving does.

sunstone• 6 years ago

Who would have thought that it might be tricky to replicate the capabilities of an organism exquisitely designed by evolution over more than 10 million years to dash through the jungle canopy with break neck speed, split second coordination and decision making?

Self-driving cars turned out to be harder than expected

Related Stories