Tesla releases FSD Beta v12 with AI vehicle controls

https://electrek.co/2024/01/22/tesla-releases-fsd-v12-last-hope-self-driving/

by rntn • 3 years ago

28 40 3 years ago

barbazoo• 3 years ago

They were also encouraged by Musk saying that Tesla would deliver v12 not as “beta” software.
The update is still listed as “beta” in the release notes.

jgalt212• 3 years ago

Musk being Musk.

AlexandrB4• 3 years ago

Kinda hate that "Beta" is now mostly a marketing term for "be nice to us". When you're shipping software with potential safety implications to thousands of vehicles in the wild that doesn't seem like a beta in the classic sense. It's just a release that you don't want to be judged too harshly.

blowsand2• 3 years ago

Musk’s statement was that FSD will exit being a beta program in the v12 major version chain, not that that v12.n is not beta.

rsynnott2• 3 years ago

... So, v1 to v12.x where x is unknown: beta, v12.y where y>x: not beta.

That is a _spectacularly_ weird version scheme, but of course has the benefit that there is no rush; provided they always stick with 12.something, it is always true.

altairprime• 3 years ago

This Electrek headline is quite poor. I think the title should be updated to:

Tesla releases FSD Beta v12 with AI vehicle controls

Which uses the various phrases present in the Electrek article and more concisely gets to the point, without the 'last hope' spin which is tangential and not itself newsworthy.

(I would personally have selected "ML" rather than "AI", but I wanted to try and write a better Electrek headline using their own article's phrases, so AI it is.)

dang• 3 years ago

Ok, let's use that above. Thanks!

rsynnott2• 3 years ago

... Until FSD v13, of course.

ProllyInfamous• 3 years ago

The thirteenth "within 18 months FSD!" announcement yet!

I loved how the AI Lex used to interview Musk (most recently) poked fun at this very topic. They're all three (AI included) master bullshitters.

AlexandrB4• 3 years ago

The biggest difference with previous FSD updates is that the vehicle’s controls would now be handled by neural nets rather than being coded by programmers.

Does anyone have details on what this means? To me this sounds like neural nets are now part of the low level control loops for the throttle/steering as well - but that doesn't seem particularly beneficial. Especially if you're interested in control stability/predictability.

HarHarVeryFunny2• 3 years ago

Yeah - Tesla software has two parts, a vision system and a driving system. The vision system turns the camera image into a description of what is in the image (road, cars, people, etc), and the driving system actually controls the car based on the vision input and route it has been given.

Up until now the driving part had been hand coded in C++, but now it is a neural net that has learnt what would a human do in similar road situation (not sure what they trained it on). Is this an improvement? Who knows - should certainly be different. This doesn't mean the car suddenly got smart and now understands what it is looking at - it just means it's using a different set of rules for when to switch lanes, brake, etc.

cdumler• 3 years ago

I would argue reasonable to believe it will be notably better until it is catastrophically wrong. The issue with neural nets is that they can be really well trained within the bounds of training set; however, when faced with novel, there is no underlying abstraction inference on which to revert. Look instances where autonomous cars are disable or have weird responses when people modify signs[1]. The driving will be better, but the when the information sits squarely outside of the norm (art on the road, abstract images that defy normal depth cues, etc), the car will do very unexpected things.

As humans, we have this issue already. We call them "optical illusions." Optical illusions are really brain neural net failures where there are competing narratives between evaluations of the information provided by our senses we conclude invalid results. We don't have the ability to process absolute light values, so we infer colorspace by relative color within our 3D mental model, which can result in inferring a difference in color where there is none.

1. https://spectrum.ieee.org/slight-street-sign-modifications-c... 2. https://en.wikipedia.org/wiki/Checker_shadow_illusion

andrewmcwatters4• 3 years ago

You’re right. comma.ai is making the same mistake and it shows in their current experimental mode.

I find the argument that you can allow both longitudinal and lateral control to be expressed by a black box difficult to accept. The proof of the driving is in the handling.

There’s a video of an engineer at comma.ai mocking self-driving teams for semantically labeling objects on the road in SAE Level 2 and beyond systems.

Which screams stupid to me. Why would you be proud that your system has no idea what a pedestrian is. Or what a traffic cone or a speed limit sign are? Idiotic.

At least Tesla still semantically labels. But I argue that you can’t get smooth driving out of neural network controls because they inherently have no idea what smooth driving explicitly is. They estimate it.

The best they can do is attempt to mock it, which without programmer-defined rails WILL result in excessively high jerk and acceleration numbers, as proven by openpilot’s implementation.

zaptrem3• 2 years ago

The problem is manual labelling doesn't scale. Switching to an unsupervised approach is what made many huge AI advances over the past couple years possible since we're mostly limited by compute now. For Tesla (who have practically unlimited driving footage) this is particularly advantageous.

hulitu• 2 years ago

Yes it means there is no difference. Neural nets are coded by programmers.

datadrivenangel• 3 years ago

“FSD Beta v12 upgrades the city-streets driving stack to a single end-to-end neural network trained on millions of video clips, replacing over 300k lines of explicit C++ code.”

That sounds terrifying.

quonn4• 3 years ago

No it doesn‘t. What‘s terrifying is the previous non-FSD code that just centers the car in the lane and will hit anything protruding into the lane. There is little reason to believe that the vision will fail worse than before or that hand rolled control code can do a better job based on wrong and unreliable input data from a net.

Additionally this does not preclude a safety check around the outputs, preventing, for example, an extreme acceleration. This would be similar to any checks put on ChatGPT output.

I wonder how they will handle the previous visualization though. Maybe it would make sense to also predict the visualization from the same network based on labeling the data (perhaps using the old methods) but unless that is somehow checked against the control decisions, it will not instill the same confidence or feeling of understanding in the user. But that doesn‘t mean that the driving is worse.

AlexandrB4• 3 years ago

What‘s terrifying is the previous non-FSD code that just centers the car in the lane and will hit anything protruding into the lane.

Are you sure this is what happened before? I've definitely seen Teslas avoiding obstacles in the road in various YouTube videos. Even "classic" (non neural net) robotics navigation systems can avoid static obstacles in their path pretty reliably.

zaptrem3• 2 years ago

Currently it appears the new planner net is not trained directly on video but on the semantic labels from the current gen perception system (e.g., BEV Lanes/Vehicles/VRUs). Accordingly, they still show them on the screen.

bratao• 3 years ago

The Tesla v12 software update is expected to introduce what CEO Elon Musk has been calling “end-to-end neural nets”. The biggest difference with previous FSD updates is that the vehicle’s controls would now be handled by neural nets rather than being coded by programmers

This sounds very scary to me. Neural Networks have great results on many classification tasks, however they "hallucinate" on completely random and/or easy inputs in an unpredictable way. I already delivered hundreds of classifiers to production in my career and I always had to safeguard inputs for cases I knew the answers or it was wrong. Production ML is always hybrid with hand-made rules. I hope this is just a marketing-term and not something forced to please Musk.

quonn4• 3 years ago

Why would this be different with a net that only handles the input? It would be the same. All v12 does is learn control outputs directly from camera data instead of, for example, learning to detect a traffic light and then have some hand rolled logic. If the input is unpredictably new it would have failed in either case.

ijhuygft776• 3 years ago

How can this title be true? Did Musk force them to use these exact words?

haunter• 3 years ago

Clickbait by the author's own admission

Sure, maybe I should have written "my" last hope, but come on.

https://twitter.com/FredericLambert/status/17494587384428711...

HarHarVeryFunny2• 3 years ago

The Tesla v12 software update is expected to introduce what CEO Elon Musk has been calling “end-to-end neural nets”. The biggest difference with previous FSD updates is that the vehicle’s controls would now be handled by neural nets rather than being coded by programmers.

Up until now Tesla's software used a neural net for vision (i.e to turn camera view of road into an internal representation of road, cars, people, traffic signs, etc), but used conventional programming (300K lines of C++!) to actually drive the car - change speed, steer, switch lanes etc to follow the desired route given the cars position on the road/etc provided by the vision input.

This new release is now replacing those 300K lines of C++ with another neural net that has learned when to change lanes, how to steer etc. There is still just a vision system that "sees" the road, and a driving system that steers the car based on this vision input.

Too bad the real world isn't a video game or Tesla simulator world ...

Maro• 3 years ago

Given that this is an entirely new FSD engine, Elon should reset the counter for "X million miles driven with only Y accidents".

andrewmcwatters4• 3 years ago

I think the biggest lie about the Tesla sensor suite as it exists today is that it’s suitable for Tesla Vision and that it can satisfy Musk’s fair concept of “if humans can drive with only vision, a computer should be able to, too.”

As far as I understand, those cameras are roughly equivalent to functionally 1080p devices which can’t even coherently recreate license plate characters a few meters away from your vehicle.

Kirby642• 3 years ago

The current resolution of the cameras on the HW3 systems is 1.2 megapixels. HW4 is 5 megapixels. So, less than 1080p per camera for most of the Tesla cars is HW3. 1280x960, I believe.

Not sure why reading license plates is a meaningful indicator of capability though. Most people can't read license plates clearly from a distance that doesn't impede driving, and there's many lighting-relating issues that make reading license plates difficult for cameras.

andrewmcwatters4• 3 years ago

At less than 1080p, you can’t make out digits one car length in front of you.

How are you going to read the speed limit?

bigtex• 2 years ago

Has anyone else been able to download the Tesla Network app from the Apple App Store because I can’t find it.

hacoo• 2 years ago

Some background here: in the autonomous vehicles world, it's common to split things up into 'perception' (detecting things around the car and creating a world model), 'planning' (deciding what to do), and 'controls' (giving gas, brake, steering etc so the car smoothly follows the plan). Saying telsa is using AI 'controls' here is probably a misnomer, Musk's statement indicates they are moving the entire planning stack to a NN.

'End to end' is also a bit misleading, as this usually indicates that perception and planning are fused into one network. They aren't doing this (which is probably a good idea), it's only 'end to end' from the planning and possibly controls side.

Tesla releases FSD Beta v12 with AI vehicle controls

Related Stories