Tesla’s FSD continues to expand and learn, and V13 won’t be any slouch on this. V12 implemented end-to-end AI, and V13 brings a host of new features to help it reach feature completeness.
Today, FSD relies almost entirely upon visual data acquired from the vehicle’s cameras. Of course, it does pull information from other sensors, but the primary input is vision. While Tesla previously used radar in its vehicles and still ships the Models S and X with deactivated HD Radars, it relies on vision to guide its decision-making.
But that’s all changing—a revised version of V13 will bring audio-based decision-making to FSD for the first time. FSD is famously designed to work like a human driver—it relies on vision—but now it’ll also begin relying upon audio—both as an input… and an output.
We touched upon these items in our article on FSD V13, but it's time to really dig into them.
Listening for Emergency Vehicles
FSD will soon be able to detect emergency vehicles by analyzing the sounds it hears. Interestingly, this will be done through the internal microphone—the same one used for voice commands. That’s because sirens are loud—loud enough for humans (and microphones) to hear them inside a moving car.
This will enable FSD to identify the distinct sounds of an approaching siren - helping to ensure that FSD detects emergency vehicles earlier and takes the correct maneuvers to move out of the way and safely pull over.
In addition, by analyzing the actual sound of the incoming siren, FSD should also be able to make a reasonable determination about whether the siren is approaching or just echoing off the city streets using the Doppler effect and some fairly simple math.
With the release of FSD V13.2 to early access testers and now FSD 12.5.6.4, Tesla has added a new item to the release notes that lets drivers opt-in to sharing audio data. For users who opt-in to sharing this data, Tesla will now receive 10-second audio clips in certain situations. Tesla will listen for certain sounds and then send this data back to Tesla for further analysis. This will help them improve certain sound detections.
In the release notes, Tesla specifically mentions detecting emergency vehicles by sound, but it seems that it will also be used for other things, such as listening for other vehicles honking or potentially someone yelling at the vehicle. These additional capabilities will help FSD navigate a world made for humans, which couldn’t be done with vision alone.
FSD Will Honk
So we’ve covered inputs… what about outputs? Ashok Elluswamy, Tesla’s VP of AI, mentioned that FSD will gain the ability to honk. That means FSD will be able to provide an audio cue to other vehicles - just like a real, human driver would. Whether that’s someone cutting FSD off, or someone dozing off at a traffic light, FSD gaining the ability to honk will be extremely valuable - it’s the first ability FSD will have to communicate with the outside world and with other drivers.
Humans have developed different types of honks, such as short, friendly taps of the horn or louder, longer horn presses for emergency situations. It’ll be interesting to see if Tesla also implements different types of honks as well.
This is one of the key steps to humanizing FSD - one of the final puzzle pieces. This is expected to be the final set of inputs necessary for FSD to be able to drive like a human, and it’s exciting to see Tesla get so close with just vision.
FSD will soon be able to see, hear, and honk. Let’s just hope Tesla’s initial implementation for honking is better than Waymo’s (video below of Waymo vehicles honking repeatedly at 4 am).
Imagine being woken up at 4 a.m. by cars honking at each other. That's what some San Francisco residents have been dealing with for weeks, as the Waymos can be heard in this video honking and blinking headlights in a parking lot outside of their condo. https://t.co/2cVmDfUk4i pic.twitter.com/pkxNTT5vXd
— ABC7 News (@abc7newsbayarea) August 13, 2024
