How Tesla's AIgorithms See the Streets of Paris

Hackers at the 'First International Tesla Hacking Conference in Paris' figured it out. JL

Kabir Chibber reports in Quartz:

A Tesla car uses a series of cameras, ultrasonic sensors, and radars that are interpreted by algorithms inside the onboard computer to present a vision of the the road, one that allows its Autopilot software to drive semi-autonomously for extended periods of time. Tesla even says the hardware needed for full self-driving capability is already in place—it is the software that has to catch up.

A Tesla car uses a series of cameras, ultrasonic sensors, and radars that are interpreted by algorithms inside the onboard computer to present a vision of the the road, one that allows its Autopilot software to drive semi-autonomously for extended periods of time (albeit with a few crashes here and there). Tesla even says the hardware needed for full self-driving capability is already in place—it is the software that has to catch up.

We have no idea how Autopilot actually sees the road—except for when hackers get involved.

Now, two hackers hanging out at the “first international Tesla hacking conference in Paris” have built on previous attempts to let us see the world as a Tesla sees it, posting their explanations and videos on Reddit.  Their stroke of luck seems to be buying an Autopilot 2.5 computer on eBay that happened to have be a fully unlocked developer version of the software. “Now [a] fully unlocked unit changes all of this since it allows any modifications to be performed and so it’s really pure gold from research perspective,” says one of the hackers. Using the internal data that comes out the computers and combining it with some together data visualizations, here is how they think a Tesla sees the streets of Paris…The hackers are quick to state some caveats. “Keep in mind our visualizations are not what Tesla devs see out of their car footage and we do not fully understand all the values either (though we have decent visibility into the system now as you can see),” they said. “Since we don’t know anybody inside Tesla development, we don’t even know what sort of visual output their tools have.”

That said, it is an interesting attempt to piece together how a Tesla (and also other cars using similar hardware) is trying to interpret the millions and millions of variables that are happening as you attempt to drive a car at speed. “It’s too bad Tesla is so secretive about their progress in the area and that we need to resort to these measures to shed at least some light on the progress,” the hackers said on Reddit. “Hopefully this will prompt Tesla to also make some official footage available?”

That is unlikely. But Tesla founder Elon Musk has tweeted that Tesla is about to release Autopilot version 9, its biggest software update in two years, which includes dashcam footage using the car’s Autopilot cameras for the first time.