Leaked Data Suggest Uber Driverless Program Far Behind Waymo

Performance data perceptions...JL

Timothy Lee reports in ars technica:

Prior to last Sunday's fatal crash in Tempe, Arizona, Uber's self-driving cars in Arizona were "struggling" to go 13 miles between interventions by a safety driver—known as a disengagement. Waymo's self-driving cars in California traveled 5,600 miles between incidents. Cruise, GM's self-driving car subsidiary, had a safety-related disengagement once every 1,250 miles in the state. We don't know either company's statistics in Arizona because Arizona law doesn't require them to be disclosed.
Insiders have long viewed Uber as a laggard in the driverless car race, but internal documents obtained by The New York Times suggest that the company's self-driving car program may be even further behind its rivals than had been publicly known.
The key statistic: prior to last Sunday's fatal crash in Tempe, Arizona, Uber's self-driving cars in Arizona were "struggling" to go 13 miles between interventions by a safety driver—known as a disengagement.
The Times points out that, in 2017, Waymo's self-driving cars in California traveled 5,600 miles between incidents in which a driver had to take over for safety reasons. Cruise, GM's self-driving car subsidiary, had a safety-related disengagement once every 1,250 miles in the state. We don't know either company's statistics in Arizona because Arizona law doesn't require them to be disclosed.The Times presents the Uber and Waymo paragraphs back to back, suggesting they're directly comparable. But it's not clear if they are. The Waymo and Cruise figures are for safety-related disengagements—situations when the driver has to take over to prevent an accident. The figures don't include situations when the vehicle gets stumped by a tricky situation like a construction site and needs the safety driver to take over even though there's no immediate danger of a crash.
It's not clear from the Times report whether that 13-miles-per-disengagement figure is for safety-related disengagements—which would be comparable to the California numbers—or for all disengagements—which wouldn't be.
Moreover, as an Uber spokesman pointed out to the Times, the disengagement rate depends on many factors, including the type of roads the car is being tested on, the kinds of tests being performed, and how a car's software is configured. However, Uber is testing its cars in the Phoenix metro area—a region whose wide suburban streets are generally considered among the easiest in the country to navigate. In contrast, Cruise CEO Kyle Vogt has boasted about testing cars on the crowded and chaotic streets of urban San Francisco.
If it is an apples-to-apples comparison, then Uber would have a lot of ground to make up. In 2016, Waymo's (then Google's) cars in California went more than 5,000 miles between disengagements. In 2015, the figure was 1,250 miles per disengagement. So that would mean Uber's cars need human help 100 times as often as Waymo's cars did in 2015.
Even if the the statistics aren't directly comparable, the figure isn't a great sign for Uber. As Ars reader SymmetricChaos points out, "Waymo now has some cars without safety drivers. If Waymo cars were shutting down in the middle of the street with any regularity, people would have noticed."
Prior to Sunday's crash, Uber apparently believed it was close to being ready for a commercial launch. The Times reports that "Uber was planning to seek regulatory approval by December to start a self-driving car service in Arizona." Uber also placed a massive order last November for 24,000 Volvo vehicles for its driverless car program. Deliveries are scheduled to begin in 2019 and continue into 2021.
The Times reports that Uber CEO Dara Khosrowshahi considered shutting down the driverless car program shortly after he took the reins last August. However, he "became convinced that it was important to Uber's long-term prospects."
Correction: In the original version of this story, I assumed that the California numbers were comparable to the Uber figure in Arizona. Reader tie convinced me that might be wrong, so I've re-written the story to reflect that uncertainty. I'll update the story again if I get a clear answer on whether the numbers are comparable. I apologize to readers for jumping to conclusions in the original version.