The Main Item
It’s no secret that we at Newcomer are hoping for self-driving cars to succeed. The technology could save thousands of lives, as Eric pointed out, and excitement around AI has brought fresh enthusiasm to a sector that failed to live up to earlier hype.
In fact, amid all the hoopla around generative AI, autonomous vehicles have gotten much closer to reality recently, as years of work developing and integrating an array of technologies bears fruit. Google’s Waymo is running a full-fledged self-driving taxi service in San Francisco as of late June, and investment in autonomous vehicle ventures rebounded strongly in Q2, according to Crunchbase. Cruise, GM’s self-driving affiliate, is back on the road after an ugly accident.
But the biggest question facing the industry comes from Elon Musk. He insists that only computer vision—lots of cameras and AI software—is needed for cars to drive themselves, even as the rest of the industry deploys radar and another range-finding technology called lidar. Musk contends that any self-driving car that relies on lidar is “doomed,” as he put it at a Tesla Investor Day in 2021, and he’s called it “unnecessary” as far back as 2015.
Instead, he’s promoted Tesla’s camera-based driver-assist system as “Full Self-Driving,” and contends it’s improving every day with machine learning. If he’s right, he’ll have a far cheaper solution, and be poised to dominate the industry.
A Wall Street Journal investigation published this week, though, presents a lot of evidence that he is in fact wrong, and dangerously so.
This long-running industry debate may now be reaching its moment of truth.
The Journal got a hold of hundreds of Tesla accident reports and videos from the car cameras—information that is sealed by federal regulators but sometimes available via local authorities. It’s not a pretty picture, with numerous accidents, some fatal, that appear to result from the computer vision simply not seeing things.
A former Tesla employee says the cameras on the cars are not calibrated well enough to function optimally and properly train the AI. The latest investigation builds on years of the Journal’s reporting on Tesla Autopilot’s problems.
Musk’s strategy is logical, if cold: persuade people to buy, and use, a feature literally called Full Self-Driving, enabling it to collect the video data needed to train the software while waving off responsibility for safety by pointing out that it warns people to be ready to take over.
It’s remarkable that federal regulators have allowed this to go on, and they could decide they’ve had enough at any time.
Ethics aside, it’s also not clear that machine learning as currently conceived is the right solution to the self-driving challenge.
Neural networks produce results that are ultimately probabilities. Based on data from the past, is that apparition in the road ahead most likely a construction crew, or an artifact of some sort? Lidar, a cousin of radar where a sensor uses pulsed laser beams to determine where objects are in space, provides a very different sort of data: there is definitely an object in the road ahead.
Robotaxi companies like Waymo, Cruise, and Amazon-backed Zoox, trucking companies like Waabi and Aurora, and autonomous delivery startups like Nuro all use a combination of lidar and radar sensors to map out their surroundings, as well as some computer vision.
“We ought to have radar and lidar in there because they do things that human eyes can’t do,” said Jon McNeill, the vice chairman of Cruise’s board and former Tesla president of global sales and marketing. “If we want acceptance of autonomous driving, it needs to be as safe as we can reasonably make it. Penny pinching doesn’t make sense in this context.”
“Lidar and computer vision is a trade off,” notes Coelius Capital’s Zach Coelius, who owns a stake in Cruise. “At the moment lidar is clearly better, but more expensive and it would be difficult or impossible to roll it out to all the Tesla cars.”
Tesla says it will soon introduce its robotaxi, but it’s difficult to see how a commercial service could be offered any time in the foreseeable future, given the current state of Tesla’s technology. There’s also the fact that it hasn’t even begun the kind of careful testing on public roads that Waymo undertook for years.
That should give competitors a pretty good window. And even with both tech giants and auto giants well down the road on autonomous vehicles, there’s still a lot of opportunity.
Waabi, a self-driving trucking startup, trains its autonomous system on a large database of simulated scenarios, said investor Kanu Gulati, so that its trucks will be prepared for possible edge cases before they ever embark on the road. “Tesla’s approach in contrast is to collect real world data, but many scenarios happen rarely, but not never, and thus their safety case is more expensive and arguably weaker,” she said.
Waabi struck a deal with Uber last year to pair its autonomous technology with Uber Freight delivery. The partnership seems to be paying off: the team raised $200 million in Series B funding this June from existing investors Uber and Khosla Ventures. NVIDIA jumped into the round along with carmakers Volvo and Porsche.
And last week, autonomous delivery startup Nuro was given the greenlight to test its latest vehicles in four California cities, a win after a year of layoffs and restructuring.
American self-driving startups are also feeling the pressure from China. Internet giant Baidu has launched over 500 robotaxis in the city of Wuhan, with plans to rapidly expand. Pony.ai, which began in Silicon Valley but is now headquartered in China, recently received approval to begin operating self-driving freight trucks between Guangzhou and Beijing, the first inter-city permit of its kind in the country.
And investors are also betting on alternative forms of autonomous vehicles, such as ones that run on more fixed routes. Khosla Ventures invested in Glydways, a startup that’s developing autonomous vehicles that operate along protected routes like bike lanes. “This setup makes it much safer by cutting out unpredictable elements like intersections and pedestrian traffic,” Glydways CEO Gokul Hemmady told me.
The fresh excitement about autonomous vehicles again is evident in the funding numbers. VC investment in passenger car and trucking startups jumped to $2.7 billion in Q2, according to Crunchbase. That’s still well below the heyday of 2021, which saw over $12.7 billion in VC dollars invested in the sector, but it’s above the pace of 2022 and 2023, which stood at $5.9 billion and $5.7 billion, respectively.
The road to real, widespread self-driving is a long one yet, and still involves plenty of risky bets. But the promise of big payoffs, and new ways to navigate the world, beckon more than ever.