Auto Companies Want To Use AI To Monitor Drivers' Emotions Behind the Wheel

Safety is a by-product at best. The real target is harvesting data for resale, as usual. JL

Todd Feathers reports in Motherboard:

Automakers and advertisers have come to realize that as cars become more autonomous and embedded with screens, "passengers are a captive audience." (They) plan to deepen data mining with in-cabin cameras linked to emotion-detecting AI—algorithms that monitor changes in facial expression in order to determine a person’s emotional state at any given time. Automakers market the systems as safety features. The systems also hold potential for harvesting the behavioral data that Google, Facebook, and other surveillance capitalists have exploited to target ads and influence purchasing habits.

On February 18, executives from the technology firm Cerence rang the Nasdaq opening bell, then stepped into a meeting with their investors and representatives from some of the largest automakers in the world. Over several hours, they pitched a strategy designed to help double the company’s revenue in five years: Record every movement, every glance, every smile, frown, and wrinkled brow of drivers across the world—then sell the resulting data for a profit.

Over the last century, the car has been enshrined as a quintessential piece of Americana—a symbol of freedom and self-expression. It is a space where people share their first kisses, cry after work, and soothe restless babies to sleep, comforted by a sense of autonomy and control. But as the Cerence presentation laid bare, the car’s second century will be very different.

The Burlington, Mass. company is far from a household name, but its technology—including microphones, virtual assistants, and gaze-monitoring cameras—is already installed in more than 325 million vehicles, from which it uploads more than 100 million data transactions to the cloud every month, according to its investor documents. Very soon, Cerence announced, it plans to deepen that data mining operation with in-cabin cameras linked to emotion-detecting AI—algorithms that monitor minute changes in facial expression in order to determine a person’s emotional state at any given time. 

“This is data that I think is the untapped potential that Cerence has for the future,” Prateek Kathpal, the company’s chief technology officer, told investors. “What we’re looking at is sharing this data back with the [automakers] and helping them monetize it.”

Over the next two years, companies like Cerence, Affectiva, Xperi, and Eyeris plan to roll out emotion- and object-detecting systems for cars in partnership with many of the world’s largest automakers, according to company documents and interviews with executives. Their plans are bolstered by a European Union law mandating that all new cars be equipped with at least rudimentary driver-monitoring by mid-2022, and a similar bill recently introduced in the U.S. Senate.

To the public and to legislators, automakers market the systems as safety features. If a car can detect that a driver is angry or looking at their phone immediately before a crash, these companies reason, the onboard AI may be able to offer a warning the next time it senses similar behavior. Or, if it can determine how a child is positioned in the back seat, the car might deploy airbags more effectively in the event of a collision.

But safety is only one attraction of in-cabin monitoring. The systems also hold huge potential for harvesting the kind of behavioral data that Google, Facebook, and other surveillance capitalists have exploited to target ads and influence purchasing habits.

Automakers and advertisers have come to a “vast realization” that as cars become more autonomous and embedded with screens, “many passengers in your vehicle are kind of a captive audience in an entertainment context,” Gabi Zijderveld, Affectiva’s chief marketing officer, told Motherboard.

Affectiva spun off from the MIT Media Lab in 2009 and has been a pioneer in the often-controversial field of emotion-detecting AI. The company has its roots in marketing and consumer analytics and has been tapped by some of the country’s largest brands to measure focus group reactions to advertisements and entertainment. But over the last four years, it has devoted a significant amount of its attention to developing in-cabin monitoring and has worked with automakers including Kia, BMW, and Porsche. It has also pitched its technology to rideshare companies, suggesting in one product brochure that riders might be willing to be recorded and have their emotions analyzed in vehicles in exchange for free or discounted trips. “That data is tremendously valuable, from a monetization perspective, to the advertisers,” Zijderveld said.

Competitors Xperi and Eyeris are also looking at ways to capitalize, or help automakers capitalize, on the data their products gather.

Jeff Jury, the general manager of Xperi’s automotive group, told Motherboard that the company’s in-cabin monitoring system is a safety feature first, but added that Xperi is exploring ways to combine the system with the sophisticated entertainment recommendation engine it recently acquired through a merger with TiVo.

Eyeris CEO Modar Alaoui likewise told Motherboard that while his company’s technology is primarily designed to improve safety, “we do foresee at some point that [automakers] will try to leverage the data for several use cases, whether it be for advertising or [determining] insurance” premiums.

Inform and consent

Cerence, Affectiva, Xperi, and Eyeris officials all told Motherboard that their companies simply create products with many possible functions. It’s up to the car manufacturers, they said, to decide how the systems will be used, what data will be collected, how that data will be processed internally, to whom it can be sold, and whether to share any of that information with the customer.

With the exception of Volvo, all the automakers contacted for this story—including GM, Toyota, Honda, Volkswagen, and Kia—either did not respond to requests for comment or declined to answer questions about how they will inform drivers about the data collected from their vehicles. Volvo does not currently have plans to integrate systems that monitor passengers or use emotion recognition, a company official told Motherboard.

ADVERTISEMENT

European Union regulators have prepared for the likelihood that automakers will want to use in-cabin monitoring systems as data vacuums. 

In January, the European Data Protection Board issued guidelines governing the use of data from connected cars. They mandate that, among other restrictions, no personally identifying information can leave the car without explicit driver or passenger consent, and that a car cannot collect any more data than is needed to operate its safety system or to perform another task for which the owner has given specific consent. As a result, it is shaping up to be very difficult for automakers to deploy the kinds of in-cabin monitoring systems in the EU that they are developing for American and Asian markets, Anne-Gabrielle Haie, a Belgium-based data protection attorney with DLA Piper, told Motherboard.

The U.S. is another matter. Several laws, ranging from the Fair Credit Reporting Act (FCRA) to the Federal Trade Commission Act (FTCA), provide limited mechanisms to prevent companies from egregiously and repeatedly misusing the kind of data an in-cabin monitoring system will collect. For example, under the FCRA, a car manufacturer would have to notify its customers before providing certain kinds of information to the driver’s insurance agency.

But the laws currently in place are insufficient to protect consumers from the technology that will soon be rolling off production lines, Maneesha Mithal, associate director of the FTC’s Division of Privacy and Identity Protection, told Motherboard. “That’s part of the reason why the majority of [FTC] commissioners have recommended the enactment of federal privacy legislation that would set up the rules of the road in this area,” she said.

ADVERTISEMENT

In-cabin monitoring systems will also be highly attractive to police, who, as Forbes has documented in detail, have been demanding data from connected cars for years. In the U.S., there is nothing to stop police from going after that data, whereas the EU rules place special considerations on the circumstances under which law enforcement can access data that may evidence criminal activity, such as speeding.

The Georgia Supreme Court recently ruled that police must have a warrant before accessing car data, but the case law in this specific space is in its infancy, Chelsey Colbert, the policy counsel for mobility and location data at the Future of Privacy Forum, told Motherboard. “If car manufacturers are worried about law enforcement access, they should consider privacy and security by design, she said. “For example, they could use technologies that don’t collect or store identifiable data.”

On the edge of privacy

Twenty of the largest automakers have promised to self-govern their privacy practices for connected cars and, in 2014, signed on to an unenforceable, industry-created set of principles. There is ample evidence those guidelines won’t stop them from cashing in on driver data, though. GM, for example, is one of the signatories. But in 2018, the Detroit Free Press revealed that the company had collected data on the radio listening habits of more than 90,000 drivers in an attempt to find correlations between what people listened to in their cars and what products they purchased.

Affectiva, Xperi, and Eyeris say that their systems are all designed so that the most sensitive data, such as actual video recordings of passengers, can be processed in the car—known as edge processing—rather than uploaded to a cloud controlled by an automaker or rideshare company. 

But since the automakers are setting their own rules, they could, of course, decide they want that data after all. Cerence’s boasts about the amount of data it uploads to the cloud from vehicles each month suggest its partners—which, in addition to automakers, includes Microsoft, LG, and a number of other tech companies—have little interest in ensuring edge processing for privacy purposes.

“If they install sensors in the car, it’s going to go out of the car and be viewed by the car manufacturer,” Ben Volkow, the CEO of Otonomo, a car data brokerage based in Israel, told Motherboard. “[Automakers] need to get approval, and after you get approval you need to give the driver the opportunity to, at any point, say, ‘please erase my data.’ That’s supposed to be supported, but I’ll tell you in reality it’s definitely not supported.”