An Emotion-Tracking Artificial Intelligence For Drivers In Autonomous Vehicles

It's when there are no longer drivers, but only passengers, in autonomous vehicles that the data capture gets really interesting. JL

Khari Johnson reports in Venture Beat:

An MIT Media Lab spinoff, (this) is Affectiva’s third service for tracking the emotional response of product users. It’s a  strategy to build emotional profiles drawn from smart speakers, autonomous vehicles, and other platforms with a video camera or conversational interface. In semi-autonomous vehicles, Affectiva will monitor drivers to increase safety and facilitate the handoff between human and machine. Once fully autonomous vehicles hit the road, Affectiva could switch its focus to passengers, using voice and facial recognition to personalize things inside a vehicle.
Affectiva announced the launch of its Automotive AI service that lets creators of autonomous vehicles and other transportation systems track users’ emotional response. The Automotive offering is Affectiva’s third service for tracking the emotional response of product users. It’s part of a long-term strategy to build emotional profiles drawn from smart speakers, autonomous vehicles, and other platforms with a video camera or conversational interface.
An MIT Media Lab spinoff, Affectiva launched a voice analysis tool last year for the makers of AI assistants and social robots, but its affective computing services have been available to marketers and advertisers since 2010.
To develop its solution for cars, Affectiva has been working with OEMs, vehicle safety system providers like Autoliv, and robotaxi startup Renovo over the past 18 months, an Affectiva spokesperson told VentureBeat in an email.
Inside vehicles, Affectiva can use a combination of traditional RGB cameras and — starting today — near infrared cameras to create confidence scores based on things like the number of eyelid blinks per minute, and AI models built to identify drowsiness, yawning, and other signs of driver fatigue.
Factors like whether the car is on a highway or residential street or the weather conditions could also be used to determine whether a driver should be in control at any given moment.
In semi-autonomous vehicles, Affectiva will focus on monitoring drivers to increase safety and facilitate the handoff between human and machine.
“It needs to know whether the driver is paying attention or not. Is the driver tired? Is the driver asleep? Is the driver watching a movie? It really needs to understand the state of the driver, or the copilot in this case, so that it can safely transfer control back to a human driver,” CEO Rana el Kaliouby told VentureBeat in an interview.
The use of facial recognition to comprehend it a driver is under the influence of drugs or alcohol will not be part of Automotive AI at launch but is part of the product’s roadmap, El Kaliouby said.

Once fully autonomous vehicles with no driver hit the road, Affectiva could switch its focus from drivers to passengers, using voice and facial recognition to recognize occupants and personalize things such as temperature and lighting inside a vehicle. In a fully autonomous vehicle world, Affectiva could be used to sense the number of riders in the car, their age, race, and gender, and attempt to understand social dynamics like whether they’re friends, coworkers, or family.
“Maybe it’s about personalizing the music or the content you’re watching or even the ambience of the car, like the general temperature and all that, based on who is in the car and also what are they feeling,” she said.

El Kaliouby believes Affectiva is well-positioned to enter the automotive market to compete with companies like Seeing Machines because it uses AI to continuously improve its product and it has now expanded to include a range of devices that can use cameras and microphones to detect emotion — from smart speakers to cars.
Android Auto and Apple’s CarPlay have been around for some time now, but in the past year carmakers like Ford, Mercedes-Benz, and Hyundai have made voice control available for their vehicles, and in January, Toyota agreed to bring Alexa into their cars.
More autonomous vehicles are expected to be on the road in the years ahead, and more than half of U.S. households are expected to bring smart speakers with digital assistants like Siri, Alexa, and soon Bixby into their homes in the next five years.
“Eventually, these devices will all be tied together, and ideally they would have a sense of who you are,” she said. “We feel strongly you should own your emotional profile, but I would imagine that it would make a lot of sense if these devices we already are interacting with on a daily basis had more context around who we are, what we’re doing, how we’re feeling, and what we’re thinking that could make for better user experiences.”