Your Car May Soon Be Able To Read Your Face

Of course once driverless cars become more common, riders may want to read their car's face...JL

John Quain reports in the New York Times:

There are practical reasons for detecting a driver’s emotional state: A calm driver is safer. Cars that recognize when you are prone to road rage could play mellow jazz to soothe you. Biometrics make cars more difficult to steal. Using cameras with facial recognition and biometric indicators, automakers personalize the driving experience, adjusting seats and driving modes. They may anticipate your wants and desires by playing music based on your mood.

When you shop for cat food online, Amazon knows who you are. When you look for the best airfares for summer vacation, Google knows who you are. Soon, when you get behind the wheel, your car may recognize you, too.

Using cameras with facial recognition software and other biometric indicators, automakers are looking to personalize the driving experience with cars that stare back at you, quietly adjusting seats and driving modes. They may even anticipate your wants and desires by playing your favorite music based on your mood. And it’s not only about convenience, but also about the possibility of improving safety and security.

“It’s not just about personalization,” said Zachary Bolton, a systems and technology engineer at Continental Automotive Group, explaining the sophistication of such systems. “We can use the gleam, the twinkle in your eye to determine precisely where you’re looking.” Engineers can then dynamically adjust the so-called human machine interface, putting critical information, say, about a stalled car up ahead, or the fact that you are about to exceed the speed limit, directly in a driver’s line of sight on the dashboard or in a display on the windshield. Conversely, by tracking downward eye movements, the car could “see” that a driver was distracted and sound a warning.

Continental has already demonstrated in-car systems that allow drivers to register their faces using something as simple as a driver’s license picture. An interior infrared camera is used to overcome potential obstacles like sunglasses, which would stymie a conventional video camera. Putting the camera in the center instrument cluster also helps to pinpoint the driver’s eyes, even if she’s wearing a hat. The biggest technical challenge — glare caused by sunlight — can be filtered out using machine learning, Mr. Bolton said.

Once the car knows who you are, systems in vehicles like the Chrysler Portal concept car would automatically adjust the seat for maximum comfort, select a driving mode (for example, one driver likes to let the car do most of the work; another likes taking control in sport mode) and suggest a destination based on the owner’s past behavior.

Watching a driver’s face can also give a car important clues about that person’s state of mind. For several years, carmakers like Ford and tech companies like Intel have been interested in determining whether a driver is happy or sad. Depending on your mood, a car could change its tune, playing a bouncy Beach Boys song and changing the interior lighting to improve your attitude.

Honda’s NeuV concept car, for example, has a large customizable LCD dashboard and a cloud-connected, onboard computer that uses artificial intelligence to interact with drivers. NeuV employs what the company coyly refers to as an “emotion engine” to grease the wheels of the conversation, and its automated personal assistant can read “facial skin vibrations” to help it isolate the driver’s voice and better understand spoken commands.

There are practical reasons as well, designers say, for detecting a driver’s emotional state: A calm driver is a safer driver. So cars that recognize when you are becoming angry and thus prone to road rage could potentially quell annoying bells and chimes in the car and play some mellow jazz to soothe you.

By replacing keys and remote control fobs, biometrics like facial recognition could also make cars more difficult to steal. In its prototype FF 91 sport utility vehicle, the electric car start-up Faraday Future uses an external camera mounted in the door frame to detect the car’s owner and automatically unlock the vehicle. Such techniques can create new security challenges, however.

In this digital age, our faces are everywhere: in online company profiles, on Twitter accounts, even tagged in friends’ Facebook accounts. Finding an image to print out and foil a car’s facial recognition system would not be very difficult.

Fortunately, engineers have devised high-tech countermeasures. Stereoscopic video cameras, for example, can tell the difference between a flat image and a three-dimensional object. Continental’s cameras measure the distance of reflected light off various parts of a person’s face, ensuring that it is a real object rather than a high-resolution shot of the owner’s visage.

“Some systems have added blink detection and aliveness detection,” said Steve Grobman, chief technology officer for security at Intel. But he acknowledged that it was still possible to thwart such technology. “We had 3-D masks printed that we ordered and were able to trick some systems,” he said. But he said most thieves would be unlikely to go to such extreme lengths to steal a car.

“It all depends on the level of accuracy you need,” said Yoni Heilbronn, chief marketing officer of Argus Cyber Security, which works with automakers to short-circuit hacking threats. “Retina scans are even better than facial recognition” as a potential solution, he said, “but by adding another level of authentication you lose some of the convenience.”On the other hand, high-tech personalization could be used not only to create amenities for single owners, but also to instantly adapt a vehicle to suit a variety of drivers. Valets, for example, could be automatically prevented from accessing personal information in a navigation system or driving faster than, say, 30 miles per hour. In a ride-sharing situation, such systems could also be used to quickly tailor a car’s interior to the physical characteristics of different drivers and passengers.

Rental cars would be easier to operate and safer, Mr. Bolton of Continental said. “If I know where your head is and where your eyes are,” he said, “I can adjust the position of the steering wheel and the mirrors so you don’t have to fumble looking for the right buttons.”

Even airbags could be fine-tuned, reducing the intensity of their deployment depending on the size and position of a driver or young passenger.

Some elements of the personalized driving experience are already coming to cars. By the end of the year, Ford plans to add Amazon’s Alexa personal assistant to some of its cars, said Dave Hatton, manager of Ford’s mobile applications for connected vehicles. It will not only allow personalized music stations to play with a voice command, but also enable drivers to juggle chores like adding items to an existing grocery list with just a few words.

Such convenience may come with some trade-offs on privacy.

“It’s a huge concern,” said John Simpson, privacy project director at Consumer Watchdog, a nonprofit advocacy group. “All that data is in some database without your consent or knowledge about how it’s going to be used,” he said, adding that there was little if any current government regulation to curtail such use.

Using traffic-tracking programs like Waze, some consumers have already signaled their preference for convenience in return for giving up some information like their location. Programmers also point out that such services are optional: You don’t have to let the company track you — but then you may get stuck in traffic for 45 minutes.

Today, basic biometric technologies like facial recognition software are used for everything from signing into Windows laptops to thwarting toilet paper thieves in Beijing. Fingerprint readers are commonly used to unlock smartphones. As consumers become more accustomed to such systems, the introduction of the technology in vehicles may seem like a natural evolution rather than a creepy intrusion.

And it could be fun. Consider the entertainment and social media repercussions in the vein of James Corden’s “Carpool Karaoke” segments. A built-in camera could record and broadcast your singalong on Facebook or Twitter — assuming the car was in autonomous driving mode. Of course, drivers could grab quick selfies on the road, too.

“It’s a novel ideal,” Mr. Bolton said, “but remember that an infrared camera makes your face look a little like a skeleton, so it’s not that flattering.”