US Gov't Study of Facial Recognition Systems Finds 'Empirical Evidence' Of Bias

Age, race and ethnicity. That includes pretty much everybody. JL

Jon Porter reports in The Verge:

A new study has found that many of the world’s top facial recognition algorithms are biased along lines of age, race, and ethnicity. The National Institute of Standards and Technology (NIST), found algorithms currently sold can misidentify members of some groups up to 100 times more frequently than others. It found “empirical evidence” that characteristics such as age, gender, and race impact accuracy for the “majority” of algorithms. The group tested 189 algorithms from 99 organizations, which together power most of the facial recognition systems in use globally. “The technical considerations cannot be divorced from the social implications.”

A new federal study has found that many of the world’s top facial recognition algorithms are biased along lines of age, race, and ethnicity. According to the study by the National Institute of Standards and Technology (NIST), algorithms currently sold in the market can misidentify members of some groups up to 100 times more frequently than others.

NIST says it found “empirical evidence” that characteristics such as age, gender, and race impact accuracy for the “majority” of algorithms. The group tested 189 algorithms from 99 organizations, which together power most of the facial recognition systems in use globally.

The findings provide yet more evidence that many of the world’s most advanced facial recognition algorithms are not ready for use in critical areas such as law enforcement and national security. Lawmakers called the study “shocking,” The Washington Post reports, and called on the US government to reconsider plans to use the technology to secure its borders.

Lawmakers called the results “shocking”

The study tested “one-to-one” checks, used to match someone against a passport or ID card, as well as “one-to-many” searches, where someone is matched with a single record in a larger database. African American women were inaccurately identified most frequently in one-to-many searches, while Asians, African Americans, Native Americans, and Pacific Islanders were all misidentified in one-to-one searches. Children and the elderly were also falsely identified more. In some cases, Asian and African American people were misidentified as much as 100 times more than white men. The highest accuracy rates were generally found among middle-aged white men.

The NIST study relied on organizations voluntarily submitting their algorithms for testing. But missing from the list was Amazon, which sells its Rekognition software to local police and federal investigators. Previous studies have raised concerns about the accuracy of Amazon’s system, and AI researchers have called on the company to stop selling its “flawed” system. Amazon claims that its software cannot be easily analyzed by NIST’s tests (despite the fact tech companies with similar products have no problem submitting their algorithms) and its shareholders have resisted calls to curb sales of Rekognition.

Experts say bias in these algorithms could be reduced by using a more diverse set of training data. The researchers found that algorithms developed in Asian countries, for example, did not have as big a difference in error rates between white and Asian faces.

However, even fixing the issue of bias won’t solve every problem with facial recognition when the technology is used in ways that doesn’t respect people’s security or privacy.

“What good is it to develop facial analysis technology that is then weaponized?” Joy Buolamwini, an AI researcher who has spearheaded investigations into facial recognition bias, told The Verge last year. “The technical considerations cannot be divorced from the social implications.”