Is Microsoft Software To Grade Employee Meeting Body Language, Facial Expressions Based On False Premise?

Microsoft has patented a surveillance software tool to grade organizational meeting 'quality' by assessing such factors as body language, facial expressions and speech patterns as well as monitoring whether meeting participants - whether remote or in person - are simultaneously texting or surfing the internet. 

The potentially false premise on which this system is based is that there is a 'normal' way for people to behave in meetings. Given what has been learned over time about optimal performance - especially since the beginning of the pandemic - this software appears to be intrusive and subjective without offering any causal link to outcomes. The greater danger is that it stifles innovation, initiative and diversity which could potentially make it counterproductive as the economy moves to greater flexibility in work location and style. JL

The BBC reports:

Microsoft filed a patent for a system to monitor employees' body language and facial expressions during meetings and give events a "quality score." It allows managers to keep track of individuals' use of Microsoft's software. Sensors could record which invitees attend a meeting; body language and facial expressions; the time each participant spent contributing; speech patterns consistent with boredom and fatigue. Employees' mobile devices could monitor whether they were simultaneously texting or browsing the internet. "Surveillance software operates on the false premise there is a normative way people work optimally." Microsoft is already under fire over a separate "productivity-score" tool.

Technology giant Microsoft has filed a patent for a system to monitor employees' body language and facial expressions during work meetings and give the events a "quality score".

A filing suggests it could be deployed in real-world meetings or online virtual get-togethers.

It envisions rooms being packed with sensors to monitor the participants, which could raise privacy concerns.

Microsoft is already under fire over a separate "productivity-score" tool.

'Employee-surveillance tool'

That feature was introduced last year but came to prominence only after a public demo at a corporate event.

It allows managers to keep track of individual workers' use of Microsoft's Office 365 software - including Outlook email and the Teams meeting and Excel spreadsheet apps.

Microsoft points out the facility is not enabled by default and suggests its primary goal is to identify IT shortcomings.

But critics say it is effectively an employee-surveillance tool.

Facial expressions

Companies do not always make use of patents they register.

But they often reveal ideas in development before they appear in commercial products.

Details of the "meeting-insight computing system" were filed in July, ahead of being made public this month.

They say the sensors could record:

• which invitees actually attend a meeting
• attendees' body language and facial expressions
• the amount of time each participant spent contributing to the meeting
• speech patterns "consistent with boredom [and] fatigue"

They also suggest employees' mobile devices could be used to monitor whether they were simultaneously engaged in other tasks - such as texting or browsing the internet - as well as to check their schedule to take into account whether they had had to attend other meetings the same day.

• Is my boss allowed to spy on me?
• 'I monitor my staff with software that takes screenshots'
• H&M fined £32.1m for illegal employee surveillance

All that information would then be combined with other factors, such as "how efficient the meeting was, an emotional sentiment expressed by meeting participants, [and] how comfortable the meeting environment was", into an "overall quality score", Microsoft says.

'False premise'

The patent also suggests the technology could be used to identify problems that can make meetings ineffectual.

For example, if too many people were packed into a room or if the location becomes uncomfortably hot in the afternoon sun.

"Many organisations are plagued by overly long, poorly attended, and recurring meetings that could be modified and/or avoided if more information regarding meeting quality was available," the patent says.

But one privacy campaigner suggested the system would be "invasive" and a "major step back for workers' rights".

"This type of employee surveillance software obstructs diversity in workplaces by operating on the false premise that there is a uniform, normative way that people work optimally," Big Brother Watch director Silkie Carlo said.

"A lot of surveillance tech is marketed as 'innovative' but in reality is astoundingly retrograde."

'Discriminatory outcomes'

A Microsoft representative said it applied for "many patents" to protect its engineers' work

"However, the application of a patent doesn't necessarily indicate that the technology described will be implemented in a product," they added.

It comes as the Trades Union Congress (TUC) releases a report into the use of technology in the workplace - including when artificial intelligence (AI) starts to take on management functions.

The report "looks at exactly these types of technologies and the dangers they pose of unfair and discriminatory outcomes for workers", TUC policy officer Mary Towers said.

"In particular, we advocate more worker consultation and transparency over when technologies like facial recognition and speech recognition are used and how they are being used to make decisions about people at work," she added.