AI Can Now Tell Your Boss What Skills You Lack - and How To Get Them

Whether knowing that employers are reviewing the data their coursework generates will discourage employees from signing up - or whether they will object to being judged in this way by AI systems, is one of the questions this approach raises. JL

Elizabeth Woyke reports in MIT Technology Review:

Companies that subscribe to training programs see which of their employees are earning top scores in classes; how their employees’ skills measure up to their competitors’; and what courses would help fill any knowledge gaps. Companies will be able to access the tool, which uses machine learning to derive insights, in the online dashboard of their profiles. Online-learning providers are using AI to match learners with courses, assess their ability, and tweak class content in response to feedback.
Here’s the conundrum with corporate online learning: there are so many classes available from sites like Coursera, edX, and Udacity that companies don’t know what content to offer their employees. And once companies do choose a learning program, it’s tough for them to figure out what skills their employees pick up and to what degree they’ve mastered them. They need an objective metric to evaluate proficiency.
A new AI-powered tool developed by Coursera aims to be that metric. The feature, which the Bay Area startup announced, lets companies that subscribe to its training programs see which of their employees are earning top scores in Coursera classes; how their employees’ skills measure up to their competitors’; and what courses would help fill any knowledge gaps. Companies will be able to access the tool, which uses machine learning to derive insights, in the online dashboard of their Coursera profiles later this year.The new feature is just one example of the ways online-learning providers are using AI to match learners with courses, assess their ability, and tweak class content in response to feedback. Coursera has a data-science team that does everything from “collecting and storing data in a warehouse to interpreting information for making internal decisions to building algorithms that feed back into the site,” according to Emily Glassberg Sands, who leads the group. Udacity’s AI research team, which it established in 2017, analyzes student sentiment to see how lessons can be improved and computes whether learners like the changes. Udacity has also used AI-based chatbots to help students find relevant courses and answer common questions during the enrollment process. edX says it experiments with AI to increase how well people learn and teach.
By addressing some of the most pernicious challenges in online learning, these AI features could inspire more people—and companies—to sign up for such training. Increasing the effectiveness of educational programs and measuring their impact are the top two priorities for companies that pay to train their staff, according to a 2017 survey of US-based businesses by Training magazine.
The market has plenty of room to grow; Coursera’s business program, which targets companies, currently has 1,400 customers. (Overall, Coursera has 31 million learners, which makes it the world’s largest online-learning provider.) “The Achilles’ heel of the corporate learning industry is that no one knows how to demonstrate the return on investment,” notes Leah Belsky, who heads the Coursera for Business program. “Companies know their people need to learn new skills to stay competitive, but they have a hard time communicating what the value of that learning is.”
Coursera, too, wanted to be able to quantify the benefits of its classes. So a year and a half ago, its data-science team began developing machine-learning algorithms to map the 40,000 skills taught on Coursera’s platform. First, the team used natural language processing (NLP) to determine how often instructors mentioned particular concepts during their classes. That information helped it identify which classes imparted which skills. Coursera can get some of that data by simply asking its instructors. But it also uses NLP because educators often think of themselves as teaching theoretical concepts while learners want to know what specific tools and technologies they will master, says Glassberg Sands.

Companies that subscribe to Coursera will be able to see how they rank compared to their competitors in specific skills.

Coursera

Most recently, the team incorporated a psychometrics methodology called item response theory (IRT) into some of its machine-learning models to gauge learners’ abilities based on how they performed on Coursera quizzes and assignments. The approach allowed the team to measure proficiency in a given skill area across learners who were answering different questions of varying difficulty, says Glassberg Sands. That’s important because a more advanced learner will, on average, take harder courses on Coursera and attempt harder questions than a beginner. The IRT model accounts for that by figuring out what skill a particular quiz question is assessing and the relative difficulty of that quiz question before evaluating a learner’s proficiency. “What you get is a percentile for your employees in each skill area, relative to whatever comparison group you choose, whether that’s all the companies on Coursera or just companies in your industry or your country or companies of a similar size,” says Glassberg Sands.
Most online-learning providers let their corporate customers see which of their employees are enrolled in classes, how they’re progressing, and the feedback they’re providing on their experiences. Coursera says machine learning enables it to examine how its millions of learners are performing across its platform and provide predictive insights, such as a given learner’s skill level, along with more standard student metrics. In theory, these machine-learning models should also get smarter as learners and instructors create more data that can be fed back into the system.

Coursera expects the skills-benchmarking information, which will be updated daily, to be useful to learning-and-development specialists, HR professionals, and hiring managers, who previously would have had to guess what skills other companies were acquiring and which of their employees were most expert in, say, a particular programming language. But it’s also bound to rankle some Coursera learners who don’t want their bosses making employment decisions based on their online-learning performance.
Glassberg Sands and Belsky contend that the technology will benefit individuals in other ways, such as bringing attention to skilled people who might otherwise be overlooked. Software giant Adobe, which has been testing the new Coursera feature for the past month, agrees. “We probably have some experts who are located in remote offices where we may not be able to identify their level of talent easily,” says Justin Mass, who leads Adobe’s digital-learning program. “These features give us a true sense of what our employees know and are good at, which should help us assess talent more intelligently.”