Moneyball For Business: How AI Is Changing Talent Management

Using big data and predictive algorithms to help enterprises do a more accurate job of determining who will thrive, who will stay - and who will not. JL

Austin Carr reports in Fast Company:

A survey of 1,000-plus C-suite executives found attracting and retaining talent is their number-one concern. Human resources relies on “biased” evaluations of “irrelevant” staples such as résumés and cover letters. Recruiters review each résumé on average for six seconds; three-fourths of candidates are cut at this phase; and new hires fail 30% to 50% of the time. Productivity software and enterprise tools (give) management access to reams of data on employee activity. The Unilevers and Burger Kings are exploiting data to examine performance and predict potential.

The online games were easy–until I got to challenge number six. I was applying for a job at Unilever, the consumer-goods behemoth behind Axe Body Spray and Hellmann’s Real Mayonnaise. I was halfway through a series of puzzles designed to test 90 cognitive and emotional traits, everything from my memory and planning speed to my focus and appetite for risk. A machine had already scrutinized my application to determine whether I was fit to reach even this test-taking stage. Now, as I sat at my laptop, scratching my head over a probability game that involved wagering varying amounts of virtual money on whether I could hit my space bar five times within three seconds or 60 times within 12 seconds, an algorithm custom-built for Unilever analyzed my every click. With a timer ticking down on the screen . . . 12 . . . 11 . . . 10 . . . I furiously stabbed at my keyboard, my chances of joining one of the world’s largest employers literally at my fingertips.

More than a million job seekers have already undergone this kind of testing experience, developed by Pymetrics, a five-year-old startup cofounded by Frida Polli. An MIT-trained neuroscientist with an MBA from Harvard, Polli is pioneering new ways of assessing talent for brands such as Burger King and Unilever, based on decades of neuroscience research she says can predict behaviors common among high performers. “We realized this combination of data and machine learning would be hugely powerful, bringing recruiting from this super-antiquated, paper-and-pencil [process] into the future,” explains Polli, sitting barefoot on a couch at her spartan office near New York’s Flatiron District on a humid May morning, where about four dozen engineers, data scientists, and industrial-organizational psychologists sit behind glowing iMacs.
Pymetrics is part of a legion of buzzy startups leveraging artificial intelligence, big data, and other tech tools to disrupt the hiring space. Research firm CB Insights expects VC investments in HR-tech startups to hit $2.9 billion in 2018, up 138% year over year. When Polli and I meet, she’s in the midst of raising a Series B round.
What’s driving all this investment? A January 2018 survey of 1,000-plus C-suite executives found that attracting and retaining talent is their number-one concern, outranking anxiety over the threat of a global recession, trade war, and even competitive disruption. Yet human resources, Polli complains, is still an “archaic system” that relies on “biased” evaluations of “irrelevant” staples such as résumés and cover letters. Polli spouts off numbers like a lab-coat-wearing scientist in a disaster movie whom the townspeople perilously ignore: Recruiters review each résumé on average for a mere six seconds; three-fourths of candidates are cut at this phase, often arbitrarily; and surviving new hires ultimately fail in their positions 30% to 50% of the time. With the unemployment rate at an 18-year low and a historic scarcity of mission-critical skills plaguing every industry (there are now more open jobs in the U.S. than there are active job seekers to fill them), talent acquisition has reached a breaking point. “This system is fundamentally not working for companies, candidates, or the economy,” Polli says.Companies are therefore turning to new technologies to help inform increasingly granular employment decisions, from hiring to productivity. “It’s Moneyball for HR,” says Polli, referencing the best-selling Michael Lewis book about the 2002 Oakland Athletics who, led by forward-thinking general manager Billy Beane, upended the game of baseball by applying statistical analysis to build a roster that could compete against its deeper-pocketed rivals. (The 2011 movie adaptation featured Brad Pitt as Beane and earned six Oscar nominations.) Beane’s numbers-based approach valued RBIs and on-base percentages over traditional metrics such as batting average and anecdotal assessments like the look of a player’s swing. The corporate world has been eager to adapt that recruiting model ever since, but until fairly recently lacked the tools to fully assess employees. Now, as companies adopt more productivity software and enterprise tools like Slack and Workday, management has access to reams of data on employee activity. So just as the Oakland A’s used data analysis to determine that the potbellied catcher who almost never struck out was a better bet than the golden-armed Adonis who made old-school scouts salivate, the Unilevers and Burger Kings of the world are increasingly exploiting data to examine performance and predict potential.

“When you have a 4% unemployment rate and a skills gap in every sense for the talent we’re competing for today, you have to use data to win,” says Travis Kessel, senior director of recruiting for Walmart’s Jet.com, which also utilizes gamified testing like Pymetrics. “The war for talent is so hot right now that you can’t afford not to.”

---

After I survived my Pymetrics-designed Unilever test, my results were instantly calculated, determining that I’m a cautious risk-taker, yet 72% more likely to “use trial and error to formulate a plan” than more deliberate strategizing. Pymetrics’s games collect troves of this kind of data, which, once fed to its algorithm, can determine where each applicant might fit within an organization (if at all). For Unilever, Pymetrics matches prospects to seven internal divisions: a person suited for Unilever’s finance department, for example, might not have any trouble solving the probability puzzle that flummoxed me. “It’s like Netflix or Spotify matching you with exactly what [content] you’re looking for,” Polli says.

That’s more than the 2002 Oakland A’s could boast: Billy Beane didn’t have AI to augment his data crunching. Nor did he have computer vision, another tool in Unilever’s kit, which now enables companies to automate initial interviews, using webcams to analyze facial expressions and voice tonality. “We capture tens of thousands of data points–emotions, words you use, active versus passive verbs, how often you say ‘um’–and automatically score [candidates] based on [qualifications] Unilever gave us,” says Loren Larsen, CTO of HireVue, the startup behind the service. “If you never smile, you’re probably not right for a retail position.”
Unilever’s digitized approach enabled it to expand on-campus recruiting in recent years from around 20 colleges to more than 2,500, surfacing a dramatically more diverse selection of candidates. “The best student at New Mexico State is probably as good as the average student at Harvard,” says Larsen. “Businesses are trying to get good talent fast–if Facebook and Google get there first, they’re toast.”
Mike Clementi, HR head for Unilever’s North American operations, says return on investment of its new hiring methodology is substantial: When the company first tested the system for certain entry-level jobs and internships in late 2016, applicant volume shot up 100%, to 275,000 candidates globally. At each step of the application process, algorithms (such as the ones Pymetrics and HireVue developed) sharply narrowed that talent pool, eventually to just 300 final-round candidates in the U.S. and Canada–before applicants interviewed with humans. “We bet pretty big on this,” says Clementi, touting that the company “moved from a process that took four months to four weeks.”
Unilever’s HR costs also went down, Clementi says. And though it’s too early to say whether recent hires will thrive at the company in the long run (they only began their careers last year), the fact that they even received offers is a strong indication of the recruiting machine’s accuracy: Human HR managers extended job offers to eight out of every 10 of the robots’ final-round picks. “In my mind, there’s no question that data and digital automation is the way we’re going,” Clementi says.

---

Several weeks after my visit to Pymetrics, I’m in New Brunswick, New Jersey, visiting Johnson & Johnson’s stately headquarters, where Sjoerd Gehring, the company’s VP of talent acquisition, is showing off the inner workings of Shine, a sleek new platform that’s profoundly altering the 132-year-old healthcare giant’s approach to hiring.

Jopwell cofounder Porter Braswell cautions that an overreliance on AI might compound racial bias at work. [Photo: ioulex]

Shine is a web and mobile product developed, as the name suggests, to illuminate the historically opaque process of getting a job. Analytics power most interactions between employer and job seeker today, but traditionally only one side has any visibility into the data. J&J wants to rectify that to improve the “guest” experience. Now, J&J job applicants can log in to the service to track every step of their journey via a bright, clean interface. Gehring, whose tidy office is surrounded by colorful printouts of product road-map deadlines, compares the experience to monitoring a package delivery on Amazon. “Our vision is to bring a much more consumer-like approach to recruiting,” he says.

When the Dutch-born Gehring arrived at J&J from Accenture’s talent innovation lab three years ago, he discovered that the company’s recruiting process wasn’t so much a black box as a black hole. The company was taking what Gehring calls a “post and pray” approach: Its job listings were filled with jargon, and its hiring process was “broken,” rarely informed by data. “We were flying blind,” he says. J&J had been receiving 1.2 million applications per year for 15,000 job openings, each requiring multiple interviews. It wasn’t uncommon for candidates to endure months of silence only to receive a rejection letter. “It was the definition of a slow-moving, traditional recruiting organization,” recalls Gehring. At J&J’s scale, this not only threatened the caliber of its in-house talent, but also risked tainting the company’s brand with the million-plus people it rejected annually–people who, Gehring says, are sick of companies “treat[ing] them like crap.”
Gehring’s boss, chief human resources officer Peter Fasolo, tells me that the company desperately needed to move away from the “myths” that long had driven its approach to hiring. “Don’t give me anecdotal, ‘I feel’ this does or doesn’t work–go in and study it,” he says. Gehring set about “re­imagining recruiting,” bringing aboard designers and data scientists who scoured for pain points in J&J’s process. The key was building out a cohesive ecosystem that prioritized the user–not simply injecting myriad new technology tools now at their disposal. “A lot of my peers see a new AI tool or a chatbot and say, ‘Oh, let me add that so I can show I’m doing something with AI,’ ” Gehring says. “When you do that year after year, you end up with 40 to 50 tools that are super disconnected and don’t deliver results.”
The Shine team, which Gehring formed in mid-2016, needed to create a personalized experience because, Gehring explains, “candidates are now assessing organizations more than organizations are assessing talent.” He and his colleagues expanded the tracking features so that users can see not only which leg of the journey they’ve reached (e.g., “You’re at Stage 3: Recruiter Screening”) but also receive estimated response times and insights on progress (“15% of candidates get this far”) as well as feedback on what’s coming next. To guide people through the process, J&J partnered with career website the Muse to create 80 original advice-filled videos and articles that are sprinkled throughout Shine.
Another major focus for J&J has been diversity. For its career listings, the company teamed up with Textio, an AI startup that analyzes job descriptions to remove gender-biased language while also measuring which terms are most effective in attracting talent. “If you’re trying to hire competitively against companies like Google and Apple, you really need to find innovative ways to succeed,” says Textio CEO Kieran Snyder. “It’s not an accident that

Amazon uses the word maniacal in their job posts 11 times more often than the rest of the industry.”
When J&J rolled out this system last October, response rates to recruiting write-ups rocketed 24%, and the Shine system’s Net Promoter Score–an industry-standard metric of how likely users are to recommend it–quintupled within four months. Most compellingly, J&J’s Textio-enhanced job listings, dissected to remove gender bias, resulted in a 14% increase in qualified female applicants for STEM roles and a 7% uptick in hires.
This is exactly the type of tectonic shift that AI startups are promising in every field today. But Porter Braswell, cofounder of Jopwell, a career platform designed for minority job seekers, says companies shouldn’t see technology as a panacea to their inclusion problems. Not only is there the risk of machine bias, when algorithms are inadvertently engineered to favor one demographic over another, but diversity is a cultural issue that runs deeper than anything a machine alone can address. Companies still need to have the “awkward and uncomfortable conversations about what diversity actually means and where [they’re] struggling with it,” he says. (A J&J spokesperson says its approach to diversity is multifaceted, including investing in unconscious bias training and STEM education programs for underrepresented minorities.)

Parag Kothari, 22, was just out of college when he applied for a finance position through Shine. He tells me that he found its transparent tracking refreshing and the overall process enjoyable–despite not having landed the job. “It felt like they were looking after candidates as if they were already employees,” he says.
As Gehring and his team get ready to complete Shine’s worldwide rollout by October (the service will be integrated with WeChat in China to appeal to local tastes), they’re paying specific attention to the rejection page so that it can surface additional opportunities. This way, J&J won’t lose candidates like Kothari simply because they didn’t get the first job they applied for.

---

On a recent rainy Tuesday, around 70 developers are gathered for a hackathon at IBM’s Mass Lab, the tech giant’s largest software hub in North America, in Littleton, Massachusetts. A handful of youthful engineers stand at the front of a large conference room, explaining how they’re trying to give a voice to Watson Career Coach, an AI-assisted talent adviser. The bot is designed to help employees navigate their careers at IBM, providing feedback via a mobile app on everything from reskilling opportunities to job advancement. Employees set specific career goals on the app, and Watson will guide them through possible job paths, explain the training required, estimate times to promotions, and provide automated answers to questions.

The team presenting today hopes that speech will make Watson’s chat experience more accessible. Their prototype is very rough–and Watson is having trouble with the local dialect. “Intense accents can be tougher than a foreign language,” says cognitive-software engineer Cameron MacArthur, who jokes that the system is “built to take in Spanish or Mandarin–but not built for Bostonians.”Watson Career Coach is currently a pilot with 12,000 employees but will be available to IBM’s global workforce of 366,000 later this year as part of the company’s commitment to retaining and retraining talent, especially as millennials–a job-hopping generation nearly twice as likely to defect than older colleagues, according to a 2016 Gallup report–become the largest part of the U.S. labor force this year. “Things have changed dramatically with the availability of big data and AI,” says chief HR officer Diane Gherson. “There’s a real skills shortage in this new era.”
Yes, even a company that receives 2.5 million résumés per year for tens of thousands of positions still faces a talent deficit, and innovations like Watson Career Coach are helping IBM recalibrate for the future. After all, the company boasts more than 1,500 blockchain employees alone, a field that didn’t exist a decade ago. How better to hire for those positions than by home-growing talent? The company invests $500 million annually in employee learning and retraining. It runs an internal developer academy and partners with Coursera and Udacity on a portfolio of online courses. Watson keeps track of employee skill advances via a digital badging system: IBMers have earned 40,000 AI certifications for completing courses in areas such as architecture foundations and conversational services, for example. “We know what your skills are and will tell you if your skills are in decline so you can pivot,” says Gherson. “If you’re a Java programmer, and the demand for that skill is going down, [Watson] will say, ‘Here are six blockchain programmers. They took these courses and had these [work] experiences, and they’ve since been promoted three times.’ ”

Gherson recognizes that these training efforts are only likely to make employees more enticing to competitors. It’s one reason why the company has also invested seriously in workforce analytics. Through Watson, executives and supervisors can also analyze employee performance and growth–a “Fitbit for managers,” as Gherson has called it–that will automatically alert them if a team member, say, isn’t earning enough based on his or her proficiency. Through patented proactive retention algorithms, the system is getting to a point where IBM is starting to anticipate employee departures. By comparing data trends of departed workers with current ones, the company is able to identify patterns of flight risk.
Last year, the company says this warning system resulted in roughly $100 million in net savings based on how much it would have cost to replace that lost talent.

---

On June 14, Billy Beane delivered the opening keynote at the Recruiting Automation Summit in San Francisco. Now the Oakland A’s executive VP of operations, he has become a fixture at hiring and data conferences in recent years, and on this bright Thursday morning, Beane once again sings the gospel of Moneyball. “Moneyball was about paying for the right skills that would help us win–not who looked good in a baseball uniform,” Beane told the audience of 400, stressing that data–not myths–are key to hiring the right talent.
Given that the Oakland A’s haven’t won a World Series since 1989, has the analogy gone too far? Wharton School professor J. Scott Armstrong has applied Beane’s statistical analysis to his research on recruiting–one paper he coauthored was called “The ‘Moneyball’ Approach to Hiring CEOs”–but even with the rise of artificial intelligence Armstrong hasn’t seen enough scientific evidence to support its widespread adoption in HR. “It’s mass hysteria: ‘Everyone else believes it, so we believe it too,’ ” he says of the industry.
It’s likely, though, that companies will only grow more dependent upon algorithms for hiring talent. For some job seekers already at the mercy of the machine, it’s a scary prospect, as evinced in the many online forums, such as Reddit’s popular “Recruiting Hell” section, where applicants share horror stories as well as tips on how to game new-age systems like Pymetrics and HireVue. “Make sure to look at the camera like it’s a person’s eyes, [because] it captures that,” a user cautioned of HireVue’s automated video interviews. Others talk of the unsettling experience of completing Pymetrics’s online assessment–only to receive a rejection email from an employer almost immediately after the computer tabulates the results. “I spent two hours filling out [the company’s] application and playing 12 of their fucking games,” one user wrote. “I got an email about 10 minutes after finishing the games telling me I had automatically been turned down from the position and will be unable to apply to any job in the company for 12 months . . . What the fuck?” (A spokesperson for Pymetrics clarifies that rejected candidates are allowed to apply to different jobs at the same company and are encouraged to do so in rejection notices.)
Laszlo Bock, the cofounder of data-science startup Humu and former HR guru at Google, is wary of this increasingly automated future. He advises companies to proceed with caution when implementing AI systems, lest they inadvertently punish the very people they’re trying to attract. “There’s risk of tremendous harm,” Bock says. “If you’re building a new sales system and you mess up, you don’t get the sale, but if you’re building a people-related system and you mess up, you have ruined someone’s life. You have made someone feel terrible. You have just shut someone out of an opportunity they deserve.”

Companies must strike the right balance between human judgment and machine learning, says Dane Holmes, head of human capital management at Goldman Sachs, which now uses HireVue and is exploring the use of virtual reality in recruiting. Otherwise, this tale of modern Moneyball may start to become more like an episode of Black Mirror. “We have no interest in creating a dystopian society here,” Holmes says. “We don’t ever view this as a process where you turn on an algorithm and wait for it to spit out [the right hire] in an app.”
Reid Hoffman, the venture capitalist and LinkedIn cofounder, agrees. His new book on building organizational efficiency, called Blitzscaling, details the novel ways startups are growing fast, but even he advises against completely succumbing to modern hiring techniques. “Look, I don’t think Billy Beane said, ‘I’m just going to hire whoever the stats say [to hire].’ He said, ‘The stats are telling me this player is a lot more valuable than the market thinks, so I’m going to take a look,’ ” Hoffman says.
Hoffman’s words reminded me of something I heard during my visit to IBM’s campus in Massachusetts. After the hackathon, I had a chance to catch up with a small set of developers, all fresh out of school in their early twenties, including Cameron MacArthur, the cognitive-software engineer helping to build the company’s career-analytics platform. Does he ever worry about eventually engineering himself out of a career?
With senior IBM executives listening, MacArthur, with a sunny smile, told me he’s not concerned. After all, this is just a learning system that gives you career choices. “That’s the crux of AI: The whole special sauce is that you can say, ‘No, you’re totally wrong. I hated that job.’ And then the computer can learn, ‘Oh, that’s really interesting: If you hated that job, then maybe you’ll like this one?’ ” the young engineer says. “The approach of, ‘The computer is deciding where you belong’–I don’t think any company wants that.”
I shook MacArthur’s hand and wished him the best of luck in his future career. In this data-driven era of employment, he probably won’t need it.
Update: This story has been updated to include a clarification from a spokesperson from Pymetrics, who explains that job candidates rejected by its software are allowed and encouraged to apply to different jobs at the same company they applied to.