Why Most Csuite Hiring Practices Don't Effectively Predict Superior Performance

Despite significant advances in the methodologies for identifying suitable candidates for Csuite jobs who are likely to provide superior performance, many organizations continue to fail at hiring the best candidates, largely because they rely on familiar practices including face to face interviews and overdependence on search firms. 

These are not effective predictors of future performance because they overweight biases like affability and search firms' own 'models' - for which they overcharge the client - thereby focusing attention on attributes who benefit certain individuals in the enterprise - or search firms' financials - rather than the organizations' strategic priorities. JL

Barry Conchie and Sarah Dalton report in MIT Sloan Management Review:

Despite employing due diligence when assessing whether someone will perform well in the role, most companies’ predictive powers rank alongside astrology in reliability. Despite advances in the fairness, validity, and reliability of candidate assessments, organizations haven’t proved very effective in predicting superior job performance. Reasons include: Likability in interviews has a disproportionate impact on how a candidate is weighed but has no correlation to how well they can perform in a specific role. Search firms are incentivized to advocate for candidates already on their rosters rather than identifying new applicants who might be better suited. Reference checks are unreliable and have half the validity of structured interview assessments in predicting future job performance.

Selecting the right candidate for an executive role ranks high on the list of the most consequential organizational decisions. But despite employing the common tools of due diligence when assessing whether someone will perform well in the role, most companies’ predictive powers rank alongside astrology in reliability.

 

Organizations consistently make poor hiring decisions, but they rarely rethink their approaches. Despite advances in the fairness, validity, and reliability of candidate assessments, these haven’t proved very effective in predicting superior job performance. Every selection decision is a prediction, but instead of ensuring those predictions are accurate, companies selecting leaders make easily correctable errors. 

 

1. Selection Decisions Are Left to Chance

 

We met a CEO and founder who was nearing retirement. His company had enjoyed success due to an effective product mix and sparse competition. Recently, new players had captured a worrisome chunk of the market, and this organization now had to learn to play offense and defense. 

 

The new circumstance required a change in leadership, yet the kind of agile leaders required were (and are) in short supply. The CEO knew what he wanted. “I just need a strong, firm handshake,” he said. ”I’ll know when I can see the whites of their eyes!” The CFO position he needed to fill remained vacant as he had hired and burned through two of his favored candidates within a year. We asked why they left. “They weren’t up to the job,” he explained, even though they both apparently had strong handshakes and made the right impression. This is a pattern of behavior we see all too often — selection decisions made on instinct and gut feeling.

 

Although this is an extreme example, consider the implications for your own organization’s selection methods. How many people are hired because they effectively wooed the interviewers? What biases are inherent in interviews that hinder an organization’s ability to be objective?

 

2. Face‑to‑Face Interviews Reward Likability

 

The biggest impediment we see to effective selection is an overreliance on face‑to‑face interviews, and almost every company uses them.Simply put, face‑to‑face interviews are a waste of time unless they have a clear focus, use meaningful assessment criteria, and employ effective questioning. Likability in interviews has a disproportionate impact on how a candidate is weighed against others. Generally, a candidate’s affability will have no correlation to how well they can perform in a specific role — we’ve checked.

 

If these types of interviews are a key component of your organization’s selection process, you shouldn’t be surprised if your company is made up of the kind of people your hiring managers could be friends with. A company that struggles with diversity will see progress toward equitable representation only when its leaders realize that their organization reflects those whom the hiring managers are prepared to allow in. When the most talented, capable candidates don’t get along well with the interviewers, it hinders companies that want to appoint the best leaders to drive exceptional performance.

 

3. Unstructured Interviews Show Poor Validity

 

The weaknesses of face-to-face interviews are compounded when there is no formalized process by which the interviewer conducts their questioning or evaluates candidate responses. Every candidate will have a different interview experience as the hiring manager racks their brain for esoteric questions they believe will provide some insight into the candidate. Whatever companies think they learn from candidate responses, we can guarantee that the outcome is nonpredictive in terms of performance.

 

Unstructured interviews cannot meaningfully predict future performance because they rely on the subjective opinions and interpretations of both the interviewer and the candidate. Unstructured questions introduce variability into the selection process, and such variability cannot produce a reliable prediction of employee effectiveness.

 

Some organizations attempt to apply structure to their interview process, creating templates of questions that match the requirements of the role. However well-intentioned, these often fail on two fronts. First, typically no credible research has been conducted to explore the range of responses an interviewer might encounter and which of them is more predictive of job performance. Second, interviewers introduce their own preconceptions and biases when interpreting a candidate’s responses. No two interviewers will interpret an interaction in quite the same way, and the recommendations that follow are clouded by misjudgement. 

 

4. Companies Mislead Candidates and Damage Their Brand

 

Face‑to‑face interviews often have the unintended effect of encouraging candidates who are not likely to be selected. In part, our natural desire to have positive social interactions means that interviews are usually friendly, affirming conversations. Companies want applicants to feel valued during the process, and this can result in inadvertently misleading candidates about their chances if communication doesn’t set clear expectations.

 

Mismanagement of the process in this way is damaging to the candidate as well as the company’s recruitment brand. Too few organizations apply serious care to handling the outcome of selection decisions, particularly the experience of unsuccessful candidates. Companies that get this part wrong — effectively “ghosting” those they have rejected — are undermining their future recruitment efforts one jilted candidate at a time.We recommend sending a disclaimer to candidates that clearly outlines expectations for the process. This can greatly reduce the confusion and uncertainty that candidates face. Companies that adopt such a disclaimer go a long way toward ensuring that their brand will be viewed as fair and transparent by both successful and rejected candidates.

 

5. 360-Degree Assessments Are Subjective at Best, Biased at Worst

A 360-degree assessment requires individuals who are peers, direct reports, work partners, or supervisors to score a leader on a series of defined characteristics. Research has shown that individuals demonstrate flaws and biases when evaluating others, and aggregating these flawed judgments doesn’t correct these errors. Further, 360 assessments have a significant history of reinforcing the biases that cause diversity challenges and, a growing body of research indicates, lead to unfair judgments of women and minority individuals. These assessments also introduce confidentiality issues for candidates. This is especially true for executives who might prefer to pursue other career opportunities discreetly, not tip off their current employer via requests to complete 360 assessments. Violating this confidentiality can diminish a hiring company’s brand and will lead to the early withdrawal of otherwise excellent candidates.

 

6. Traditional Search Firms Have Conflicts of Interest

 

Traditional candidate search firms are a juggernaut in employee selection, especially at the leadership level. Over the past 20 years, their role has expanded beyond simply identifying eligible candidates. Many executive search firms have acquired assessment tools so that they can market themselves as full-service selection, succession, and guidance providers. These companies search for candidates, vet them internally, and then make recommendations to their clients.

 

This model has a troubling flaw. Search firms that identify candidates and then determine their capabilities are guilty of a serious conflict of interest. They are most profitable when they can quickly recommend candidates, secure their commission, and then move on to the next assignment. They are implicitly incentivized to advocate for candidates who are already on their rosters rather than identifying new applicants who might be better suited for a role.

\ 

The assessments that search firms use need to be broad enough that their candidates will fit a role somewhere, and quickly, but lack predictive value because they rarely measure the long-term performance of their recommendations. Further, the search industry is largely unregulated, subject only to voluntary codes of conduct, unlike the assessment industry, which has to meet stringent legal requirements for reliability, validity, and fairness.

 

Occasionally, we have the good fortune to work with highly professional and talented individuals employed by a search firm. They distinguish themselves from their peers by focusing entirely on the needs of their clients rather than the candidates on their roster. Search-firm professionals who become a candidate’s advocate lose all objectivity in a way that is damaging to their credibility and is not helpful to the candidates or the companies that pay their fees.

 

We argue that traditional search firms exert too great an influence over all aspects of executive selection and that few companies will succeed by consuming the entire menu these firms provide. The expertise of search firms lies in identifying candidates who have the requisite skills and experience for the role in question. The assessors are then able to objectively measure who will most likely succeed. By splitting sourcing and assessing between independent parties, the conflict of interest is resolved and the company can trust that the best, rather than the first, candidate will be hired.

 

7. Confidential Reference Checks Are Unreliable

 

Every failed executive received glowing references from someone who vouched for them, claiming to know them well. So why did they fail? How can it be that a referee — someone who is prepared to put their reputation on the line to vouch for a candidate — can get it so terribly wrong?

 

The reason is simple: People aren’t that good at picking people — even people they know well. A close working relationship with someone doesn’t confer special powers of evaluation. You might work shoulder to shoulder with someone for 10 years and believe you know them well, but your experience will be based on a unique set of circumstances in one role. The characteristics observed by a referee may not translate to an elevated role in a new company with a new boss, different expectations, and unique systems and processes. On balance, professional referees are no more capable of evaluating a person than anyone else, and companies should stop assigning weight to their perspective.

 

In their landmark 1998 research paper, “The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings,” Frank Schmidt and John Hunter found that reference checks had half the validity of structured interview assessments in predicting future job performance. Yet reference checks persist but add little value to selection decisions.

 

Do reference checks have any utility? We believe that reference checks should always be carried out and diligently considered but with this important caveat: If all you hear is a glowing reference with limited insight and no serious reservations, you should discount the information entirely.This extends to references that, in “criticism,” note that a candidate “works too hard and doesn’t know when to stop” or “doesn’t like to take credit for the many significant achievements they have accomplished.” Ignore them all when considering your decision. However, if you hear significant negativity that is supported by evidence, then you should take it seriously and give it additional weight. Unfortunately, in our experience, this happens all too rarely. Accept that most of the work that goes into reference checks will be a waste of time.

 

One final point about references and the role of search firms: Many search firms offer to check references on your behalf and do the work that companies might prefer to avoid. Never outsource this work; always do it yourself. In our view, there is no substitute for hearing evidence firsthand rather than having it filtered by a third party strongly sponsoring a candidate under consideration. Addressing the significant errors discussed above requires every organization to evaluate its own selection process. Honestly examining the current state of your hiring is difficult. In a large company, there may be many disparate approaches to selection. A smaller organization may improvise its approach with each opportunity. If your company is like most, your current selection process consists of mostly subjective measures, and it doesn’t track its validity, reliability, or effectiveness. This leads to hiring decisions whose results are no more predictable than chance.

 

Any company can make improvements to its selection process and improve its predictive capability. At the heart of an effective, predictive selection process is a validated assessment that helps identify the individuals who are best suited to the role for which they’re being considered.