Luck versus Skill: Sensing the Secret of Success

Napoleon was once asked what characteristics he sought in his best generals, the Marshals of France; was it bravery, strength, intelligence...? 'No, no,' he interrupted, 'I want them to be lucky.'

Recent research has confirmed Napoleon's instinct - and experience. Chance appears to play an outsized role in determining success.

This should not come as much of a surprise, even to the 'skill and effort' advocates who may, after all, take some solace in the notion that you make your own luck.

A review of recent business successes and failures of various types suggests that the 'best' or most skilled/intelligent/determined does not always win, nor does the most seemingly hapless always lose. A web of connections, chance meetings, errant or optimal timing and simple doggedness may play a far greater role in which idea, product or person prevails. From Microsoft and Google to MF Global and MySpace, business history is littered with tales of the annointed who disappointed and the seemingly star-crossed who eventually emerged triumphant.

The lesson is not so much that one is 'the' answer and the other is not, but that the interplay is complex, frequently unpredictable and that usually, both are present in some form at some time. JL

Robert Frank reports in the New York Times:

THERE may be no topic that more reliably divides liberals and conservatives than the relationship between success and luck. Many conservatives celebrate market success as an almost inevitable consequence of talent and effort. Liberals, by contrast, like to remind us that even talented people who work hard sometimes fall on hard times through no fault of their own.
It’s easy to see why each side is wary of the other’s position. Conservatives, for example, understandably fret that encouraging people to view life as a lottery might encourage them just to sit back and hope for the best. Liberals, for their part, worry that encouraging people to claim an unrealistically large share of the credit for their own success might make them more reluctant to aid the less fortunate.

Both sets of concerns have important implications for public policy, so it would be good to know more about how important luck actually is. Unfortunately, it’s an inherently tough question to answer. But recent experiments suggest that chance events may influence market outcomes far more heavily than previously thought.

The sociologists Duncan J. Watts, Matthew Sagalnik and Peter Dodds carried out some of these experiments, which Mr. Watts described in his superb 2011 book, “Everything Is Obvious* (*Once You Know the Answer).” Their work focuses on online markets, but it has much broader implications. It suggests that although market success does depend on the quality of a product, the link is extremely variable and uncertain. Even the best contestant in a product category may fail, and even the worst one sometimes wins. And for an overwhelming majority of contestants in the intermediate-quality range, they found success to be largely a matter of chance.

The researchers invited subjects to a temporary, experimental Web site called Music Lab, which listed 48 recordings by little-known indie bands. In the control version of the experiment, subjects could download any of the songs free if they agreed to give a quality rating after listening.

The average of these ratings then served as an “objective” rating of each song’s quality in subsequent versions of the experiment. In the control group, subjects saw no information other than the names of the bands and the songs, so their individual ratings were completely independent of the reactions of other participants.

Those independent ratings were extremely variable. Some songs got mostly high marks or mostly low marks, but a substantially larger number received distinctly mixed reviews.

The researchers then ran eight other versions of the same experiment. In each, Music Lab displayed two new pieces of information: how many times each song had been downloaded by others, and the average rating it had received so far. Participants in these groups thus received easily digestible feedback on which songs that others in their group were listening to and how much they liked them.

This social feedback produced sharply higher inequality in song ratings and download frequencies. In each of the eight groups, the most popular songs were far more popular, and the least popular songs far less popular, than their counterparts in the control group.

There was also enormous variability in the popularity rankings across the eight groups, and in the fates of songs with a given objective rating. The song “Lockdown,” by the band 52 Metro, is a case in point. Ranked 26th out of 48 in the objective ratings, it finished at No. 1 in one of the eight groups, but at No. 40 in another.

The most striking finding was that if a few early listeners disliked a song, that usually spelled its doom. But if a few early listeners happened to like the same song, it often went on to succeed.

That a song’s fate depended so heavily on chance doesn’t mean that success was purely a matter of luck. After all, bands that didn’t work hard and lacked even a modicum of talent wouldn’t have managed to record songs good enough to have been included in the first place.

IN their experiments, the sociologists showed how feedback could be a vitally important random effect. And it can be seen in many other situations: it’s often hard to find information about the quality of a particular product, so we rely on the reactions of friends and acquaintances who’ve already tried it. Any random differences in the early feedback we receive tend to be amplified as we share our reactions with others. Early success — even if unearned — breeds further success, and early failure breeds further failure. The upshot is that the fate of products in general — but especially of those in the intermediate-quality range — often entails an enormous element of luck.

We always knew that it was good to be smart and hard-working, and that if you were born or raised with those qualities, you were incredibly lucky, just as you were lucky if you grew up in the United States rather than in Somalia. But the sociologists’ research helps us understand why many people who have those qualities never find much success in the marketplace. Chance elements in the information flows that promote that success are sometimes the most important random factors of all.

Of course, we should keep celebrating the talented, hard-working people who have succeeded in their businesses or careers. But the research provides an important moral lesson: that these people might also do well to remain more humbly mindful of their own good fortune.