And none more so than those for athletic talent. That is why complex physical and psychological tests, scouting reports, performance try-out combines and university scholarships have been devised to attempt to sift the truly extraordinary from the merely exceptional.
The money is so huge that contracts in the hundreds of millions for soccer, football, baseball and basketball stars are now unremarkable. So before committing the big bucks, team owners want some assurances. And an entire industry has sprung up to provide them with answers. What could be more perfect: market demand being met by quantitatively creative suppliers whose metrics assist in supporting the meritocracy of sporting competition.
Except that analysis and experience demonstrate that the outcome of all this research remains little better than random selection.
Exhibit A is professional basketball phenomenon Jeremy Lin. The child of Taiwanese immigrants to America who couldnt get a scholarship from a big state university basketball factory. So he went to Harvard, which hasnt produced a professional basketball player in 50 years. Where he majored in Economics as opposed to typical jock programs like Sports and Society, or Health and Fitness Management. And so, after bouncing around the league as a substitute's substitute, the only Asian currently playing (his predecessor having been the 7 foot tall Yao Ming), he is suddenly leading one of the National Basketball Associations marquee teams to an eight game win streak - and has captured the sports world's imagination.
Which is simply the exclamation point at the end of a longer story. That all of our efforts to 'scientifically' identify talent rarely succeed. Not that we shouldnt keep trying. And maybe someday society will figure it out. But in the interim, beware of statisticians bearing 'solutions.' JL
Jonah Lehrer reports in Wired:
Linsanity! For those who don’t follow the NBA, or read the NY Post, or check Twitter during Knicks games, Jeremy Lin is a new point guard who has been churning out a series of incredible performances. Last week, he torched the Lakers for 38 points; two nights ago, he hit a three pointer at the buzzer to beat the Raptors (and last night he beat the Kings, making his streak eight wins in a row). What makes Lin’s story even more remarkable is that he has been repeatedly spurned by basketball professionals. Although Lin dreamt of attending UCLA or Stanford, neither college offered him a scholarship. He ended up at Harvard.
Although Lin excelled in the Ivy League and broke numerous basketball records, he wasn’t selected in the 2010 NBA draft.
Linsanity! For those who don’t follow the NBA, or read the NY Post, or check Twitter during Knicks games, Jeremy Lin is a new point guard who has been churning out a series of incredible performances. Last week, he torched the Lakers for 38 points; last night, he hit a three pointer at the buzzer to beat the Raptors. What makes Lin’s story even more remarkable is that he has been repeatedly spurned by basketball professionals. Although Lin dreamt of attending UCLA or Stanford, neither college offered him a scholarship. He ended up at Harvard.
Although Lin excelled in the Ivy League and broke numerous basketball records, he wasn’t selected in the 2010 NBA draft.
Eight teams had invited Lin to predraft workouts. Diepenbrock said that NBA tryouts do not play five on five. Lin acknowledged that the workouts were “one on one or two on two or three on three, and that’s not where I excel. I’ve never played basketball like that.”
Lin was eventually signed by the Warriors, but he was soon placed on the inactive list and repeatedly sent down to the D-League affiliate. Last December, Lin was waived by the Warriors, picked up and then waived by the Rockets, before finally being claimed by the Knicks as a backup point guard.
New York considered releasing Lin before his contract became guaranteed on February 10 so they could sign a new player. However, after the Knicks squandered a fourth quarter lead in a February 3 loss to the Boston Celtics, coach Mike D’Antoni decided to give Lin a chance to play. “He got lucky because we were playing so bad,” said D’Antoni.[94] Lin had played only 55 minutes through the Knicks’ first 23 games.
By now, you probably see the theme: nobody thought Lin could make it in the NBA. He was too short and too weak, with a mediocre jump shot. And that’s why, if I were an NBA coach or scout or GM, his remarkable success would keep me up at night. Professional sports, after all, are supposed to be a pure meritocracy, in which those with the most talent are carefully vetted and tested. Those who make it in the NBA are supposed to be a pure distillate of athletic potential: the players are richly rewarded because they really are the very best.
But how effective is this meritocracy? Are there lots of Jeremy Lins out there? How many benchwarmers could hit game winners? Unfortunately for professional sports, the evidence suggests that there’s plenty of room for improvement: teams are terrible at identifying talent. Take the NFL. Although the stakes for these sports personnel decisions are huge – a failed draft pick will not only waste millions of dollars in salary but will also come with a high opportunity cost – teams remain tethered to useless metrics and measurements. My favorite example is the NFL scouting combine, which is like a big job fair for prospective players. (The combine includes everything from the 40 yard dash to a battery of psychological tests.) In recent years, the combine has become a major press event, and teams regularly cite combine results when justifying draft picks. But this is a mistake, as the combine turns out to be a big waste of time. Here’s the abstract of a recent analysis performed by the economists Frank Kuzmits and Arthur Adams:
Combine measures examined in this study include 10-, 20-, and 40-yard dashes, bench press, vertical jump, broad jump, 20- and 60-yard shuttles, three-cone drill, and the Wonderlic Personnel Test. Performance criteria include 10 variables: draft order; 3 years each of salary received and games played; and position-specific data. Using correlation analysis, we find no consistent statistical relationship between combine tests and professional football performance, with the notable exception of sprint tests for running backs. From a practical standpoint, the results of the study should encourage NFL team personnel to reevaluate the usefulness of the combine’s physical tests and exercises as predictors of player performance. This study should encourage team personnel to consider the weighting and importance of various combine measures and the potential benefits of overhauling the combine process, with the goal of creating a more valid system for predicting player success.
But wait: it gets worse. Not only have NFL teams failed to find relevant variables for predicting future player performance, but they typically pretend otherwise. Instead of embracing their uncertainty – human talent remains mysterious, even with reams of college statistics – most NFL teams default into overconfidence, trading up for draft picks and bestowing massive contracts on these new recruits.
A few years ago, Cade Massey and Richard Thaler came out with a paper that looked at the “return on value” from these early draft picks. In essence, they constructed a model in which all the players at a given position – quarterback, running back, linebacker, etc. – were ranked according to the order in which they were picked in the draft. Then, they compared any two of these players in consecutive order, so that a tight-end taken early in the first round might be pitted against a tight-end taken late in the third. The comparisons were based on a number of performance metrics, such as number of games started and yards per catch. As Thaler notes, if teams knew nothing, the player that went higher in the draft would outperform the lower ranked player 50 percent of the time. In other words, drafting talent would be roughly equivalent to a coin flip; all the scouting would be perfectly useless. In contrast, if teams knew what they were doing – if they could effectively identify the best college players – then the higher draft picks should outperform their competition close to 100 percent of the time.
So what did Thaler find? Flipping a coin is the apt metaphor, as the higher picks proved better only 52 percent of the time. The teams beat randomness, but barely.
Jeremy Lin is a reminder that similar problems almost certainly apply to the NBA, which is why we shouldn’t be so surprised that a benchwarmer cut from multiple teams is quickly becoming a star. There is talent everywhere. We just don’t know how to find it.
0 comments:
Post a Comment