The Myth That 'Markets Get Prices Right' Just Won't Die

The reality is that data-driven, high speed, algorithmic market performance is based on the arbitrage opportunity offered less by efficiency than by information asymmetries. JL


Noah Smith reports in Bloomberg:

For economists, it’s an article of faith that changes in the prices of stocks come from changes in investors’ risk preferences, rather than from swings of irrational sentiment or the market’s failure to take important information into account. If an anomaly survives scrutiny, and persists, EMH’s adherents will try to classify it as a “risk factor,” claiming extra return compensates for risk that can’t be diversified away. (But) the market is a complex ecosystem of imperfections, offering investors opportunities to make money, with mispricings shifting and disappearing as investors trade them away.
At this point, the efficient markets hypothesis makes for an easy rhetorical target. Precious few economists and financial professionals are willing to stand up in public anymore and assert that financial markets produce the best estimate of fundamental asset values. A series of enormous bubbles, plus decades of outsized profits for hedge funds like Renaissance Technologies, has turned the EMH into a bit of a joke.
Meanwhile, finding ways to beat the market has become a parlor game in the world of financial economics. Hundreds of papers have been published claiming to have found new ways to predict returns. Defenders of the EMH counter that many of these so-called “anomalies” are probably false alarms, much of it due to what's known as publication bias. As economists Campbell Harvey, Yan Liu and Caroline Zhu pointed out in a famous 2013 paper, the statistical hurdles that researchers demand that anomalies pass are probably not high enough. That leads to a sort of “monkeys on typewriters” effect -- if enough finance professors sift through the data for long enough, they’ll find a bunch of apparent EMH violations that are exciting enough to get published, but which evaporate upon further scrutiny. That concern seems to be supported by the fact that most anomalies disappear over time.
This finding allows efficient market theory’s defenders to raise the suspicion that most of the violations of their theory are fake. But this proves to be only a small respite. First of all, just because anomalies are fleeting doesn’t mean they weren’t real to begin with. A recent paper by David McLean and Jeffrey Pontiff found that anomalies often hold up even after they’re discovered, but then disappear only after finance professors publish papers about them. That implies that market flaws are real, but traders learn about them by reading academic papers, then trade against the mispricings until they fade. A follow-up paper by Heiko Jacobs and Sebastian Muller found that in most countries, markets are even less efficient than in the U.S. and the anomalies discovered by finance professors persist for a much longer time. 
Now Andrew Chen and Tom Zimmermann have a paper claiming that publication bias is much more subdued than previously believed. They reason that if an anomaly comes from data mining, it will probably just barely be statistically significant. But many anomalies show up with extremely strong signals in the data -- much stronger than would be required to claim significance. That implies that many are real, and can be used to beat the market -- at least, until enough traders catch on that the mispricing gets traded away.
Efficient market theory’s defenders have a second tactic to deal with challenges. If an anomaly survives scrutiny and replication, and persists for a long time, the EMH’s remaining adherents will generally try to classify it as a “risk factor,” claiming that the extra return compensates investors for taking extra risk that can’t be diversified away. For some economists, it’s almost an article of faith that changes in the prices of stocks come from changes in investors’ risk preferences, rather than from swings of irrational sentiment or the market’s failure to take important information into account. Eugene Fama, who won the Nobel Prize for his formulation of the EMH, has by now expanded his preferred set of “risk factors” to five, while others identify many more.
A new paper by business school professors Lars Lochstoer and Paul Tetlock investigates whether risk-based stories hold up. The researchers develop a new statistical procedure for separating changes in a company’s expected cash flows from changes in investors’ risk preferences, combining information about differences between companies at a given point in time with information about changes in stock values over long periods of time. They find that most of the excess returns that investors can get from factor investing -- or, if you prefer, from trading against anomalies -- come not from changes in risk preferences, but from changes in the long-term path of companies’ cash flows. Furthermore, they find that when overall risk tolerance in the market goes up, tolerance to the specific risks represented by the popular risk factors tends to go down, a strange phenomenon for which there’s no explanation as yet.

Lochstoer and Tetlock’s result implies that anomalies probably don’t come from variations in investors’ preferences. That doesn’t prove that risk isn’t the culprit -- a company that suffers some change that causes it to make less money over the long term also probably becomes riskier. But the authors’ evidence is also consistent with another story -- the idea that when a stock does badly, investors irrationally expect it to keep doing badly.
In other words, efficient markets theory isn’t dead yet. There are still
loopholes its defenders can thread, that allow them to claim that nobody really beats the market, and that all excess returns are just compensation for taking on extra risk. But as more anomalies persist, and more risk-based theories are found wanting, the window for the EMH grows smaller and smaller.

Eventually, it seems likely that a new consensus will replace the EMH -- that the market is a complex ecosystem of hundreds or thousands of imperfections, offering clever investors plenty of opportunities to make money in the short term, but that these mispricings are constantly shifting, appearing and disappearing as investors chase them and trade them away.