Is Something Wrong With Online Prediction Markets?

It's not just the quality of the data, it's how humans are conditioned to react to it and then use it. JL

Andrew Gelman and David Rothschild report in Slate:

Belief in the correctness of prediction markets causes them to be too stable. When new information comes in, it gets discounted. People are using the current prediction odds as an anchor and updating too slowly. Long term, this should right itself; as investors become aware of this bias. But right now, it appears that prediction markets have arrived at a paradoxical place: Their reliability, the very source of their prestige, is causing them to fail.

Prediction markets took a lot of heat for their prediction on Brexit (holding steady with a predicted probability 25 percent during the week leading up to the vote while polls did not flip from “Leave” to “Remain” until the final day). Probabilistic predictions can and will “fail, ” but it is good to understand why this is happening. Prediction markets have a strong track record and people trust them. And that actually may be the problem right now.

Prediction markets are real-money exchanges that allow people to buy and sell futures contracts on upcoming events. They have a long and colorful history; $10 million was wagered on the 1916 election (about $220 million in 2016 dollars), or $12 per vote actually cast. Prediction markets lay dormant in the United States for a few decades before coming back with the Iowa Electronic Market (a small academic exchange), gaining fame with Intrade (an Irish-based exchange that lured U.S. traders before the Commodity Futures Trading Commission shut it down), and now carried forward by Betfair (a massive publicly traded company) and PredictIt (a legal U.S. exchange).

Canonically, a trader can buy a contract on an outcome, such as the Democratic nominee to win the 2016 presidential election, and it will be worth $1 if the outcome occurs and $0 if the outcome does not occur. The price at which people are willing to buy and sell that contract can be interpreted as the probability of the outcome occurring, or at least the collective subjective probability implicitly assigned by the crowd of people who trade in these markets.

Prediction markets work because traders can see all of the standard forecast data and aggregate it along with other idiosyncratic or dispersed information about the election. Traders can aggregate polling data or follow published poll aggregates, and they can add in any other information they think is relevant.

As researchers in political science and as economists, we see three enticing advantages of forecasts from prediction markets, compared with poll-based forecasts of election outcomes: 1) Markets have full coverage of outcomes. They covered all primaries and caucuses in 2016, when polls covered about 50 percent of possible contests. Similarly, for Brexit, there are forecasts not just on whether the United Kingdom would vote to leave the European Unions, which could be generated from polls, but also now on whether they will actually invoke Article 50 and leave. 2) They are dependably accurate (e.g., the 2016 primary elections). 3) They update fast, more or less in real time.

But more recently, prediction markets have developed an odd sort of problem. There seems to be a feedback mechanism now whereby the betting-market odds reify themselves.

Belief in the correctness of prediction markets causes them to be too stable.

What do we mean by that? In the case of Brexit, it goes like this: Different surveys give different results, and we all know not to trust the polls, which have notoriously failed in various British elections such as the 2014 Scottish Referendum and the 2015 British parliamentary elections. But we do watch the prediction markets, which all sorts of experts have assured us capture the wisdom of crowds.

So, serious people who care about the election watch the prediction markets. The markets say 25 percent for “Leave.” Then there’s other information, the latest poll, and so forth. How to think about this information? Informed people look to the markets. What do the markets say? Twenty-five percent. OK, then that is the probability.

This is not an airtight argument or a closed loop. Of course, real information does intrude upon this picture. But something is amiss when prediction markets stay stable for too long.

In the past, traders followed the polls too closely and sent the prediction markets up and down. But now the opposite is happening. Traders are treating market odds as correct probabilities and not updating enough based on outside information. Belief in the correctness of prediction markets causes them to be too stable. Take a look at the prediction market–based probabilities from the past three elections. During the past few weeks the prediction markets have been extremely stable relative to the exact same time period in previous cycles, despite this year’s politics being extraordinarily eventful.

Andrew Gelman and David Rothschild

What we are seeing now with the Clinton-Trump race is similar to what we saw with Brexit. Initial odds are reasonable, based on whatever information people have. But then when new information comes in, it gets discounted. People are using the current prediction odds as an anchor and updating too slowly.

Related to this is the concern that, at least with Brexit, pollsters and pundits were also to some extent anchoring themselves off the prediction odds. When polls showed solid and consistent majorities for “Leave,” pollsters looked to the markets, which were firmly supporting “Remain,” and doubted their own polls.

The implication for bettors is that when the odds seem too stable, trade on the news. Don’t assume that the markets have already incorporated all openly available information. Right now the prediction market prices for the presidential election have been incredibly stable for nearly a month, despite waves of news on business scandals, fundraising money, poll swings, and possible indictments. A mature prediction market should show measured, but real, swings in prices in response to news.

Long term, or even medium term, this should right itself; as investors become aware of this bias (in part because of this article!), it should diminish or disappear. But right now, it appears that prediction markets have arrived at a paradoxical place: Their reliability, the very source of their prestige, is causing them to fail.