Why Tech, Big Data and AI Haven't Yet Improved Scientific, Economic Predictions

The model to the right is less than three months old. It purported to predict the course of the Covid pandemic through early 2022. And then the omicron variant emerged. Whose existence it hadnt predicted. 

Despite ever more powerful technology, better and more voluminous data and enhanced analytical techniques like machine learning and AI, humanity's ability to predict really epochal events remains sub-optimal. The issue is not technology or, necessarily, the data, but the human proclivity to base predictions on recent experience - and, perhaps, equally human failures of imagination. JL

Greg Ip reports in the Wall Street Journal:

Advances in computing power, artificial intelligence and big data were supposed to make prediction a science. But technology can’t see around corners any better than humans. An algorithm can predict what song you want to hear or video you want to watch based on past habits but can’t predict what has never happened or happens so rarely the data provides no reliable pattern. Perceptions of risk are heavily influenced by the recent past and the news cycle. A world of existential uncertainty calls for humility and a willingness to revise one’s models in response to new information. The biggest threats are things we hadn’t imagined.

Every year the World Economic Forum asks business, political and thought leaders to rank the biggest risks in the coming year or two. At the start of 2020, infectious disease didn’t make the list. Covid-19 became the most disruptive pandemic in a century. At the start of this year, inflation didn’t make the list either. It is now the most vexing problem in the U.S. economy.

This isn’t to mock the World Economic Forum, merely to note how flawed our perceptions of risk are. They are heavily influenced by the recent past and the news cycle—extreme weather now routinely makes the WEF’s threat list. Yet the biggest threats are regularly things we hadn’t imagined.

Advances in computing power, artificial intelligence and big data were supposed to make prediction a science. But technology can’t see around corners any better than humans. An algorithm can predict what song you want to hear or video you want to watch based on past habits but can’t predict what has never happened or happens so rarely the data provides no reliable pattern.

These limitations have become all too apparent in the past two years. Early in the pandemic a widely followed epidemiological model at the University of Washington, working from behavioral patterns established in other countries, struggled to predict state level deaths with any precision. Just last week a model used by the Centers for Disease Control and Prevention estimated that the Omicron variant comprised 73% of new Covid-19 infections in the week ended Dec. 18.

---

On Tuesday it revised that down to 23%, which fell outside the 95% confidence interval of the original estimate.

Economic models haven’t fared much better. Last spring, predictions of inflation at year’s end from 68 economists surveyed by The Wall Street Journal ranged from 1.3% to 4.3%. Federal Reserve officials projected with 70% confidence inflation would lie between 1.5% and 3.3% now. It hit 6.8% in November, by the consumer-price index (5.7% using the Fed’s preferred index).

Economists’ models failed because they didn’t incorporate unprecedented changes to spending patterns and shortages, in particular of labor, caused by the pandemic. An economist would have had to predict how social distancing would push up demand for laptop computers, game consoles and associated semiconductor chips, thereby reducing the supply of chips for autos, and thus constricting the supply and raising the price of new cars.

A world of existential uncertainty calls for humility and a willingness to revise one’s mental models in response to new information. Unfortunately, much of the world seems to be sorting itself into polarized camps where mental models are fixed and new information is always filtered through pre-existing biases.

This is a global phenomenon, and one of its signs is the withering political center. Chile, under center-left and center-right governments, achieved one of Latin America’s highest per capita incomes and lowest rates of poverty, and steady reductions in inequality.

Chileans, though, seemingly dissatisfied with this performance, chose radically different approaches for addressing it. No centrist candidate made it past the first round of this year’s presidential election. In the runoff, voters had to choose between the extreme left and extreme right: Gabriel Boric, the eventual winner, a former student protest leader whose allies include communists, and José Antonio Kast, a defender of the military dictatorship that ruled Chile until 1990.

Americans, too, have faced binary choices. Presidents Donald Trump and Joe Biden both sought to remake the country according to the preferences of their political base rather than the country as a whole.

“The center is weaker than it once was,” said Bill Galston, co-founder of the New Center, a think tank devoted to centrist policies. “But politicians act as though it has vanished altogether, which it has not.”

Commitment to one agenda makes it hard to change course when circumstances require another. Mr. Biden built his Covid strategy around mass vaccination, and lacked a backup (such as mass testing) when many people declined the vaccine and a variant that evades some of the protection of vaccines, Omicron, emerged. He built his economic strategy around boosting demand and reducing inequality via generous aid to families and the unemployed, which is unsuited to the current challenge of inflation and shortages, in particular of labor.

Meanwhile, polarized voters are inclined to blame errors and policy shifts on politics or incompetence rather than the inherent difficulty of making decisions under uncertainty. Evidence on the optimal response to Covid has steadily evolved, from washing hands and disinfecting surfaces to lockdowns to targeted restrictions to mask and vaccine mandates, yet public attitudes increasingly fall along partisan lines. The same is true for inflation: Democrats think it will average a bit above 2% in the next five years, around its historic level, while Republicans think it will top 4%, according to the University of Michigan.

So here’s a prediction: One of these groups is going to be wrong. If you want to be prepared for the future, flexibility, not ideology, is your friend.