Why Mathematical Models Can Never Get Things Quite Right

The past is almost never prologue, statistically or realistically. JL

Barry Ritholtz reports in The Big Picture:

The number of infections and deaths we expect is a guess. Epidemiologists have modeled various probabilities, with a range of possible outcomes. We don't live in objective reality; we function in a model of our own construction. (But) we fail to recognize that models are “not a report sent back from the future. Models are constructed of “stuff we know, stuff we think we know, and stuff we have no idea about." The universe is more complicated than any our models suggest. Every model is flawed because the underlying premise is that the future will look like the past. "Stuff we have no idea about” destroys that premise.

The economy is frozen because of the coronavirus. In all likelihood, it can't fully reopen until we have proper testing and tracing, an effective vaccine or treatment, or population-wide immunity, none of which may ever happen. The total number of infections and deaths we expect this year and next is still a guess. Epidemiologists have modeled various probabilities, with a wide range of possible outcomes.

Yet, we have placed a great deal of faith in those models and many others because so much of our lives is guided by them. We don't live in objective reality; in truth, we function in a model of our own construction. Our brains generate mental outlines, continually filling in missing information to form a picture that we can discern and identify. It is an evolutionary trait that has allowed us to thrive in a world where for a very long time we were as much prey as hunter.

So why is that a problem? Because we fail to recognize that models are “not a report sent back from the future," according to journalist Jonathan V. Last. Models are constructed of “stuff we know, stuff we think we know, and stuff we have no idea about." There's a lot of stuff we think we know about Covid-19 and probably even more that we don't, which is why the coronavirus models have given such a wide range of possible outcomes in terms of infection mortality.

The standard caveat from statistician George E. P. Box was that “all models are wrong, but some are useful.” Mathematical models can help us make sense of the world, assuming our assumptions are valid and we don't feed bad data into them. Box reminds us that mathematics creates a shadow of reality, and that the universe is much more complicated than any of our models suggest.

More to the point, every model is flawed because the underlying premise of all of them is that the future will look like the past. Nothing throws a model off more than when “stuff we have no idea about” destroys that fundamental premise.