Model, Citizen? How Almost Everyone - Including the National Weather Service - Botched the NYC Blizzard Forecast

Snowzilla! Blizzardpocalypse! To listen to sundry elected officials and authority figures prior to the most recent impending weather emergency, you would not have been faulted for believing that The End was nigh and that you had best make peace with your Maker as well as your local liquor store.

And let's be clear: in certain parts of New England, this storm did set records - three feet of snow in Worcester, Mass - an all-time historic record. Some severe damage was done and a couple of unlucky souls died. But Worcester is not New York. And, neither, for that matter, is Boston. Which, frankly already had its hands full confronting a different kind of storm: national indignation over deflated footballs while its citizens nurtured their well known penchant for resentment over the fact that outsiders would actually dare to think their team would ever do anything so nefarious on purpose (again)!

But we digress. New York was promised/threatened with a storm of such magnitude as to equal The City's belief in its own surpassing importance. As the nation's - and arguably, the world's - media capital, nothing but a tempest of supernatural power would do. And so offices closed early, public services and transportation were shut down, all available man and woman power was mobilized. The world waited breathlessly...for... meh...

Nice weather for sledding. Good cross country skiing in Central Park. Pretty photos all over Facebook. And merchants did a land office business in the essentials: milk, organic beer, kale, gluten free bread, quinoa and yes, lingerie (seriously: sales up 500% prior to the storm. Look for evidence in nine months). But all that sturm und drang for...what.

Well, lots of fingers are being pointed, as usual. Modelers are defending their models. Authorities are intoning the benefits of prevention. The media want to know why there wasn't perfect information. And politicians who are quick to blame anyone when we aren't forewarned, are now assigning blame for being overwarned.

But really, this is a good lesson for all of us in the big data era. Because it reinforces the message that so few want to hear: about imperfection. In computers, and algorithms, and data and in our ability to interpret what we see, especially when, as so often happens, the data are imprecise or contradictory or confusing.

We may have thought that all this information would totally remove uncertainty forever. But guess what? Not happening. And this outcome, however frustrating, is called life. JL

Eric Holthaus reports in Slate:

Meteorologists rely heavily on computer weather models for everything from temperature forecasts to the tracks of hurricanes to snowstorms. They’re pretty good. But they frequently disagree—and when that happens, you need to quickly assess what information to use and what to toss. Which is where the humans come in.

In the run-up to this week’s blizzard, some serious differences emerged when it came to the New York City snowfall forecast.

On the one hand, there was the National Weather Service, armed with thousands of meteorologists, a newly upgraded forecasting supercomputer, a nationwide network of weather radars and balloons, and satellite technology.* It even sent the Hurricane Hunters to fly through the storm to take additional data.

Advertisement

Early on, the NWS called for “historic” snowfall totals of 20 to 30 inches in New York City. It cautioned that if an intense snowfall band ended up camping out over the city (as several model forecasts suggested would happen), 3 feet wasn’t out of the question.

That obviously didn’t happen. One NWS forecaster in Philadelphia went so far as to issue a public apology via Twitter for the botched forecast. The director of the National Weather Service also scheduled a Tuesday afternoon press conference to discuss the forecast, a rare step likely aimed at restoring public trust. (The NWS forecast for Boston, by the way, is right on track. That city’s still on pace to record its biggest snowstorm in history.)

So what went wrong in New York City? In a series of Facebook posts explaining its forecast, the NWS said, “These bands are nearly impossible to predict until they develop.” True, but someone did manage to forecast it: the Weather Channel.

Throughout most the day on Monday, the Weather Channel was forecasting 12 to 18 inches for New York City, while the National Weather Service insisted a record-breaker was possible. By nightfall the Weather Channel had scaled back its forecast even further, calling for 8 to 12 inches. And that’s exactly what fell. As late as 5 p.m. Monday, the National Weather Service was still talking about top-end scenarios of up to 3 feet in the Bronx.

These days, meteorologists rely heavily on computer weather models for everything from temperature forecasts to the tracks of hurricanes to snowstorms. Usually, they’re pretty good. But the problem is, they frequently disagree—and when that happens, you need to quickly assess what information to use and what to toss. Which is where the humans come in.

As best I can piece together, the Weather Channel’s method for forecasting storms like this is not to throw out any model information, no matter how off-base it may seem at the time. And for this storm, the potential spread of model forecast placement of the most intense snow band was exceptionally large for the New York City area. This is a perfect situation where probabilistic forecasts are useful. Instead of banking on one or two specific models like the NWS did (and which turned out to be the wrong ones), the Weather Channel chose to blend the models and weight them a little more equally.

In contrast, the National Weather Service took the gutsy step of disregarding the GFS model—its own, newly improved model, by the way—and opted almost entirely for a blend of the ECMWF and NAM, two of the historically best-performing models for this sort of storm and lead time. Though that’s a more traditional style of forecasting, it’s prone to busts. Sure enough, the storm’s center ended up tracking about 120 miles farther east and about three hours faster than the ECMWF forecasted, closer to where the GFS predicted it would be, a pretty big difference.

For my forecast updates here on Slate, I sided almost exclusively with the numbers from the National Weather Service because 1) they seemed reasonable, looking at much of the same data myself that they used to construct this forecast, and 2) they’re the official forecast source. Entire governments make plans based on their forecasts, so I figured that’s good enough for Slate. I was wrong.

On Tuesday, I spoke with the Weather Channel’s winter weather expert Tom Niziol by phone in an attempt to understand how he and his colleagues got it right—and, by extension, what the National Weather Service may have missed.

“As we saw the guidance changing throughout the day, we adjusted that forecast as necessary,” Niziol told me. “I don’t want to make it sound too simple.”

This forecast was truly complicated. The Weather Channel’s forecast for a 30-mile stretch including New York City ranged from more than 18 inches on Long Island to less than 8 inches in New Jersey. It really could have gone either way. In contrast, the Boston forecast, Niziol said, was a “no-brainer.”

ADVERTISEMENT

In recent years, there’ve been several weather-related controversies in New York state, suggesting that a storm like this was bound to become political. The twin hurricane threats of Irene in 2011 and Sandy in 2012 produced vastly different effects in New York City, despite the National Weather Service calling for a significant coastal flood in each case. Irene turned out to be nearly as catastrophic in upstate New York as Sandy was in the city, but downstate residents likely remember Irene as a busted forecast. Then New York City Mayor Michael Bloomberg took heat during the run-up to Sandy for initially downplaying its flooding threat, probably because of Irene, at least in part.

More recently, New York Gov. Andrew Cuomo was criticized for blaming the National Weather Service for a sluggish response to the Buffalo mega-lake-effect storm, which brought more than 6 feet of snow to the region in just three days.

National Weather Service forecasters (and the governor) likely had these experiences in mind this week. The NWS may have overstressed the blizzard’s worst-case scenario, and the governor may have acted too hastily, both in an effort to cover themselves in case it actually came to pass.

Niziol, who used to head up the Buffalo office of the National Weather Service, knows a lot about messaging during major weather events. He told me that emergency managers frequently want the most likely forecast and the worst-case scenario as well. In the aftermath of this storm, local leaders used that philosophy to explain the unnecessary shuttering of NYC schools and public transportation.