Author: Zach Winn | MIT News<\/p>\n
In 2021, Texas suffered the worst energy infrastructure failure in state history, following three severe winter storms in February. At least 246 people were killed as a result of the disruptions, which led to food, water, heat and electricity shortages.<\/p>\n
If energy officials had had more time to prepare for the storms, for example by bringing in more trucks of natural gas to disrupted areas, damage from the event could have been mitigated. Unfortunately, weather forecasts beyond two weeks are notoriously unreliable, making for a tighter window for energy operators, farmers, and emergency response officials around the globe.<\/p>\n
A new technique by a team including researchers at MIT significantly improves weather forecasting beyond two weeks. The researchers developed a machine-learning model that learns to correct errors in existing forecasting models to improve their output over the two-to-six-week time frame. When applied to a gold-standard forecasting model, it doubled forecasting accuracy over that time frame, placing first in a prominent international forecasting competition.<\/p>\n
In a new paper in Nature<\/em><\/a>, the researchers describe their approach and show how it enables forecasts that for the first time beat climatologic averages between two and six weeks.<\/p>\n \u201cNot only does this do better on time averaged forecasts for this range, which is already very useful for farmers and members of the energy industry trying to match supply with demand, but it also improves predictions of extreme weather events, whether it\u2019s flooding or cold spells or something else,\u201d co-author and MIT Research Scientist Judah Cohen says. \u201cIt\u2019s really important to predict both, but extreme events might be the most impactful part of this for society.\u201d\u00a0<\/p>\n Joining Cohen on the paper are lead authors Hannah Guan, a Harvard College undergraduate; Soukayna Mouatadid, a University of Toronto postdoc; Genevieve Flaspohler SM \u201918, PhD \u201922; Paulo Orenstein, an associate professor at Brazil\u2019s National Institute for Pure and Applied Mathematics; Jeremy Berman of the data company DTN; Jakob Schloer and Joshua Talib, researchers with the European Center for Medium-Range Weather Forecasts; and Alex Lu, Haiyu Dong, Jonathan Weyn, Zekun Ni, and Lester Mackey, all researchers at Microsoft.<\/p>\n Longer forecasts<\/strong><\/p>\n In the 1970s, longtime MIT Professor Edward Lorenz came up with the idea of chaos theory to describe how tiny changes in initial weather conditions lead to completely different outcomes.<\/p>\n \u201cHe came up with this theoretical two-week limit where beyond that, we just can\u2019t predict weather because the noise in the system grows so much that it overwhelms any signal you start with,\u201d Cohen says. \u201cThat had a strong influence on the field that may have held people back from exploring forecasting beyond the two-week range. I never really believed that completely. In theory the atmosphere can create an infinite number of patterns, but it likes to fall into these favorite patterns over and over again. Once one thing happens, I think you can predict the domino that will fall after that, and the domino that will fall after that.\u201d<\/p>\n Today, the best weather forecasts come from either dynamical models, which numerically simulate how the atmosphere and oceans evolve over time, or artificial intelligence models, which learn to make predictions from historical weather data. AI models have shown improvements in predicting weather at short ranges, but they\u2019re even worse than dynamical models at longer time ranges.<\/p>\n Cohen, who is based in the MIT Department of Civil and Environmental Engineering, focuses on forecasting weather two to six weeks in advance, which is known as subseasonal forecasting. \u201cThat range is referred to as the \u2018predictability desert,\u2019 because in the short range you can capitalize on the initial conditions in the climate system and the atmosphere, and on the longer term you have these slow, varying boundary forces, like the super El Ni\u00f1o everyone\u2019s focused on this winter,\u201d Cohen says. \u201cThe subseasonal range can\u2019t take advantage of either of those. In most cases forecasting in the subseasonal ranges is no different than going to a casino and rolling the dice.\u201d<\/p>\n Cohen credits his co-author Lester Mackey, whom he has been collaborating with for more than a decade, with the initial idea of using machine learning to reduce errors in forecasting models.<\/p>\n \u201cWe built this wrapper \u2014 Lester calls it postprocessing,\u201d Cohen says. \u201cWe know AI models have a lot of errors, so we thought maybe we can quantify those errors and remove them.\u201d<\/p>\n While other forecasting systems do more simplistic error correction, the research team\u2019s postprocessing layer uses AI to learn from a forecasting system\u2019s mistakes based on historic data and shifts its forecasts. In that time range, forecasters use so-called probabilistic forecasts, where the likelihood of a range of possibilities if given.<\/p>\n \u201cAs more forecasts and data become available, our model adaptively improves its correction,\u201d Cohen says.<\/p>\n To build the model, the researchers used MIT Engaging, a supercomputing cluster hosted by MIT\u2019s Office of Research Computing and Data, but the resulting model does not require a supercomputer to run.<\/p>\n To put their model to the test, last year the research team submitted three models that included their post-processing layer to the European Center for Medium-Range Weather Forecasts (ECMWF), which is considered the best forecasting center in the world. Each season ECMWF holds forecasting competitions between models.<\/p>\n The submission that won first place combined ECMWF\u2019s forecasting model with another, and then applied the postprocessing correction on top. That approach ended up doubling the original model\u2019s predictive power in the subseasonal range. For the first time, the model offered an improvement over relying on climatologic averages.<\/p>\n \u201cUp until now, no one had\u00a0been able to show a model predicts weather better than long term averages for these subseasonal forecasts,\u201d Cohen says. \u201cThey\u2019ve pretty much all been worse than that \u2014 you could literally flip a coin and do better. People have questioned if it\u2019s even worth investing in models at all for this time range. We consistently beat climatology averages with our models.\u201d<\/p>\n The model also showed a statistically significant improvement in the ability to predict extreme weather.<\/p>\n \u201cThat\u2019s one of the more important findings, because it\u2019s so difficult to accomplish,\u201d Cohen says. \u201cWhen the ECMWF developed their own, non-AI, error-correction to their model, it got worse at predicting extreme weather. I think that\u2019s because traditional error, or bias, correction is always trying to get to the mean, whereas extreme weather is on the fringe of probabilities.\u201d<\/p>\n Preparing for disaster<\/strong><\/p>\n In the Nature<\/em> paper, the team showed their technique could be used to improve land forecasts in most regions across the globe. They also applied their bias correction technique to other models and showed it improved those as well, putting even lesser models on par with the ECMWF\u2019s forecasts.<\/p>\n \u201cWhat\u2019s nice about our model is it\u2019s an add-on wrapper,\u201d Cohen explains. \u201cWe used it with the ECMWF\u2019s dynamical model because that was considered the best, but it can be used with any model. It even improves models that are already bias corrected using another AI algorithm. So it shows a robust improvement. And it\u2019s open source and low cost.\u201d<\/p>\n Cohen says the meteorological office for the Republic of Kenya has already reached out about implementing the model to improve forecasting of the annual monsoon season to improve agricultural yields this winter. He hopes other groups will also use the model.<\/p>\n \u201cThis could help you decide when to fertilize around rain, when there\u2019s a freeze coming to protect crops during winter harvests, or help you avoid droughts,\u201d Cohen says. \u201cIt could also be useful for energy and emergency management. I\u2019ve talked to energy officials who say, \u2018If I\u2019m told two days before that an ice storm\u2019s coming, I don\u2019t have enough time, but if I had two weeks, that could be enough to make adequate preparations.\u201d<\/p>\n The work was supported, in part, by the U.S. National Science Foundation, the Carlos Chagas Filho Foundation for Research Support, and the European Union.<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"