MIT 团队用机器学习后处理将两到六周天气预报准确率提高一倍

2021年，在2月连续三场严重冬季风暴之后，得克萨斯州遭遇了该州历史上最严重的能源基础设施故障。这场中断导致食品、水、取暖和电力短缺，至少造成246人死亡。

In 2021, Texas suffered the worst energy infrastructure failure in state history, following three severe winter storms in February. At least 246 people were killed as a result of the disruptions, which led to food, water, heat and electricity shortages.

如果能源官员有更多时间准备应对风暴，例如向受灾地区调运更多天然气卡车，这场事件造成的损失本可以减轻。不幸的是，两周以上的天气预报众所周知地不可靠，这使得全球的能源运营商、农民和应急响应官员可用的窗口更加紧张。

If energy officials had had more time to prepare for the storms, for example by bringing in more trucks of natural gas to disrupted areas, damage from the event could have been mitigated. Unfortunately, weather forecasts beyond two weeks are notoriously unreliable, making for a tighter window for energy operators, farmers, and emergency response officials around the globe.

包括麻省理工学院研究人员在内的一个团队开发的一项新技术，显著改善了超过两周的天气预报。研究人员开发了一种机器学习模型，它学会纠正现有预报模型中的误差，从而改善两到六周时间范围内的输出。当应用于一个黄金标准预报模型时，它在该时间范围内将预报准确率提高了一倍，并在一个著名的国际预报竞赛中排名第一。

A new technique by a team including researchers at MIT significantly improves weather forecasting beyond two weeks. The researchers developed a machine-learning model that learns to correct errors in existing forecasting models to improve their output over the two-to-six-week time frame. When applied to a gold-standard forecasting model, it doubled forecasting accuracy over that time frame, placing first in a prominent international forecasting competition.

在Nature的一篇新论文中，研究人员描述了他们的方法，并展示了它如何使预报首次在两到六周之间超过气候学平均值。

In a new paper in Nature, the researchers describe their approach and show how it enables forecasts that for the first time beat climatologic averages between two and six weeks.

“这不仅在这个范围的时间平均预报上表现更好——这对试图匹配供需的农民和能源行业成员已经非常有用——而且还改善了对极端天气事件的预测，无论是洪水、寒潮还是其他什么，”共同作者、麻省理工学院研究科学家Judah Cohen说。“预测两者都非常重要，但极端事件对社会来说可能是其中影响最大的部分。” 

“Not only does this do better on time averaged forecasts for this range, which is already very useful for farmers and members of the energy industry trying to match supply with demand, but it also improves predictions of extreme weather events, whether it’s flooding or cold spells or something else,” co-author and MIT Research Scientist Judah Cohen says. “It’s really important to predict both, but extreme events might be the most impactful part of this for society.” 

与Cohen共同撰写该论文的还有第一作者、哈佛学院本科生Hannah Guan；多伦多大学博士后Soukayna Mouatadid；Genevieve Flaspohler SM ’18、PhD ’22；巴西国家纯数学与应用数学研究所副教授Paulo Orenstein；数据公司DTN的Jeremy Berman；欧洲中期天气预报中心的研究人员Jakob Schloer和Joshua Talib；以及微软的研究人员Alex Lu、Haiyu Dong、Jonathan Weyn、Zekun Ni和Lester Mackey。

更长期的预报

Joining Cohen on the paper are lead authors Hannah Guan, a Harvard College undergraduate; Soukayna Mouatadid, a University of Toronto postdoc; Genevieve Flaspohler SM ’18, PhD ’22; Paulo Orenstein, an associate professor at Brazil’s National Institute for Pure and Applied Mathematics; Jeremy Berman of the data company DTN; Jakob Schloer and Joshua Talib, researchers with the European Center for Medium-Range Weather Forecasts; and Alex Lu, Haiyu Dong, Jonathan Weyn, Zekun Ni, and Lester Mackey, all researchers at Microsoft.

Longer forecasts

20世纪70年代，长期任职于麻省理工学院的教授Edward Lorenz提出了混沌理论的概念，用以描述初始天气条件的微小变化如何导致完全不同的结果。

In the 1970s, longtime MIT Professor Edward Lorenz came up with the idea of chaos theory to describe how tiny changes in initial weather conditions lead to completely different outcomes.

“他提出了这个理论上的两周限制，超过这个限制，我们就无法预测天气，因为系统中的噪声增长得如此之大，以至于压倒了你开始时的任何信号，”Cohen说。“这对该领域产生了强烈影响，可能阻碍了人们探索两周范围之外的预报。我从未真正完全相信这一点。理论上，大气可以创造出无限多的模式，但它喜欢一遍又一遍地落入这些偏好的模式。一旦一件事发生，我认为你可以预测接下来会倒下的多米诺骨牌，以及那之后会倒下的多米诺骨牌。”

“He came up with this theoretical two-week limit where beyond that, we just can’t predict weather because the noise in the system grows so much that it overwhelms any signal you start with,” Cohen says. “That had a strong influence on the field that may have held people back from exploring forecasting beyond the two-week range. I never really believed that completely. In theory the atmosphere can create an infinite number of patterns, but it likes to fall into these favorite patterns over and over again. Once one thing happens, I think you can predict the domino that will fall after that, and the domino that will fall after that.”

如今，最好的天气预报要么来自动力模式——它通过数值方法模拟大气和海洋随时间的演变，要么来自人工智能模型——它从历史天气数据中学习如何做出预测。AI 模型在短期天气预报上已表现出改进，但在更长的时间范围内，它们甚至比动力模式还要差。

Today, the best weather forecasts come from either dynamical models, which numerically simulate how the atmosphere and oceans evolve over time, or artificial intelligence models, which learn to make predictions from historical weather data. AI models have shown improvements in predicting weather at short ranges, but they’re even worse than dynamical models at longer time ranges.

科恩就职于麻省理工学院土木与环境工程系，专注于提前两到六周的天气预报，这被称为次季节预报。“那个范围被称为‘可预报性荒漠’，因为在短期，你可以利用气候系统和大气中的初始条件，而在较长期，你有这些缓慢变化的边界强迫，比如今年冬天大家都在关注的超级厄尔尼诺，”科恩说。“次季节范围无法利用这两者中的任何一个。在大多数情况下，次季节范围的预报无异于去赌场掷骰子。”

Cohen, who is based in the MIT Department of Civil and Environmental Engineering, focuses on forecasting weather two to six weeks in advance, which is known as subseasonal forecasting. “That range is referred to as the ‘predictability desert,’ because in the short range you can capitalize on the initial conditions in the climate system and the atmosphere, and on the longer term you have these slow, varying boundary forces, like the super El Niño everyone’s focused on this winter,” Cohen says. “The subseasonal range can’t take advantage of either of those. In most cases forecasting in the subseasonal ranges is no different than going to a casino and rolling the dice.”

科恩将利用机器学习减少预报模型误差的最初想法归功于他的合著者莱斯特·麦基，两人合作已超过十年。

Cohen credits his co-author Lester Mackey, whom he has been collaborating with for more than a decade, with the initial idea of using machine learning to reduce errors in forecasting models.

“我们构建了这个包装器——莱斯特称之为后处理，”科恩说。“我们知道 AI 模型有很多误差，所以我们想也许可以量化这些误差并消除它们。”

“We built this wrapper — Lester calls it postprocessing,” Cohen says. “We know AI models have a lot of errors, so we thought maybe we can quantify those errors and remove them.”

其他预报系统进行的是较为简单的误差校正，而研究团队的后处理层利用 AI 从历史数据中学习预报系统的错误，并据此调整其预报。在那个时间范围内，预报员使用所谓的概率预报，即给出各种可能性的概率。

While other forecasting systems do more simplistic error correction, the research team’s postprocessing layer uses AI to learn from a forecasting system’s mistakes based on historic data and shifts its forecasts. In that time range, forecasters use so-called probabilistic forecasts, where the likelihood of a range of possibilities if given.

“随着更多预报和数据的出现，我们的模型会自适应地改进其校正，”科恩说。

“As more forecasts and data become available, our model adaptively improves its correction,” Cohen says.

为构建该模型，研究人员使用了 MIT Engaging——由麻省理工学院研究计算与数据办公室托管的超级计算集群，但最终得到的模型运行起来并不需要超级计算机。

To build the model, the researchers used MIT Engaging, a supercomputing cluster hosted by MIT’s Office of Research Computing and Data, but the resulting model does not require a supercomputer to run.

为检验他们的模型，去年研究团队向欧洲中期天气预报中心（ECMWF）提交了三个包含其后处理层的模型，该中心被认为是世界上最好的预报中心。ECMWF 每个季节都会举办模型之间的预报竞赛。

To put their model to the test, last year the research team submitted three models that included their post-processing layer to the European Center for Medium-Range Weather Forecasts (ECMWF), which is considered the best forecasting center in the world. Each season ECMWF holds forecasting competitions between models.

获得第一名的提交将 ECMWF 的预报模型与另一个模型相结合，然后在其上应用后处理校正。这种方法最终使原模型在次季节范围内的预测能力翻了一番。该模型首次实现了优于依赖气候平均值的改进。

The submission that won first place combined ECMWF’s forecasting model with another, and then applied the postprocessing correction on top. That approach ended up doubling the original model’s predictive power in the subseasonal range. For the first time, the model offered an improvement over relying on climatologic averages.

“到目前为止，还没有人能够证明某个模型在这些次季节预报中比长期平均值更好地预测天气，”科恩说。“它们几乎全都比那更差——你完全可以抛硬币还做得更好。人们一直质疑在这个时间范围内投资模型是否值得。我们的模型始终优于气候平均值。”

“Up until now, no one had been able to show a model predicts weather better than long term averages for these subseasonal forecasts,” Cohen says. “They’ve pretty much all been worse than that — you could literally flip a coin and do better. People have questioned if it’s even worth investing in models at all for this time range. We consistently beat climatology averages with our models.”

该模型在预测极端天气的能力上也显示出统计上显著的改进。

The model also showed a statistically significant improvement in the ability to predict extreme weather.

“这是比较重要的发现之一，因为它很难实现，”科恩说。“当ECMWF开发出他们自己的、非AI的模型误差校正时，它在预测极端天气方面反而变差了。我认为这是因为传统的误差或偏差校正总是试图趋近均值，而极端天气处于概率的边缘。”

“That’s one of the more important findings, because it’s so difficult to accomplish,” Cohen says. “When the ECMWF developed their own, non-AI, error-correction to their model, it got worse at predicting extreme weather. I think that’s because traditional error, or bias, correction is always trying to get to the mean, whereas extreme weather is on the fringe of probabilities.”

为灾害做准备

Preparing for disaster

在《自然》论文中，团队展示了他们的技术可用于改善全球大多数地区的陆地预报。他们还将自己的偏差校正技术应用于其他模型，并表明这些模型也得到了改善，使甚至较弱的模型也能与ECMWF的预报相媲美。

In the Nature paper, the team showed their technique could be used to improve land forecasts in most regions across the globe. They also applied their bias correction technique to other models and showed it improved those as well, putting even lesser models on par with the ECMWF’s forecasts.

“我们模型的好处在于它是一个附加封装，”科恩解释道。“我们将它与ECMWF的动力模型一起使用，因为那被认为是最好的，但它可以与任何模型一起使用。它甚至能改善已经使用另一种AI算法进行偏差校正的模型。因此它显示出稳健的改进。而且它是开源且低成本的。”

“What’s nice about our model is it’s an add-on wrapper,” Cohen explains. “We used it with the ECMWF’s dynamical model because that was considered the best, but it can be used with any model. It even improves models that are already bias corrected using another AI algorithm. So it shows a robust improvement. And it’s open source and low cost.”

科恩说，肯尼亚共和国的气象部门已经联系他们，希望实施该模型以改善对年度季风季节的预报，从而提高今年冬季的农业产量。他希望其他团体也能使用该模型。

Cohen says the meteorological office for the Republic of Kenya has already reached out about implementing the model to improve forecasting of the annual monsoon season to improve agricultural yields this winter. He hopes other groups will also use the model.

“这可以帮助你决定何时围绕降雨施肥，何时在寒潮来临时保护冬季收获的作物，或帮助你避免干旱，”科恩说。“它也可能对能源和应急管理有用。我与能源官员交谈过，他们说，‘如果提前两天告诉我冰暴要来，我没有足够的时间，但如果我有两周时间，那可能足以做好充分准备。’”

“This could help you decide when to fertilize around rain, when there’s a freeze coming to protect crops during winter harvests, or help you avoid droughts,” Cohen says. “It could also be useful for energy and emergency management. I’ve talked to energy officials who say, ‘If I’m told two days before that an ice storm’s coming, I don’t have enough time, but if I had two weeks, that could be enough to make adequate preparations.”

这项工作部分得到了美国国家科学基金会、卡洛斯·查加斯·菲略研究支持基金会和欧盟的支持。

The work was supported, in part, by the U.S. National Science Foundation, the Carlos Chagas Filho Foundation for Research Support, and the European Union.

注意转载授权 · 阅读原文。中文正文为 AI 翻译，请以原文为准。

注意转载授权 · 阅读原文