Ensembles: The latest trend in AI forecasting tools

What do you picture when you imagine a weather forecast? Likely, you’re visualising a single number which provides you with an indication of what the weather will be like at a certain time and day. It could be 23C and blue skies (sadly a rarity in the UK), or at 10am it could be particularly windy, with wind force predicted at 14km/h.

The reality of forecasting is, of course, much more complicated than a single number. The weather itself is a chaotic system, meaning there will always be uncertainties that make it impossible to perfectly predict. A tiny error of the initial state of the atmosphere as an input could mean large errors in the resulting forecast.

One of the most famous representations of this is of course, the butterfly effect, a term closely associated with the work of mathematician and meteorologist Edward Norton Lorenz. He posed the question, “Does the flap of a butterfly's wings in Brazil set off a tornado in Texas?”, meant to visually demonstrate how seemingly small and ineffectual actions could have unpredictable, larger impacts within a chaotic system.

Ensemble forecasting is a state of the art forecasting method created to help combat the chaotic, unpredictable nature of weather.

A brief history of ensemble forecasting

For the majority of time that forecasts have been operational, they were reliant on Numerical Weather Predictions (NWPs), a type of forecasting that uses mathematical models of the atmosphere to predict the weather based on current conditions.

In the early days of forecasting centres, the larger, established organisations like the European Centre for Medium-Range Weather Forecasts (ECMWF), focused on using NWPs to create one “deterministic” forecast, using a singular input and forecasting a singular result. Then, in the 1980s, the ECMWF and Professor Tim Palmer experimented with using the ensemble forecasting method for short and medium range weather forecasting.

The ensemble (or group) of forecasts attempts to encompass the uncertainty of weather by running the forecasting model a number of times from slightly different starting conditions within a plausible range of what’s possible. This provides a range of possible outcomes and the likelihood of, for example, hazardous weather or extremes. Take agriculture as an example: a farmer needs to know the range of possible conditions the crops may experience so that they can be protected. Ensemble forecasts show how big that range is at different forecast times.

These forecasts generate multiple predictions, which forecast providers can then use to aggregate into a probabilistic model, meaning they can use the many forecasting results to determine the most likely future of the weather.

In December 1992 operational ensemble forecasts began at ECMWF, as well as the US National Centers for Environmental Prediction. This was closely followed by many other forecasting centres around the world implementing NWP ensemble forecasts operationally.

AI weather forecasting & ensembles

As mentioned above, ensemble forecasts are usually made with NWP models, changing the initial conditions from which the NWP is created, developing bounds of uncertainty, and then running the mathematical equations forward to get different predictions of weather patterns over time.

NWPs take hours to run, whereas AI weather prediction models are much faster, and so are able to generate ensembles much, much quicker. Let’s use Google DeepMind’s GenCast as an exploratory example.

GenCast comprises an ensemble of 50 or more predictions, each representing a possible weather trajectory. Generating a single 15-day GenCast forecast takes about 8 minutes on moderately powerful hardware, and an ensemble of forecasts can be generated in parallel. In comparison, it would take traditional forecasting methods using NWPs many hours running on a super computer, requiring a lot more compute power and using much more energy.

Further, results showed that when tested against traditional models, GenCast was more accurate in over 97% of forecast targets.

Can we use weather ensembles to improve AI-led renewable generation forecasting?

There are two benefits of using weather ensembles in renewable generation forecasting:

It gives energy forecasters, meteorologists, and electricity system operators, like the UK’s National Energy System Operator (NESO), a more holistic view of potential energy variations in the grid.
By using the entire ensemble, accuracy of our generation forecasts has been proven to improve significantly.

A more holistic view

As discussed, within the many members of an ensemble forecast, a single thread provides users with one ‘what if’ scenario. This ensemble member can then be used to explore how that one scenario could impact the whole electricity system across demand, solar and wind generation.

Let’s take a real-world example to illustrate, specifically the recent UK-based Storm Benjamin. When the storm hit the UK, that corresponded directly with higher wind power generation, and lower solar PV generation.

A single ensemble forecast would provide system operators, like NESO, a coherent scenario for if the storm was predicted to arrive, let’s say, earlier than expected. This enables operators to have more comprehensive overviews of potential grid-balancing problems.

More accurate forecasts

Working with NESO, we undertook an exciting research project to demonstrate how weather ensembles could improve our cutting-edge solar PV generation forecast.

NESO is one of the users of PVNet, our machine learning model that provides them with leading forecasts for solar PV generation in Great Britain. Using PVNet, we can provide two main types of forecast: deterministic and probabilistic. The deterministic forecast is seen as our ‘best’ forecast, a singular prediction with what we believe is the most accurate outcome.

Our probabilistic forecast takes this accuracy even further. It generates an interval in which we expect the actual solar generation to fall with a set probability; for example, we can predict that the PV generation will be within a certain range with 80% probability.

The confidence intervals produced by PVNet account for two things:

Weather uncertainty (what NWP ensembles account for)
Generation uncertainty (you can have different generation under the same observed weather conditions, as it’s not the only thing affecting it)

Our first experiment focused on separating these two sources of uncertainty, to provide the system operator with a better understanding of the factors affecting the generation. To do this, we ran all 50 ensembles through a pre-trained PVNet, and collected the singular, deterministic forecast from each run. This provided us with 50 versions of the forecast, which represents the spread of possibilities depending on the weather. It can be used to produce the confidence intervals that will account for weather uncertainty only, and hence will be a lot narrower: for instance, the confidence interval that is supposed to capture 96% of possible outcomes only captured 60%.

Our further experimentation focused on improving the quality of our predictions by leveraging the ensemble data. In these experiments, we ran the 50 ensemble members through PVNet and collected both the point predictions and the confidence intervals they produced, ending up with 50 versions of each interval, each relying on a different ensemble member. We then used these predictions and intervals to produce a final estimate, both as a deterministic forecast and of the pockets of uncertainty. This improved the accuracy of our main forecast and the quality of the uncertainty boundaries by 5%. This is a significant improvement at the cutting edge of energy forecasting, demonstrating the substantial impact weather ensembles can have on generation forecasting.

This only scratches the surface of the incredible work our team has undertaken to research and implement ensemble forecasting. Keep an eye out on our LinkedIn and Blog for future updates about that work!

Get in touch

At OCF, we're leveraging ensembles within our renewables generation forecasting, ensuring we continue to build the best tools to accelerate the transition to clean energy.

Do you have questions about our work, or want to access our forecasting tools?

Contact the team, we’d love to hear from you!

Discover our WP6 Project Code

Contact the team