Forecasting Electricity Prices for Optimal Usage of Renewables in Norway

Problem Statement

Since the Paris Agreement was signed in 2016, many countries have intensified their efforts to fight climate change. The agreement’s main goal is to limit global temperature rise by reducing greenhouse gas emissions.

Norway has taken a leading role in this global transition. According to the International Energy Agency (IEA), the country is a pillar of energy security and a model for adopting clean technologies. Recently, Norway announced plans to shift from oil and gas production toward becoming an exporter of renewable electricity.

This AI Challenge, hosted by Omdena in partnership with Think Outside, aimed to develop solutions that predict both water availability and electricity prices. These two elements are vital for optimizing hydroelectric power production in Scandinavia.

To achieve this, the project was divided into two parts — water inflow prediction and electricity price prediction. This article focuses on the second challenge: forecasting electricity prices to help balance renewable energy production and distribution.

Approach

Countries that are part of the Nordpool market are divided into several bidding zones for energy production and distribution. These zones are connected to form a coupled energy market, ensuring that regional conditions are accurately reflected in electricity prices.

Each bidding zone can experience either a balance, deficit, or surplus of electricity. Power typically flows from areas with lower demand and lower prices to regions where demand and prices are higher.

Norway is divided into five bidding zones:

• NO1 – Oslo
• NO2 – Kristiansand
• NO3 – Molde, Trondheim
• NO4 – Tromsø
• NO5 – Bergen

Fig. 1. Norway’s five bidding zones

In our models, we focused on predicting electricity prices for four of these zones: NO1 (Oslo), NO2 (Kristiansand), NO3 (Molde), and NO4 (Tromsø). Trondheim was excluded because, since 2014, it has shared the same bidding zone as Molde. Kristiansand was also excluded, as our descriptive analysis showed that its prices were identical to those of Bergen, with a correlation of 1. Therefore, the price data from other zones could be used as a reference.

The project pipeline followed a standard machine learning workflow. First, we defined the problem and its business relevance, followed by thorough research on the topic. Next, we gathered data from various sources and automated its preprocessing to merge all necessary features into a single dataset. After handling missing values, we conducted an exploratory data analysis (EDA) to identify correlations and visualize key trends

In the modeling phase, we tested multiple algorithms to predict the Elspot prices for each Norwegian region. Finally, we visualized and compared the results to evaluate model performance and accuracy.

Fig. 2. Project Pipeline (Source: Omdena)

Data

The dataset used for this project covered the period from 2014 to early 2022. It was divided into five main categories: Nordpool data, commodity prices, world market index, time-related data, and weather data. The following sections summarize each of these categories.

Nordpool data:

The Nordpool database served as the main source because it provides extensive electricity-related data. To better capture external market effects, we also included data from other Nordpool countries.

The features collected from this database included:

• Elspot prices
• Elspot volumes
• Regulating prices
• Elspot capacity incl. production imbalance prices (sales/purchases),
• Elspot transmission capacities and Elbas initial transmission capacities
• Consumption prognosis
• Production prognosis
• Elbas Volume

Commodity prices:

Commodity prices directly affect electricity production costs and, in turn, influence electricity prices. For this reason, we added data for gas, Brent oil, and coal obtained from finanzen.net.

World market index:

To account for global economic conditions, we included the MSCI World Index ETF. This helped represent geopolitical or financial events not captured by commodity prices. Although oil prices are often linked to this index, our analysis showed that their trends occasionally diverged. Including both improved model reliability.

Time related features:

• holidays, weekends
• weekly, monthly, and yearly encoded features (transformed with cos/sin functions)

The Nordpool database was the main source for our dataset as it offers a lot of data in terms of electricity. We did not only include data of Norway but also of other Nordpool countries in order to account for their affects regarding electricity prices as well. Commodity prices, like oil, gas, and coal affect electricity production costs and therefore directly influence electricity prices. Therefore, we included commodity price data from finanzen.net in our models. In addition, we included the world market index ‘MSCI World index ETF’, to account for geopolitical and economic global events that are not reflected in oil prices. Our data comparison showed that oil prices, although quite closely connected to the index prices, do not always follow their price trends and so it was beneficial to include the stock index. Furthermore, we included as stated above various time related features including cyclical encoding features.

Weather data:

Weather significantly influences electricity demand and renewable energy production. For example, cold winters in Norway increase electricity use for heating, while wind and precipitation impact renewable energy generation.

We used weather variables such as:

• Precipitation
• Average temperature
• Wind speed
• Relative and specific humidity
• Short-wave radiation

Initially, we gathered weather data from the Norwegian Water Resources and Energy Directorate (NVE), which provides high-resolution data for each square kilometer in Norway. However, to make the model scalable to other countries, Think Outside requested a global data source. Therefore, we used GloH2O, which provides worldwide weather data at a resolution of 0.1 degrees.

Fig. 3. Comparison of weather data resolution from NVE and GloH2O

GloH2O delivers data as raster files (netCDF), where each pixel represents a specific area defined by longitude and latitude, with values such as temperature or precipitation assigned to each pixel. Due to time constraints, we averaged the pixel values within each bidding zone to obtain a single daily measurement for every variable.

Fig. 4. Weather data extraction from netCDF files

Although this approach was practical, using sub-zones or pixel-level data could improve prediction accuracy. For instance, temperatures in remote northern areas influence demand less than those in urban regions, and precipitation near hydropower reservoirs affects electricity generation more than rainfall elsewhere.

Machine Learning models:

Before developing the models, we preprocessed and analyzed the data carefully. Missing values were filled, and exploratory data analysis (EDA) was performed to identify key relationships and trends.

We tested two main types of models: regression models (Ridge and Lasso) and SARIMAX. For both types, the dataset was divided into training and test sets, following standard machine learning practices. Each model was trained using the training data and then evaluated on the test data to measure its performance.

Regression models:

For the regression models, we shuffled the dataset so that data points were randomly assigned to the training or test sets. This approach improved the model’s performance. However, shuffling is only suitable for models that do not depend on time sequences, which is why we did not apply it to the SARIMAX model.

We also performed hyperparameter tuning using the Optuna package. Optuna’s advantage lies in its ability to efficiently search for the best parameters without requiring excessive computation time. The goal of this step was to identify the parameter combinations that produced the lowest Root Mean Squared Error (RMSE).

SARIMAX model:

The SARIMAX model (Seasonal AutoRegressive Integrated Moving Average with eXogenous factors) was used to handle time-dependent data. Since this model is computationally intensive, we included only a limited number of external features such as Elspot volumes (buy/sell), weather data, and commodity prices.

For SARIMAX to work effectively, the input data must be stationary. We tested this using the Augmented Dickey-Fuller (ADF) test, which showed that our target variable was not stationary. To correct this, we applied first differencing, which stabilized the data.

Finally, we selected the best model parameters using the Akaike Information Criterion (AIC), ensuring that the chosen configuration provided the best balance between accuracy and complexity.

Results

The figures below show the electricity price predictions for the Oslo bidding zone using both the Ridge regression and SARIMAX models.

For the Ridge regression model, hyperparameter tuning and grid search identified it as the best-performing approach. It achieved an RMSE of 5.94 and an R² score of 0.972, showing that the predicted and actual prices were closely aligned. This high accuracy demonstrates the model’s strong predictive capability.

Fig. 5. Oslo Elspot prices prediction with Ridge (Source: Omdena)

Figure 6 shows the results for the SARIMAX model for Oslo which has a higher RMSE (12.18) than the Ridge regression.

Fig. 6. Oslo Elspot prices prediction with SARIMAX (Source: Omdena)

The SARIMAX model, on the other hand, produced an RMSE of 12.18, which is higher than that of the Ridge regression model. While it did not perform as precisely, SARIMAX has the advantage of considering the entire time series history, which can be valuable for long-term forecasting.

Across all bidding zones, the regression models achieved RMSE values ranging from 5 to 8, while the SARIMAX models ranged from 7 to 12. This shows that regression models performed better overall in this project. For a broader perspective on how AI models can enhance renewable systems globally, explore our article on AI-powered solutions optimizing renewable energy production.

However, it is important to note that regression models make day-ahead predictions using only the data from the current day. In contrast, SARIMAX uses all previous data points for each prediction. As a result, regression models may adapt more slowly to sudden market changes compared to SARIMAX.

Existing research also highlights that other modeling techniques, such as deep learning models like LSTM, can further improve prediction accuracy. Due to time constraints, the team could not test these models within this project, but they remain promising candidates for future work.

Conclusion and social impact

Forecasting electricity prices is substantial not only for companies but also for society. Recent developments in the energy market (gas and electricity) caused by the Ukrainian War have shown how important and interdependent the energy market is. Hence, accurate energy price predictions that take into account several factors – as in this challenge by including very diverse exogenous features – are important for the industry and governments in order to be able to react by making adjustments or adequate policies.

Authors:

• Evanthia Fasoula: All sections apart from data description and weather data.
• Yasser Zouzou: Data description and weather data sections.

Collaborators:

Aakanksha Chouhan, Andrew Henry, Anne Losch, Deepali Bidwai, Devika Pace, Ekaterina Paerschke, Elena Barbulescu, Evanthia Fasoula, Hamzah Warsi, Huy-Thong Phan, Joan Vlasschaert, Jorge de Vivero, Kartikey Saini, Keerthana Perumal, Leon Hamnett, Miguel Sindreu, Mihaela Borta, Mircea-Margarit Nistor, Niklas Schlessmann, Noel Simonovici, Peter Rockwood, Suganthi Giridharan, Titilayo Amuwo, Torsten Walther, Yangyang Cai, Yasser Zouzou, Zaw Thu Htet, Zeyneb Chiha

Product Owner: Shrey Grover