AI Insights

Rooftops Classification and Solar Installation Acceleration using Deep Learning

July 29, 2021


article featured image

In this article, we will go through the process of commercial rooftops classification using Deep Learning methods. And accelerate the installation of solar energy in North America, as a part of Omdena’s challenge with a Techstars EnergyTech startup.

A walkthrough in understanding the use of labeling tools like CVAT in labeling satellite imagery, the difference between semantic and instance segmentations with rooftops images pixels, and training different pre-trained deep learning models to get the best roof classification results.

Solar energy hasn’t reached its full potential as a clean energy source for the United States yet. According to National Renewable Energy Laboratory (NREL) analysis in 2016, there are over 8 billion square meters of rooftops on which solar panels could be installed in the United States, representing over 1 terawatt of potential solar capacity.

Therefore, significant work remains to be done to accelerate the deployment of solar energy in the US by identifying potential locations and rooftops classification.

Problem Statement

The goal of this 2 months challenge was to detect rooftops and provide rooftops classification in North America in order to identify the potential of facilities’ solar installation in this region. Solar rooftop potential for the entire country is the number of rooftops that would be suitable for solar power, depending on the uncluttered surface area, shading, direction, and location.

The final goal of this challenge is to detect rooftops, and give understandable rooftops classification thus, accelerate the growth of solar installations in the United States.

So, the first step was to gather, preprocess and label the data.

Data preparation and images labeling

For this project, we used satellite images of the rooftops at specific GPS coordinates. We used the GPS coordinates provided by the partner. All the images were downloaded from Mapbox. Then, we manually labeled a large amount of the images using the labeling tool CVAT (Computer Vision Annotation Tool). CVAT is a free, open-source, web-based image annotation tool. It is a very powerful and efficient tool that allows multiple collaborators to label images, and then review the labeling done.

Roof classification - Source: Omdena

Demonstration of labeling done on a rooftop in CVAT — source: Omdena

Roof classification - Source: Omdena

GIF showing CVAT export formats — source: Omdena

  • Flat: Rooftops with a single flat surface with/without clutters
  • Complex Rooftop: Rooftops with multiple surfaces at different heights
  • Existing Solar: Rooftops with solar panels
  • Heavy Industrial: Rooftops with pipes, and cluttered with machinery
  • Slope: Rooftops with an inclined surface
Image labeling for roof classification - Omdena

Examples of images and their label — Source: Omdena

Rooftop Segmentation

We needed to try different segmentation methods to get the best results.

Semantic Segmentation

We started by segmenting rooftops using semantic segmentation models. Semantic segmentation is the process of attributing a class to each pixel of an image. So in our case, we want to classify all pixels as rooftop or non-rooftop. We are implementing a binary semantic segmentation of rooftops.

Example of mask used for labeling — source: Omdena

Example of mask used for labeling — source: Omdena

U-Net binary segmentation of rooftops — source: Omdena

U-Net binary segmentation of rooftops — source: Omdena

Multi-classes semantic segmentation results with U-Net — source: Omdena

Multi-classes semantic segmentation results with U-Net — source: Omdena

Deep-Lab predictions — source: Omdena

Deep-Lab predictions — source: Omdena

Instance Segmentation

Instance segmentation differs from semantic segmentation in the way that with semantic segmentation the model predicts:

Labeled rooftops images exported into COCO format — source: Omdena

Labeled rooftops images exported into COCO format — source: Omdena

‘AP-flat’: 34.46745607306496
‘AP-slope’: 9.342863008612126
‘AP-existing_solar’: 1.7519702923446772
‘AP-complex_rooftop’: 14.343733930311137
‘AP-heavy_industrial’: 5.055339100777756
Mask R-CNN results — source: Omdena

Mask R-CNN results — source: Omdena

Examples of the wrong classification with Mask R-CNN — source: Omdena

Examples of the wrong classification with Mask R-CNN — source: Omdena

Predictions Mask R-CNN — source: Omdena

Predictions Mask R-CNN — source: Omdena

Mask R-CNN improved results with learning rate = 0.001 and 4000 epochs — source: Omdena

Mask R-CNN improved results with learning rate = 0.001 and 4000 epochs — source: Omdena

Box Localization Loss — 0.70
Confidence Loss — 1.6
Mask Loss — 1.1
Semantic Segmentation Loss — 0.40

Yolact’s results for roof classification — source: Omdena

Yolact’s results for roof classification — source: Omdena

Final Deliverable

Our final product is a Streamlit application that uses the Yolact pre-trained weights in rooftops classification on images that the user can either upload directly on the app or can access through GPS coordinates.

Visual of the rooftops classification Streamlit application in the class prediction section of the application — source: Omdena

Visual of the roof classification Streamlit application in the class prediction section of the application — source: Omdena

Conclusion

This 2 months project was a lot of fun and we learned so much. Using satellite images is always a challenge, but I think, the most challenging part of this project was the labeling effort done by the team which is incredible!

More approaches accomplished by Omdena in the Energy sector:

This article is written by Margaux Masson-Forsythe

Ready to test your skills?

If you’re interested in collaborating, apply to join an Omdena project at: https://www.omdena.com/projects

media card
Harnessing AI to Monitor and Optimize Reforestation Efforts in Madagascar
media card
Ace Your Next Interview: How AI-Powered Mock Interviews are Revolutionizing Job Preparation
media card
How We Leveraged Advanced Data Science and AI to Make Farms Greener
media card
Interactive Geospatial Mapping for Crime Prevention