AI Insights

Revolutionizing Short-term Traffic Congestion Prediction with Machine Learning

June 6, 2024

article featured image

In this success story, we explore a smart traffic management system. By harnessing the power of machine learning and computer vision, we’ve developed a system designed to predict short-term traffic congestion on urban roads, while combating the challenges posed by environmental noise and limited data accessibility.

Here’s how we did it!

The Problem

The Challenges of Traffic Congestion in Urban Areas

Traffic congestion is a significant problem in many urban areas around the world. It is characterized by slow vehicle speeds, longer travel times, and long queues, particularly in metropolitan areas with a high number of road users. This issue presents various challenges for policymakers and city managers in their efforts to improve urban quality of life.

Traffic congestion can lead to several negative consequences:

  • Economic Losses: Congestion results in wasted time, fuel, and productivity, leading to economic costs for individuals, businesses, and society as a whole.
  • Environmental Damage: Idling vehicles emit pollutants, contributing to poor air quality and climate change.
  • Health Impacts: Exposure to traffic-related air pollution is associated with respiratory issues, cardiovascular disease, and premature mortality.
  • Reduced Quality of Life: Congestion causes stress, frustration, and reduced time for family, leisure, and personal well-being.
  • Emergency Response Delays: Gridlock can hinder the timely arrival of emergency vehicles, potentially affecting emergency response times.

To address this complex challenge, data-driven solutions that leverage advanced technologies like machine learning and computer vision must be developed. These solutions can help predict and mitigate congestion, ultimately improving the health, prosperity, and livability of cities.

The Background


What is Short-term Traffic Congestion Prediction?

Short-term traffic congestion prediction involves forecasting the level of congestion on a given road segment or network in the near future, typically within the next few minutes to an hour. This is distinct from long-term predictions, which focus on larger time horizons and are often used for strategic planning purposes.

The goal of short-term prediction is to provide actionable insights that can be used for real-time traffic management, such as adjusting signal timings, deploying traffic control measures, or providing information to drivers to help them make informed route choices. By accurately anticipating congestion levels, city managers can proactively mitigate the impacts of gridlock and keep traffic flowing smoothly.

The Untapped Potential of Traffic Cameras in Congestion Prediction

Traffic cameras have become ubiquitous in cities around the world, providing real-time visual information about road conditions. These cameras are primarily used for monitoring and surveillance purposes, helping traffic management centers identify incidents, accidents, and other disruptions to the flow of vehicles.

However, despite their prevalence, traffic cameras have been largely underutilized for the purpose of predicting and mitigating congestion. The rich visual data captured by these cameras contains valuable insights about traffic patterns, vehicle density, and road occupancy, which could be harnessed to develop more accurate and responsive congestion prediction models.

By applying advanced machine learning and computer vision techniques to analyze traffic camera footage, we can extract real-time information about traffic conditions and use this data to forecast short-term congestion levels. This approach has the potential to revolutionize traffic management, enabling cities to proactively address congestion before it becomes a major problem, rather than merely reacting to it after the fact.

The Goal

The main objective of the Traffic Congestion Prediction Project was to harness the untapped potential of traffic camera data and advanced AI techniques to revolutionize short-term traffic congestion prediction. By developing a comprehensive, end-to-end solution that leveraged machine learning and computer vision to analyze real-time traffic camera footage, we aimed to provide city managers with accurate, actionable insights for proactive traffic management.

Beyond our main goal, we set out to achieve the following as well:

  • Expand Geographic Coverage: Obtain traffic camera image data for a new city in Europe or North America to demonstrate the scalability of our approach.
  • Improve Prediction Accuracy: Enhance the performance of our machine learning models by developing techniques to eliminate environmental noise, such as trees and dual lanes, that can confound predictions.
  • Develop an End-to-End Solution: Create a complete pipeline encompassing data collection, preprocessing, modeling, and deployment, enabling seamless integration with existing traffic management systems.
  • Advance the State of the Art: Push the boundaries of machine learning and computer vision techniques applied to traffic congestion prediction, setting a new standard for accuracy and efficiency.
  • Drive Real-World Impact: Demonstrate the tangible benefits of our approach in reducing congestion, improving travel times, and enhancing urban livability, paving the way for wider adoption by cities worldwide.

By achieving these goals, we aimed to revolutionize the way cities manage and mitigate traffic congestion, harnessing the power of AI for smarter, more sustainable urban mobility.

Our Approach

Step 1. Comprehensive Data Collection

Tapping into a Wealth of Visual Data: Accessing Traffic Camera APIs Worldwide

To expand the geographic scope of our project, we identified multiple public APIs providing access to traffic camera images from cities across Europe and North America. We collected images at 5-30 minute intervals over a period of 24 hours from cameras in Ottawa, New York, Illinois, and London, amassing a dataset of over 100,000 raw images.

While some API providers imposed rate limits, such as allowing only one request per 60 seconds, we adapted our collection process to ensure adequate coverage throughout the day and night. By capturing images at regular intervals, we aimed to build a representative dataset encompassing a wide range of traffic conditions and lighting scenarios.

It’s worth noting that our data collection took place during the festive season, which may have influenced the observed traffic patterns. Nevertheless, this diverse dataset provided a solid foundation for training and testing our machine learning models, enabling us to assess their performance across different geographies and time periods.

Step 2. Collaborative Data Annotation

Harnessing the Power of Crowdsourcing: Labeling Images with Labelbox

To train our machine learning models, we needed a labeled dataset where each image was annotated with its corresponding congestion level. We focused on the 5,183 images collected from Ottawa, as they provided a good balance of highway and city scenes.

Using the Labelbox platform, our team collaborated to manually label each image as “High,” “Medium,” or “Low” congestion, as well as “Blurry” or “Camera-off” for images that were unclear or unavailable. To ensure consistency, we established criteria for each label and held group discussions to review edge cases.

Screenshot of the Labelbox interface

Screenshot of the Labelbox interface

To streamline the process, we uploaded the predictions from our base model as pre-labels, allowing annotators to quickly approve correct labels while focusing their efforts on refining inaccurate ones. This iterative approach enabled us to efficiently create a high-quality labeled dataset, which we later split into training and testing subsets to evaluate our models’ performance.

Examples of traffic light detection using the Yolov7 object detection-based filter

Examples of traffic light detection using the Yolov7 object detection-based filter

Step 3. Intelligent Data Preprocessing

Filtering Out the Noise: Developing Blurry Image and Traffic Light Detectors

To address the challenge of environmental noise that can confound traffic congestion predictions, we developed a suite of preprocessing techniques to filter out problematic images before feeding them to our models.

First, we tackled the issue of blurry images by creating two complementary solutions: a YOLOv5-based binary classifier and an anomaly detection model. Both approaches aimed to identify and remove images that were too blurry to yield accurate predictions, thereby improving the overall quality of our dataset.

Sample predictions by the anomaly detection-based filter

Sample predictions by the anomaly detection-based filter

Next, we recognized that the presence of traffic lights could also mislead our models, as they often indicate localized congestion unrelated to overall traffic flow. To mitigate this, we developed a YOLOv7-based traffic light detector, which could flag images containing traffic lights and exclude them from further analysis.

Zeroing In on What Matters: Creating Lane Masks to Isolate Key Features

Beyond filtering out noise, we hypothesized that focusing our models’ attention on the most relevant parts of the image – namely, the road lanes themselves – could significantly improve prediction accuracy. To test this, we created binary masks that isolated the lanes from the background, using a combination of manual annotation and automatic image clustering techniques.

Mask creation and Inference results on masked images

Mask creation and Inference results on masked images

By applying these masks to our images before feeding them to the models, we effectively removed distracting elements like buildings, trees, and sidewalks, allowing the algorithms to concentrate on the key features that directly influence traffic congestion levels. This targeted approach aimed to boost the signal-to-noise ratio and enable more precise predictions.

Step 4. Cutting-Edge Modeling

Pushing the Boundaries of Accuracy: Training State-of-the-Art Neural Networks

Building on the solid foundation of our preprocessed and annotated dataset, we embarked on a comprehensive modeling effort to develop highly accurate traffic congestion prediction algorithms. Our approach centered on training state-of-the-art neural networks, with a particular focus on the EfficientNet architecture, which has achieved remarkable results in computer vision tasks.

We began by fine-tuning pre-trained EfficientNet models on our labeled dataset, leveraging transfer learning to adapt these powerful networks to the specific characteristics of traffic camera images. Through a process of iterative experimentation and hyperparameter tuning, we gradually improved the performance of our models, achieving significant gains over the baseline.

Plot of training accuracy, validation accuracy, and test accuracy

Plot of training accuracy, validation accuracy, and test accuracy

To further enhance accuracy, we explored a range of techniques, including data augmentation, class balancing, and ensemble methods. By carefully crafting training pipelines that incorporated these strategies, we were able to create models that consistently outperformed benchmarks and demonstrated robust generalization across different geographies and lighting conditions.

Beyond Classification: Estimating Traffic Density through Vehicle Detection

While our primary focus was on predicting congestion levels through image classification, we recognized that a more granular understanding of traffic density could provide valuable insights for city planners and traffic managers. To this end, we developed a complementary approach based on vehicle detection and counting using the YOLOv7 object detection model.

By training YOLOv7 on a large dataset of annotated vehicle images, we created a model capable of accurately localizing and classifying vehicles in traffic camera footage. We then integrated this model with our lane segmentation masks, enabling us to count the number of vehicles in each lane and estimate the overall traffic density.

Vehicle and Vehicle Direction Detection

Vehicle and Vehicle Direction Detection

This density estimation approach offered several advantages over pure classification. First, it provided a more interpretable and actionable metric for assessing congestion levels, as the number of vehicles directly corresponds to the observed traffic conditions. Second, it allowed for more fine-grained analysis of traffic patterns, such as identifying bottlenecks or asymmetric flows between lanes.

StrongSORT tracker for traffic flow rate measurement

StrongSORT tracker for traffic flow rate measurement

By combining the strengths of our classification and density estimation models, we aimed to create a comprehensive system for predicting and understanding short-term traffic congestion, empowering cities to make data-driven decisions and optimize their transportation networks.

Step 5. Seamless Deployment

Integrating APIs, Filters, and Models: Building an End-to-End Solution

To bring our traffic congestion prediction system to life, we focused on creating a seamless, end-to-end pipeline that integrated the various components developed throughout the project. This included the Ottawa traffic camera API, the YOLOv5-based blurry image filter, the traffic light detection filter, and the base congestion prediction model.

We began by developing a Flask application that could pull live images from the Ottawa highway and city camera APIs, demonstrating the system’s ability to process real-time data from multiple sources. The application’s user interface provided an intuitive way to visualize the predicted congestion levels for each camera feed.

Smart-Traffic user interface

Smart-Traffic user interface

Next, we incorporated the blurry image filter into the pipeline, applying it to each image before passing it to the base model for congestion prediction. This ensured that only high-quality images were used for analysis, improving the overall accuracy of the system. When tested with live data from the Ottawa API, the filter performed well in most cases, although it struggled with some edge cases like foggy weather conditions. This highlighted the need to further diversify the training data to improve the filter’s robustness.

We also integrated the YOLOv7-based traffic light detection filter, which could flag images containing traffic lights and provide additional context for the congestion predictions. When a traffic light was detected with a confidence score of 30% or higher, the system would display a message indicating that users might experience temporary wait times. This feature aimed to provide a more nuanced understanding of the factors contributing to congestion at a given location.

Throughout the deployment process, we focused on optimizing the system’s performance and efficiency. For example, we traced the traffic light detection model at the start of the application to reduce inference times, achieving speeds of 2-3 seconds per image. We also carefully monitored the processing times for each component, identifying bottlenecks and implementing optimizations where possible.

By integrating these various elements into a cohesive, end-to-end solution, we demonstrated the feasibility and potential impact of our approach to traffic congestion prediction. The deployed system showcased the power of combining advanced machine learning techniques, computer vision algorithms, and real-time data streams to provide actionable insights for traffic management and urban planning.

The Outcome

  • Expanded Geographic Coverage: Successfully collected and processed traffic camera data from multiple cities across Europe and North America, demonstrating the scalability and transferability of our approach.
  • Improved Prediction Accuracy: Developed advanced preprocessing techniques, including blurry image filtering and lane masking, which significantly enhanced the performance of our congestion prediction models.
  • State-of-the-Art Models: Trained cutting-edge neural networks, such as EfficientNet, that consistently outperformed baseline models and achieved high accuracy in classifying congestion levels.
  • Comprehensive Solution: Created an end-to-end pipeline encompassing data collection, annotation, preprocessing, modeling, and deployment, providing a complete solution for short-term traffic congestion prediction.
  • Complementary Density Estimation: Developed a vehicle detection and counting system using YOLOv7, enabling granular analysis of traffic density and flow patterns.
  • Real-World Impact: Demonstrated the potential of our approach to reduce congestion, improve travel times, and enhance urban livability, laying the foundation for wider adoption by cities worldwide.
  • Knowledge Sharing: Contributed to the advancement of urban analytics and machine learning for transportation, through publications, open-source code, and collaboration with the research community.

Benefits and Applications

Transforming Transportation: The Far-Reaching Impact of Intelligent Traffic Management

The methodologies and technologies developed in this project have the potential to revolutionize traffic management and urban planning, offering a wide range of benefits and applications beyond congestion prediction.

  • Real-Time Optimization: Our system enables dynamic adjustments to signal timings, traffic rerouting, and resource deployment, reducing travel times, fuel consumption, and emissions for a more efficient and sustainable transportation network.
  • Improved Emergency Response: Integration with emergency response systems helps first responders navigate traffic effectively, ensuring faster arrival times and potentially saving lives.
  • Enhanced Public Transit Planning: Analyzing traffic patterns informs the optimization of public transit networks, encouraging sustainable travel and reducing reliance on private vehicles.
  • Smarter Infrastructure Investments: Our insights guide strategic investments in transportation infrastructure, prioritizing projects that address the most pressing needs.
  • Empowering Commuters: Integrating our predictions with navigation apps and public information systems allows commuters to make informed decisions, reducing their exposure to congestion and improving their travel experience.
  • Advancing Urban Analytics: Our techniques and methodologies contribute to the field of urban analytics, demonstrating the potential of machine learning and computer vision to extract insights from data sources like traffic camera footage. We aim to inspire further research and collaboration in this domain, driving progress towards smarter, more sustainable cities.

Want to work with us too?

media card
Using Advanced Data Mining Techniques for Educational Leadership
media card
How We Leveraged Advanced Data Science and AI to Make Farms Greener
media card
How We Created an Innovative Solution for Power Accessibility without the Available Resources
media card
How AI Can Protect Our Water: Detecting The Invisible Threats Within