For my master’s seminar, I used Mask Region Proposal Convolutional Neural Network (Mask-RCNN) to detect pavement distresses, traffic signs, and damages on images acquired from the Street View. Employing such a technique can help cities with their decision-making process regarding the maintenance of their infrastructures. The proposed technique does not require a dedicated data collection operation that makes it faster and cheaper than other conventional methods, especially in metropolises with large networks. This project was designated as one of the most innovative projects in the Innoroad 2020 competition.
A computer vision-based pavement distress inspection algorithm proposed for the network-level pavement management in an urban area. In practice, deploying this software would empower municipalities to constantly monitor the pavement condition in their networks and assign their resources optimally. During the last decade, more than 6 percent of the Tehran municipality’s annual budget has been spent on pavement maintenance and rehabilitation without any comprehensive pavement management scheme. On the other hand, the continuous monitoring of the pavement could be expensive and challenging.
In this study, a novel data acquisition technique was developed that can cope with the characteristics of this problem in urban areas. This technique uses the images acquired from a panorama imagery service (like Google Street View™) and analyzes them with the Mask Region Proposal Convolutional Neural Network (Mask-RCNN) algorithm to identify and extract the characteristics of the pavement surface distresses. Employing such a scheme can help cities with their decision-making process regarding the maintenance of their infrastructures. The proposed technique does not require dedicated data collection that makes it faster and cheaper than other conventional methods, especially when it comes to metropolises with large networks.
Like other supervised learning methods, Mask-RCNN requires a quite large dataset. In this project’s prototyping phase, I used the GAPs v2 labeled dataset to train my network. After proving the credibility of such an approach to the stakeholders, we are now creating our own dataset.
To validate the approach, two urban and rural road segments were selected which are being demonstrated in the map below.
The trained algorithm successfully detected 42 percent of all pavement distresses (different accuracy within different distress types) and about 83 percent of the cracks. We hope with the dataset we are creating and standardizing the imagery hardware the outputs become more accurate and trustworthy. It should be mentioned that the only two other research that had the same vision as this project, reached a similar accuracy. For instance, using Google Street View images, Chacra et. al. achieved 93 percent accuracy in crack detection  while the holistic approach that detects most of the pavement distresses developed by Ma et al. achieved 58.2% accuracy .
Now we are working on the following features:
 Chacra, D. A. and Zelek, J. (2018) ‘Municipal infrastructure anomaly and defect detection’, European Signal Processing Conference. EURASIP, 2018-Septe, pp. 2125–2129. doi: 10.23919/EUSIPCO.2018.8553322.
 Hoai, M., Brook, S. and Samaras, D. (2017) ‘Large-scale Continual Road Inspection : Visual Infrastructure Assessment in the Wild’, Bmvc.