Asphalt Pavement Potholes Localization and Segmentation using Deep Retina Net and Conditional Random Fields

Rana Ghazanfar Ali; Syed Muhammad Adnan; Nudrat Nida; Wakeel Ahmad; Farooq Bilal

Authors

Rana Ghazanfar Ali Department of Computer Science, University of Engineering & Technology Taxila, Pakistan
Syed Muhammad Adnan Department of Computer Science, University of Engineering & Technology Taxila, Pakistan
Nudrat Nida Air University, Islamabad, Aerospace & Aviation Campus Kamra, Pakistan.
Wakeel Ahmad Department of Computer Science, University of Engineering & Technology Taxila, Pakistan
Farooq Bilal Department of Computer Science, University of Engineering & Technology Taxila, Pakistan

Keywords:

RetinaNet, Pothole Segmentation, Conditional Random Fields (CRF), CAD tool, Region proposal

Abstract

The main aspect of maintaining the roads and highways' durability and long life is to detect potholes and restore them. A huge number of accidents occur on the roads and highways due to the pothole. It also causes financial loss to vehicle owners by damaging the wheel and flat tire. For the strategies of the road management system and ITS (Intelligent Transportation System) service, it is one of the major tasks to quickly and precisely detect the potholes. To solve this problem, we have proposed a deep learning methodology to automatically detect and segment the pothole region within the asphalt pavement images. The detection of the pothole is a challenging task because of the arbitrary shape and complex structure of the pothole. In our proposed methodology, to accurately detect the pothole region, we used RetinaNet that creates the bounding box around the multiple regions. For the segmentation we used Conditional Random Field that segments the detected pothole regions obtained from RetinaNet. There are three steps in our methodology, image preprocessing, Pothole region localization, and Pothole segmentation. Our proposed methodology results show that potholes in the images were correctly localized with the best accuracy of 93.04%. Conditional Random Fields (CRF) also show good results.

Full Text

References

C. Koch and I. Brilakis, “Pothole detection in asphalt pavement images”, Advanced Engineering Informatics, vol. 25, pp. 507–515, 2011.

A. Danti, J. Kulkarni, and P. Hiremath, “An Image Processing Approach to Detect Lanes”, Pot Holes and Recognize Road Signs. International Journal of Modeling and Optimization, vol. 6, pp. 658-662, 2012.

S. Nienaber, M. Booysen and R. Kroon, “Detecting Potholes Using Simple Image Processing Techniques and Real-world Footage”, 34th Southern African Transport Conference, 2015.

E. Buza, S. Omanovic and A. Huseinovic, “Pothole Detection with Image Processing and Spectral Clustering”, Recent Advances in Computer Science and Networking, vol. 810, pp. 4853, 2013.

M. Salman, S. Mathavan, K. Kamal and M. Rahman, “Pavement Crack Detection Using the Gabor Filter”, Proceedings of the 16th International IEEE Annual Conference on Intelligent Transportation Systems, The Hague, The Netherlands, October 6-9, 2013.

K. Azhar, F. Murtaza, M. H. Yousaf, H. A. Habib, “Computer Vision Based Detection and Localization of Potholes in Asphalt Pavement Images”, IEEE Canadian Conference on Electrical and Computer Engineering, 2016.

A. B. Makone, and A. G. Rathod., “Pothole Dimensions Measurement using Mean Shift-based Speckle Filtering” International Journal of Science Techonology and Engineering, vol. 2, 2016.

L. Huidrom, L. K. Das, S. K. Sud,“Method for automated assessment of potholes, cracks and patches from road surface video clips”, Procedia - Social and Behavioral Sciences, vol. 104, pp. 312 – 321, 2013.

Y. Jia, Z. He and H. Zhu, “A Hierarchical Segmentation Approach towards Roads and Slopes for collapse Recognition”, International Journal of Signal Processing and Pattern Recognition, vol. 6, pp.153-164, 2013.

M. Jahanshahi, F. Jazizadeh, S. Masri and B. Gerber, “Unsupervised approach for autonomous pavement-defect detection and quantification using an inexpensive depth sensor”, Journal of Computer and Civil Engineering, vol. 27, pp. 743-754,2013.

C. Koch, K. Doycheva, V. Kasireddy, B. Akinci, P. Fieguth, “A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure”, Advanced Engineering Informatics, vol. 30, pp. 208-210, 2016.

T. Lin, P. Doll´ar, R. B. Girshick, K. He, B. Hariharan, and S. J. Belongie, “Feature pyramid networks for object detection,” Computer Research Repository, vol. abs/1612.03144, 2016.

Conditional random field [Online]. Available: https://en.wikipedia.org/wiki/Conditional_random_field [Accessed].

T. L. XH, 1 and J. Ma, “Conditional Random Fields for Image Labeling”, Mathematical Problems in Engineering, 2016.

P. Krähenbühl, V. Koltun, “Efficient inference in fully connected crfs with gaussian edge potentials,” Advances in Neural Information Processing Systems, 2011.

L. Ale, N. Zhang, L. Li, “Road damage detection using RetinaNet.,” IEEE International Conference on Big Data (Big Data); 2018.

S. Ryu, T. Kim, Y. Kim, “Image-based pothole detection system for ITS service and road management system,” Mathematical Problems in Engineering. 2015.

P. Wang, “Asphalt pavement pothole detection and segmentation based on wavelet energy field”, Mathematical Problems in Engineering, 2017.

K. Christian, K. Gorgieva, V. Kasireddy, B. Akinci, P. Fieguth, “A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure,” Advanced Engineering Informatics, vol. 29, pp. 196–210, 2015.

“Fedral Highway Administration” , LTPP- Beyond FY 2009: What needs to be done? Technical Report, Publication No. FHWA-HRT-09-052, 2009

http://www.fha.dot.gove/pavement/ltpp/pubs/09025/09052.pdf

Taehyeong Kim, Seung-Ki Ryu, “Review and Analysis of Pothole Detection Methods” Journal of Emerging Trends in Computiing and Information Sciences , Vol. 5, No. 2014, ISSN 2079-8409.

Reddy, D. R., Goud, G. P. C., & Naidu, C. D. “Internet of Things Based Pothole Detection System using Kinect Sensor” 2019, Third International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), 2019.

J, D., V, S. D., S A, A., R, K., & Parameswaran, L. “Deep Learning based Detection of potholes in Indian roads using YOLO”. 2020 International Conference on Inventive Computation Technologies (ICICT). 2020.

Surekha Arjapure, D.R.Kalbande, “Deep Learning Model for Pothole Detection and Area Computation”,2021 International Conference on Communication information and Computing Technology (ICCICT), 2021.

LIN, T.-Y., GOYAL, P., GIRSHICK, R., HE, K. & DOLLÁR, P. “Focal Proceedings of the IEEE international conference on computer vision”, pp:2980-2988, 2017