An Efficient and Robust Deep Learning Approach for Vehicle Recognition using Light-weight Deep Network

Fatima Shahzad; Muhammad Hassan Khan; Muhammad Shahid Farid; Nazish Ashfaq

Authors

Fatima Shahzad Department of Computer Science, University of the Punjab, Quid-e-Azam Campus, Lahore, 100190, Pakistan, Pakistan.
Muhammad Hassan Khan Department of Computer Science, University of the Punjab, Quid-e-Azam Campus, Lahore, 100190, Pakistan, Pakistan.
Muhammad Shahid Farid Department of Computer Science, University of the Punjab, Quid-e-Azam Campus, Lahore, 100190, Pakistan, Pakistan.
Nazish Ashfaq Department of Computer Science, University of the Punjab, Quid-e-Azam Campus, Lahore, 100190, Pakistan, Pakistan.

Keywords:

Automatic number plate detection, Intelligent transportation system, Mobile Net-SSD, Optical character recognition

Abstract

In the realm of intelligent transportation systems, automatic number plate detection has emerged as a crucial research topic due to its wide range of applications, including traffic violation monitoring, support for autonomous vehicles, vehicle speed tracking, automated toll collection, stolen vehicle identification, and overall traffic management. The goal of automatic number plate detection is to accurately identify vehicles based on their number plates. This study proposes a hierarchical approach for detecting number plates. In the initial phase, a lightweight deep learning model, Mobile Net-SSD, is employed to detect number plates. Subsequently, the alphanumeric characters from the detected number plates are extracted using an Optical Character Recognition (OCR) technique. The model is, built on a convolutional neural network, and efficiently uses depth wise and pointwise convolution layers, making it suitable for mobile and embedded systems. Additionally, we introduce a dataset of 30,613 vehicle number plate images to foster further research. Experimental evaluations show that the proposed method achieves 95% accuracy on this dataset, significantly enhancing real-time number plate detection and making it suitable for large-scale implementations in smart cities and intelligent transportation networks.

References

T. S. Muhammad Attique Khan, Muhammad Sharif, Muhammad Younus Javed, Tallha Akram, Mussarat Yasmin, “License number plate recognition system using entropy-based features selection approach with SVM,” Inst. Eng. Technol., vol. 12, no. 2, pp. 200–209, 2018, doi. https.//doi.org/10.1049/iet-ipr.2017.0368.

D. C. Luvizon, B. T. Nassu, and R. Minetto, “Vehicle speed estimation by license plate detection and tracking,” ICASSP, IEEE Int. Conf. Acoust. Speech Signal Process. - Proc., pp. 6563–6567, 2014, doi. 10.1109/ICASSP.2014.6854869.

S. Mahalakshmi and R. Sendhil Kumar, “Smart Toll Collection Using Automatic License Plate Recognition Techniques,” Commun. Comput. Inf. Sci., vol. 805, pp. 34–41, 2018, doi. 10.1007/978-981-13-0755-3_3.

P. Saini, K. Bidhan, and S. Malhotra, “A Detection System for Stolen Vehicles Using Vehicle Attributes with Deep Learning,” Proc. IEEE Int. Conf. Signal Process. Control, vol. 2019-October, pp. 251–254, Oct. 2019, doi. 10.1109/ISPCC48220.2019.8988389.

S. Du, M. Ibrahim, M. Shehata, and W. Badawy, “Automatic license plate recognition (ALPR). A state-of-the-art review,” IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 2, pp. 311–325, 2013, doi. 10.1109/TCSVT.2012.2203741.

S. W. Junqing Tang, Li Wan, Jennifer Schooling, Pengjun Zhao, Jun Chen, “Automatic number plate recognition (ANPR) in smart cities. A systematic review on technological advancements and application cases,” Cities, vol. 129, p. 103833, 2022, doi. https.//doi.org/10.1016/j.cities.2022.103833.

T. T. & L. V. G. Herbert Bay, “SURF. Speeded Up Robust Features,” Comput. Vis. – ECCV 2006, pp. 404–417, 2006, doi. https.//doi.org/10.1007/11744023_32.

T. D. and J. M. R. Girshick, J. Donahue, “Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation,” 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA. Accessed. Mar. 09, 2025. [Online]. Available. https.//ieeexplore.ieee.org/document/6909475

A. Panning, A. K. Al-Hamadi, R. Niese, and B. Michaelis, “Facial expression recognition based on Haar-like feature detection,” Pattern Recognit. Image Anal., vol. 18, no. 3, pp. 447–452, Sep. 2008, doi. 10.1134/S1054661808030139/METRICS.

E. M. Rosario Capparuccia, Renato De Leone, “Integrating support vector machines and neural networks,” Neural Networks, vol. 20, no. 5, pp. 590–597, 2007, doi. https.//doi.org/10.1016/j.neunet.2006.12.003.

Y.-H. Wen, X.-Z., Fang, W., Zheng, “An algorithm based on Haar-like features and improved AdaBoost classifier for vehicle recognition,” ACTA ELECTONICA SINICA. Accessed. Mar. 09, 2025. [Online]. Available. https.//www.researchgate.net/publication/290547288_An_algorithm_based_on_Haar-like_features_and_improved_AdaBoost_classifier_for_vehicle_recognition

M. M. Dehshibi and R. Allahverdi, “Persian V ehicle License Plate Recognition Using Multiclass Adaboost,” Int. J. Comput. Electr. Eng., pp. 355–358, 2012, doi. 10.7763/IJCEE.2012.V4.511.

G. E. H. Alex Krizhevsky, Ilya Sutskever, “ImageNet classification with deep convolutional neural networks,” Commun. ACM, vol. 60, no. 6, pp. 84–90, 2017, doi. https.//doi.org/10.1145/306538.

S. L. Chen, C. Yang, J. W. Ma, F. Chen, and X. C. Yin, “Simultaneous End-to-End Vehicle and License Plate Detection with Multi-Branch Attention Neural Network,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 9, pp. 3686–3695, Sep. 2020, doi. 10.1109/TITS.2019.2931791.

C. H. Lin and Y. Li, “A License Plate Recognition System for Severe Tilt Angles Using Mask R-CNN,” Int. Conf. Adv. Mechatron. Syst. ICAMechS, vol. 2019-August, pp. 229–234, Aug. 2019, doi. 10.1109/ICAMECHS.2019.8861691.

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once. Unified, Real-Time Object Detection,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-December, pp. 779–788, Jun. 2015, doi. 10.48550/arxiv.1506.02640.

R. Agrawal, N. Gupta, and A. S. Jalal, “CACBL-Net. a lightweight skin cancer detection system for portable diagnostic devices using deep learning based channel attention and adaptive class balanced focal loss function,” Multimed. Tools Appl., pp. 1–24, Jun. 2024, doi. 10.1007/S11042-024-19485-1/METRICS.

J. C. M.-S. & E. P. Nicolás Hernández-Díaz, Yersica C. Peñaloza, Y. Yuliana Rios, “A computer vision system for detecting motorcycle violations in pedestrian zones,” Multimed. Tools Appl., 2024, doi. https.//doi.org/10.1007/s11042-024-19356-9.

C.-Y. F. & A. C. B. Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, “SSD. Single Shot MultiBox Detector,” Comput. Vis. – ECCV 2016, pp. 21–37, 2016, doi. https.//doi.org/10.1007/978-3-319-46448-0_2.

X. Zhang, X. Zhou, M. Lin, and J. Sun, “ShuffleNet. An Extremely Efficient Convolutional Neural Network for Mobile Devices,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 6848–6856, Dec. 2018, doi. 10.1109/CVPR.2018.00716.

K. K. Forrest N. Iandola, Song Han, Matthew W. Moskewicz, Khalid Ashraf, William J. Dally, “Squeezenet. Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size,” arXiv.1602.07360, 2016, doi. https.//doi.org/10.48550/arXiv.1602.07360.

A. G. Howard et al., “MobileNets. Efficient Convolutional Neural Networks for Mobile Vision Applications,” Apr. 2017, doi. 10.48550/arxiv.1704.04861.

T. Y. Lin et al., “Microsoft COCO. Common Objects in Context,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 8693 LNCS, no. PART 5, pp. 740–755, 2014, doi. 10.1007/978-3-319-10602-1_48.

Amjad, F., Khan, M. H., Nisar, M. A., Farid, M. S., & Grzegorzek, M. (2021). A comparative study of feature selection approaches for human activity recognition using multimodal sensory data. Sensors, 21(7), 2368.

P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 1, 2001, doi. 10.1109/CVPR.2001.990517.

N. O. Yaseen, S. G. S. Al-Ali, and A. Sengur, “An Efficient Model for Automatic Number Plate Detection using HOG Feature from New North Iraq Vehicle Images Dataset,” 1st Int. Informatics Softw. Eng. Conf. Innov. Technol. Digit. Transform. IISEC 2019 - Proc., Nov. 2019, doi. 10.1109/UBMYK48245.2019.8965573.

A. K. Bachchan, A. Gorai, and P. Gupta, “Automatic License Plate Recognition Using Local Binary Pattern and Histogram Matching,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 10362 LNCS, pp. 22–34, 2017, doi. 10.1007/978-3-319-63312-1_3.

S. M. S. Zahedi Morteza, “License plate recognition system based on SIFT features,” Procedia Comput. Sci., vol. 3, pp. 998–1002, 2011, doi. https.//doi.org/10.1016/j.procs.2010.12.164.

I. S. Ahmad, B. Boufama, P. Habashi, W. Anderson, and T. Elamsy, “Automatic license plate recognition. A comparative study,” 2015 IEEE Int. Symp. Signal Process. Inf. Technol. ISSPIT 2015, pp. 635–640, Jan. 2016, doi. 10.1109/ISSPIT.2015.7394415.

B. Pechiammal and J. A. Renjith, “An efficient approach for automatic license plate recognition system,” ICONSTEM 2017 - Proc. 3rd IEEE Int. Conf. Sci. Technol. Eng. Manag., vol. 2018-January, pp. 121–129, Jun. 2017, doi. 10.1109/ICONSTEM.2017.8261267.

J. W. Zheng, Danian, Zhao, Yannan, “An efficient method of license plate location,” Pattern Recognit. Lett., vol. 26, no. 15, pp. 2431–2438, 2005, doi. https.//doi.org/10.1016/j.patrec.2005.04.014.

Lubna, M. F. Khan, and N. Mufti, “Comparison of various edge detection filters for ANPR,” 2016 6th Int. Conf. Innov. Comput. Technol. INTECH 2016, pp. 306–309, Feb. 2017, doi. 10.1109/INTECH.2016.7845061.

Ashfaq, N., Sadiq, I., Junaid, M., Khan, M. H., & Farid, M. S. Optimizing Human Activity Recognition with Ensemble Deep Learning on Wearable Sensor Data.

Z. Zhao, S. Yang, and X. Ma, “Chinese license plate recognition using a convolutional neural network,” Proc. - 2008 Pacific-Asia Work. Comput. Intell. Ind. Appl. PACIIA 2008, vol. 1, pp. 27–30, 2008, doi. 10.1109/PACIIA.2008.196.

N. P. Ap, T. Vigneshwaran, M. S. Arappradhan, and R. Madhanraj, “Automatic Number Plate Detection in Vehicles using Faster R-CNN,” 2020 Int. Conf. Syst. Comput. Autom. Networking, ICSCAN 2020, Jul. 2020, doi. 10.1109/ICSCAN49426.2020.9262400.

S. Du Chunling Tu, “A hierarchical RCNN for vehicle and vehicle license plate detection and recognition,” Int. J. Electr. Comput. Eng., vol. 12, no. 1, p. 731, 2022, doi. http.//doi.org/10.11591/ijece.v12i1.pp731-737.

Y. L. Jifeng Dai, “R-FCN. Object Detection via Region-based Fully Convolutional Networks,” Neural Information Processing Systems. Accessed. Mar. 09, 2025. [Online]. Available. https.//www.semanticscholar.org/paper/R-FCN%3A-Object-Detection-via-Region-based-Fully-Dai-Li/b724c3f7ff395235b62537203ddeb710f0eb27bb

S. Poojary, M. N. Gaunkar, B. Dessai, M. Ayalli, V. K. N. Pawar, and S. Aswale, “Deep Learning Methods for Automatic Number Plate Recognition System. A Review,” Proc. 3rd Int. Conf. Intell. Eng. Manag. ICIEM 2022, pp. 19–25, 2022, doi. 10.1109/ICIEM54221.2022.9853092.

R. Laroca et al., “A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector,” Proc. Int. Jt. Conf. Neural Networks, vol. 2018-July, Oct. 2018, doi. 10.1109/IJCNN.2018.8489629.

H. Wang, Y. Yu, Y. Cai, X. Chen, L. Chen, and Q. Liu, “A Comparative Study of State-of-the-Art Deep Learning Algorithms for Vehicle Detection,” IEEE Intell. Transp. Syst. Mag., vol. 11, no. 2, pp. 82–95, Jun. 2019, doi. 10.1109/MITS.2019.2903518.

S. Yonetsu, Y. Iwamoto, and Y. W. Chen, “Two-Stage YOLOv2 for Accurate License-Plate Detection in Complex Scenes,” 2019 IEEE Int. Conf. Consum. Electron. ICCE 2019, Mar. 2019, doi. 10.1109/ICCE.2019.8661944.

Z. A. H. and M. N. A. W. Y. Y. Lee, “License Plate Detection Using Convolutional Neural Network–Back to the Basic With Design of Experiments,” IEEE Access, vol. 10, pp. 22577–22585, 2022, doi. 10.1109/ACCESS.2022.3153340.

J. Y. Sung, S. B. Yu, and S. H. P. Korea, “Real-time Automatic License Plate Recognition System using YOLOv4,” 2020 IEEE Int. Conf. Consum. Electron. - Asia, ICCE-Asia 2020, Nov. 2020, doi. 10.1109/ICCE-ASIA49877.2020.9277050.

A. Fathi, B. Moradi, I. Zarei, and A. Shirbandi, “Deep learning based System for automatic motorcycle license plates detection and recognition,” Signal, Image Video Process., vol. 18, no. 12, pp. 8869–8879, Dec. 2024, doi. 10.1007/S11760-024-03514-5/METRICS.

P. C. Janya Sainui, Chaichan Thepporn, “Thai License Plate Recognition using SSD MobileNet and EasyOCR,” IPMV ’24 Proc. 2024 6th Int. Conf. Image Process. Mach. Vis., pp. 36–41, 2024, doi. https.//doi.org/10.1145/3645259.364526.

B. Rohan Reddy, G. Swathi, E. Himesh, and S. Southarrajan, “Encompassing YOLOv8 and EfficientNet B7 for Automatic License Plate Identification,” Proc. - 4th IEEE 2023 Int. Conf. Comput. Commun. Intell. Syst. ICCCIS 2023, pp. 694–700, 2023, doi. 10.1109/ICCCIS60361.2023.10425775.

K. W. Leong, H. Nisar, V. V. Yap, K. H. Yeap, and P. K. Lo, “License Plate Detection Using Deep Learning Object Detection Models,” Proc. 9th Int. Conf. Comput. Commun. Eng. ICCCE 2023, pp. 377–382, 2023, doi. 10.1109/ICCCE58854.2023.10246050.

I. Slimani, A. Zaarane, A. Hamdoun, and I. Atouf, “Vehicle license plate localization and recognition system for intelligent transportation applications,” 2019 6th Int. Conf. Control. Decis. Inf. Technol. CoDIT 2019, pp. 1592–1597, Apr. 2019, doi. 10.1109/CODIT.2019.8820446.

M. Li, T. Sun, and H. Liu, “Image recognition of steel plate based on an improved support vector machine,” 2018 IEEE Int. Conf. Inf. Autom. ICIA 2018, pp. 1411–1415, Aug. 2018, doi. 10.1109/ICINFA.2018.8812381.

M. H. Khan, K. Shirahama, M. S. Farid, and M. Grzegorzek, “Multiple human detection in depth images,” 2016 IEEE 18th Int. Work. Multimed. Signal Process. MMSP 2016, Jan. 2017, doi. 10.1109/MMSP.2016.7813385.

T. Naito, T. Tsukada, K. Yamada, K. Kozuka, and S. Yamamoto, “Robust license-plate recognition method for passing vehicles under outside environment,” IEEE Trans. Veh. Technol., vol. 49, no. 6, pp. 2309–2319, Nov. 2000, doi. 10.1109/25.901900.

X. Lu, X. Ling, and W. Huang, “Vehicle license plate character recognition,” Proc. 2003 Int. Conf. Neural Networks Signal Process. ICNNSP’03, vol. 2, pp. 1066–1069, 2003, doi. 10.1109/ICNNSP.2003.1281053.

M. G. Muhammad Hassan Khan , Julien Helsper, Muhammad Shahid Farid, “A computer vision-based system for monitoring Vojta therapy,” Int. J. Med. Inform., vol. 113, pp. 85–95, 2018, doi. https.//doi.org/10.1016/j.ijmedinf.2018.02.010.

S. L. Chang, L. S. Chen, Y. C. Chung, and S. W. Chen, “Automatic License Plate Recognition,” IEEE Trans. Intell. Transp. Syst., vol. 5, no. 1, pp. 42–53, Mar. 2004, doi. 10.1109/TITS.2004.825086.

Q. H. Jiao, Jianbin, Ye, Qixiang, “A configurable method for multi-style license plate recognition,” Pattern Recognit., vol. 42, no. 3, pp. 358–369, 2009, doi. https.//doi.org/10.1016/j.patcog.2008.08.016.

K. Vijayalakshmi, M. Dhanamalar, V. A. Lepakshi, and S. Jamtsho, “Smart Checkpoint Management System for Automatic Number Plate Recognition in Bhutan Vehicles Using OCR Technique,” SN Comput. Sci., vol. 5, no. 5, pp. 1–8, Jun. 2024, doi. 10.1007/S42979-024-02905-2/METRICS.

Ashfaq, N., Khan, M. H., & Nisar, M. A. (2024). Identification of optimal data augmentation techniques for multimodal time-series sensory data. A framework. Information, 15(6), 343.

M. Abadi et al., “TensorFlow. Large-Scale Machine Learning on Heterogeneous Distributed Systems,” Mar. 2016, Accessed. Jun. 16, 2024. [Online]. Available. https.//arxiv.org/abs/1603.04467v2

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-December, pp. 2818–2826, Dec. 2016, doi. 10.1109/CVPR.2016.308.

H. Seya and D. Shiroi, “Lecture 6.5‐rmsprop. Divide the Gradient by a Running Average of Its Recent Magnitude,” COURSERA Neural Networks Mach. Learn., vol. 4, no. 2, pp. 26-, Apr. 2012, doi. 10.1111/GEAN.12283.

Batool, S., Khan, M. H., & Farid, M. S. (2024). An ensemble deep learning model for human activity analysis using wearable sensory data. Applied Soft Computing, 111599.

N. Klingler, “Automatic Number Plate Recognition (ANPR),” viso.ai. Accessed. Mar. 09, 2025. [Online]. Available. https.//viso.ai/computer-vision/automatic-number-plate-recognition-anpr/

V. . Kaur, E.K., Banga, “Number plate recognition using ocr technique,” Int. J. Res. Eng. Technol., vol. 2, no. 9, 2013, [Online]. Available. https.//ijret.org/volumes/2013v02/i09/IJRET20130209042.pdf

M. T. Qadri and M. Asif, “Automatic number plate recognition system for vehicle identification using optical character recognition,” 2009 Int. Conf. Educ. Technol. Comput. ICETC 2009, pp. 335–338, 2009, doi. 10.1109/ICETC.2009.54.

Y. Baek, B. Lee, D. Han, S. Yun, and H. Lee, “Character region awareness for text detection,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2019-June, pp. 9357–9366, Jun. 2019, doi. 10.1109/CVPR.2019.00959.

T. Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, “Focal Loss for Dense Object Detection,” Proc. IEEE Int. Conf. Comput. Vis., vol. 2017-October, pp. 2999–3007, Dec. 2017, doi. 10.1109/ICCV.2017.324.

A. D. P. Tote, Anuj S., Pardeshi, Sujit S., “Automatic number plate detection using TensorFlow in Indian scenario. An optical character recognition approach,” Mater. Today Proc., vol. 72, no. 3, pp. 1073–1078, 2023, doi. https.//doi.org/10.1016/j.matpr.2022.09.165.