Sight Mate – An Intelligent Visual Assistance System for Visually Impaired Individuals Using Computer Vision and Audio Feedback

Raniya Raza; Muhammad Saleem Vighio

Authors

Raniya Raza Department of Computer Science, Quaid-e-Awam University of Engineering, Science, & Technology, Nawabshah
Muhammad Saleem Vighio Department of Computer Science, Quaid-e-Awam University of Engineering, Science, & Technology, Nawabshah

Keywords:

Edge AI, Computer Vision, Object Detection, Navigation Assistance, Visually Impaired, Offline Processing, Real-time Deployment

Abstract

Globally, approximately 285 million people are visually impaired, with 39 million of them being blind. However, in developing countries, assistive technology is less effective due to unreliable network connectivity, as well as high cost and limited accessibility. In this paper, we introduce SightMate, a hybrid visual assistant that employs an edge AI architecture in offline mode, enabling the system to be aware of its surroundings and navigate independently. It utilizes computer vision for image processing and classification and generates notifications to ensure system functionality in the absence of network connectivity. The system supports multimodal accessibility with features such as voice feedback, notifications in hand, and auditory notifications. SightMate includes light-weight models such as YOLOv3-Tiny, Haar Cascade, Random Forest, Support Vector Machine, Gaussian Mixture Model, and Convolutional Neural Networks. From the experiments, the system achieved object detection accuracy between 86% and 93% with low latency, making it suitable for real-time deployment in resource-constrained environments.

References

Ruxandra Tapu, Bogdan Mocanu, Titus Zaharia, “Wearable assistive devices for visually impaired: A state of the art survey,” Pattern Recognit. Lett., vol. 137, pp. 37–52, 2020, [Online]. Available: https://www.sciencedirect.com/science/article/abs/pii/S0167865518308602

Faiyaz Ahmad, Mohd Zeeshan Ansari, “Virtual Assistance System for Visually Impaired people using Deep Learning,” Procedia Comput. Sci., vol. 258, pp. 67–75, 2025, doi: https://doi.org/10.1016/j.procs.2025.04.196.

Cheng Su, Litao Zhu, “Nav-YOLO: A Lightweight and Efficient Object Detection Model for Real-Time Indoor Navigation on Mobile Platforms,” ISPRS Int. J. Geo-Inf, vol. 14, no. 9, p. 364, 2025, doi: https://doi.org/10.3390/ijgi14090364.

Omar Kanaan Taha Alsultan, Mohammad Tarik Mohammad, “A Deep Learning-Based Assistive System for the Visually Impaired Using YOLO-V7,” IIETA J., 2023, [Online]. Available: https://www.iieta.org/journals/ria/paper/10.18280/ria.370409

“Seeing AI: New Technology Research to Support the Blind and Visually Impaired Community - Microsoft Accessibility Blog.” Accessed: Feb. 07, 2026. [Online]. Available: https://blogs.microsoft.com/accessibility/seeing-ai/

Alexandru Lavric, Cătălin Beguni, “A Comprehensive Survey on Emerging Assistive Technologies for Visually Impaired Persons: Lighting the Path with Visible Light Communications and Artificial Intelligence Innovations,” Sensors, vol. 24, no. 15, p. 4834, 2024, [Online]. Available: https://www.mdpi.com/1424-8220/24/15/4834

Wei Wang, Bin Jing, “YOLO-OD: Obstacle Detection for Visually Impaired Navigation Assistance,” Sensors, vol. 24, no. 23, p. 7621, 2024, doi: 10.3390/s24237621.

Lalitha Manirajee, Siti Qatrunnada Hanis Shariff, “Assistive Technology for Visually Impaired Individuals: A Systematic Literature Review (SLR),” Int. J. Acad. Res. Bus. Soc. Sci., vol. 14, no. 2, 2024, [Online]. Available: https://www.researchgate.net/publication/378254054_Assistive_Technology_for_Visually_Impaired_Individuals_A_Systematic_Literature_Review_SLR

Xiaochen Zhang, Xiaoyu Huang, “Advancements in Smart Wearable Mobility Aids for Visual Impairments: A Bibliometric Narrative Review,” Sensors, vol. 24, no. 24, p. 7986, 2024, doi: https://doi.org/10.3390/s24247986.

Mohammad Moeen Valipoor, Angélica de Antonio, “Recent trends in computer vision-driven scene understanding for VI/blind users: a systematic mapping,” Univers. Access Inf. Soc., vol. 22, pp. 983–1005, 2023, [Online]. Available: https://link.springer.com/article/10.1007/s10209-022-00868-w

Marwa Obayya, Fahd N. Al-Wesabi, Wafi Bedewi & Menwa Alshammeri, “An intelligent framework for visually impaired people through indoor object Detection-Based assistive system using YOLO with recurrent neural networks,” Sci. Rep., vol. 15, 2025, [Online]. Available: http://nature.com/articles/s41598-025-27603-8