AI Vision for Health Care: AI-Powered Smart Glasses for Visually Impaired Individual

Abstract

Visual Impairment can be counted among the sensory deficiencies that severely debilitate affected individuals and restrict their capabilities at performing basic everyday tasks like recognizing currency notes, managing their medicines and navigating spatially independent of any aid. In this context, there is still no effective solution available that covers all these requirements under the same umbrella. Thus, this study proposes an innovative concept of an AI-assisted vision help system with unique features. These features incorporate currency recognition, assistance in medicine consumption, and navigation. The system is based on a modular approach where YOLOv8, an advanced object detection technique, has been used for developing different assistive modules with custom annotations for training and optimization through Google Colab using GPUs. Each module was tested with 76% data in the training phase, 15% for validation, and 9% for testing purposes. Acquisition of frames was achieved by OpenCV, which is then processed by the YOLO algorithm, and detected objects are converted into relevant speech using TTS technique. The output speech is transmitted through earphones or bone conduction headphones. Currency recognition module has produced Macro precision scores of 96-98% and mAP@50 of 98.2% after 65 training epochs. Currency detection achieved a performance score of F1-score of 0.96 at confidence level 0.333, Precision of 1.00, and Recall of 1.00. The Medical prescription assistance module has shown Macro precision score of 99.2%, Macro recall of 98.0% and mAP@50 of 99.3% after 50 training epochs. Medical Prescription Detection Module achieved a performance score of F1-score of 0.99 at confidence 0.657, Precision of 1.00, and Recall of 1.00 while navigation assistance module recorded Macro precision of 99.0% and mAP@50 of 75.9% after training 50 epochs. Navigation Module showed an F1-score of 0.73 at confidence level 0.504 Confusion matrixes and Precision Recall Curve results have validated robustness in detecting objects in various lighting conditions along with occluded environments. The total number of images analyzed were 13087, which included all the above modules. It was observed that the inference time required for each frame was 28 ms. comparatively, it was found that YOLOv8 had performed 3.7% better than YOLOv5. All the modules can be run independently by sharing a common backbone detection algorithm. This will save computing power for processing other tasks with ease without any noticeable delay. The suggested technique is scalable and cost-effective enough to offer a practical solution for improving the lives of visually impaired individuals.