An End-to-End Deep Learning Pipeline for Child Detection and Activity Recognition in Surveillance Systems

Abstract

Children in developing countries often face significant safety risks such as domestic accidents, unsafe environments, and abusive social exposure. The current surveillance systems are able to identify children but, in most cases, they fail to comprehend their activities over time. This creates a critical gap in real-time child safety monitoring. This paper addresses that gap by developing an end-to-end video understanding pipeline for child detection in video frames and recognizes their activities accurately over time. A custom child detection dataset containing 19,890 annotated images was developed and split into 16,107 training and 3,783 validation samples. Several state-of-the-art object detection models, including YOLOv8s, YOLO26s, and RT-DETR-L, were trained and compared based on their performance metrics. DeepSORT is used to preserve the identities of children detected across frames. For activity recognition, a custom video dataset comprising approximately 1,960 clips across 47 activity classes was created. Four deep learning architectures (I3D, ResNet3D, ViViT, and VideoMAE) were trained using standardized configurations with 16-frame clips, 224×224 input resolution, Adam optimization, and cross-entropy loss. YOLO26s achieved the best detection performance with Precision = 97.17%, Recall = 96.18%, mAP@0.50 = 98.51%, and mAP@0.50–0.95 = 87.80%. For activity recognition, VideoMAE outperformed competing models on 448 clips with test accuracy of 86.38% and Macro F1-score of 90.49%, significantly surpassing ViViT (79.24%), ResNet3D (51.34%), and I3D (9.60%). The proposed method provides a scalable and practical solution by combining high-quality spatial localization with temporal activity understanding, enabling real-time monitoring of child activities.