Real-Time Hand Gesture Recognition with FMCW Radar and 4D Feature Extraction for Interactive HCI

Danish Ahmed; Meer Ahmad; Muhammad Ali Masood; Muhammad Junaid Hassan

Authors

Danish Ahmed Department of Computer Science, National University of Technology, Islamabad 44000, Pakistan https://orcid.org/0009-0004-4149-8877
Meer Ahmad Department of Computer Science, National University of Technology, Islamabad 44000, Pakistan https://orcid.org/0009-0001-6913-6091
Muhammad Ali Masood Department of Computer Science, National University of Technology, Islamabad 44000, Pakistan https://orcid.org/0009-0006-6114-8676
Muhammad Junaid Hassan Department of Computer Science, National University of Technology, Islamabad 44000, Pakistan https://orcid.org/0009-0000-7521-9275

Keywords:

mmWave Frequency-Modulated Continuous Wave (FMCW) Radar, Hand Gesture Recognition, Human-Computer Interaction, 4D Feature Extraction, Bidirectional Long Short-Term Memory (Bi-LSTM)

Abstract

Hand gesture recognition enables intuitive Human-Computer Interaction (HCI), however, traditional camera-based approaches suffer from privacy concerns, sensitivity to illumination variation, and occlusion. To address these limitations, this research utilizes a low-cost 24 GHz mmWave FMCW radar (RD-03D) to develop a privacy preserving and illumination robust recognition framework. The methodology involves extracting a comprehensive 4D feature set comprising Range, Doppler, Angle (Azimuth), and Time derived directly from the on-chip Fast Fourier Transform (FFT) processing of the raw radar signal. A balanced dataset of ten dynamic hand gestures was collected from six subjects, totaling 14,400 samples, in a standard lab environment. This spatio-temporal data was used to train a Bidirectional Long Short-Term Memory (BiLSTM) classification model. The trained model achieved a robust classification accuracy of 98.43% on the unseen validation dataset with a 95% confidence interval of ±0.65%. Detailed statistical reporting reveals a macro averaged precision of 0.989, recall of 0.991, and an F1 score of 0.990, demonstrating high discriminative power across all gesture classes. The system is validated for real-time recognition with a remarkably low inference latency of less than 100 ms, implemented on a host system utilizing an NVIDIA GeForce GTX 1660 SUPER for efficient processing. The results establish that the combination of FMCW radar and a specialized deep learning architecture offers a highly accurate, reliable, and privacy friendly alternative to vision-based gesture recognition interfaces for interactive HCI applications.

References

M Multi-Hand, Academic Editors, “Multi-Hand Gesture Recognition Using Automotive FMCW Radar Sensor,” Remote Sens., vol. 14, no. 10, p. 2374, 2022, doi: https://doi.org/10.3390/rs14102374.

Yu Chiao Jhaung, Yu Ming Lin, “Implementing a Hand Gesture Recognition System Based on Range-Doppler Map,” Sensors, vol. 22, no. 11, p. 4260, 2022, doi: https://doi.org/10.3390/s22114260.

J. -W. Choi, C. -W. Park and J. -H. Kim, “FMCW Radar-Based Real-Time Hand Gesture Recognition System Capable of Out-of-Distribution Detection,” IEEE Access, vol. 10, pp. 87425–87434, 2022, doi: 10.1109/ACCESS.2022.3200757.

X. Dong, Z. Zhao, Y. Wang, T. Zeng, J. Wang, and Y. Sui, “FMCW Radar-Based Hand Gesture Recognition Using Spatiotemporal Deformable and Context-Aware Convolutional 5-D Feature Representation,” IEEE Trans. Geosci. Remote Sens., vol. 60, 2022, doi: 10.1109/TGRS.2021.3122332.

L. Gan, Y. Liu, Y. Li, R. Zhang, L. Huang, and C. Shi, “Gesture Recognition System Using 24 GHz FMCW Radar Sensor Realized on Real-Time Edge Computing Platform,” IEEE Sens. J., vol. 22, no. 9, pp. 8904–8914, May 2022, doi: 10.1109/JSEN.2022.3163449.

Y. Zhao, V. Sark, M. Krstic, and E. Grass, “Low-Complexity Gesture Recognition based on FMCW Radar,” 2024 21st Eur. Radar Conf. EuRAD 2024, pp. 388–391, 2024, doi: 10.23919/EuRAD61604.2024.10734900.

K. Alirezazad and L. Maurer, “FMCW Radar-Based Hand Gesture Recognition Using Dual-Stream CNN-GRU Model,” 2022 24th Int. Microw. Radar Conf. MIKON 2022, 2022, doi: 10.23919/mikon54314.2022.9924984.

Haili Wang, Muye Zhang, “Real-Time Hand Gesture Recognition in Clinical Settings: A Low-Power FMCW Radar Integrated Sensor System with Multiple Feature Fusion,” Sensors, vol. 25, no. 13, p. 4169, 2025, doi: https://doi.org/10.3390/s25134169.

A. Fusco, Z. Amir Zaman, S. Hazra, L. Servadei and R. Wille, “Enhancing FMCW Radar Gesture Classification With Physically Interpretable Data Augmentation,” IEEE Access, vol. 13, pp. 60556–60569, 2025, doi: 10.1109/ACCESS.2025.3556565.

Yuhang Shi, Lihong Qiao, “Semi-Supervised FMCW Radar Hand Gesture Recognition via Pseudo-Label Consistency Learning,” Remote Sens., vol. 16, no. 3, p. 2267, 2024, doi: https://doi.org/10.3390/rs16132267.

Y. Zhao, V. Sark, M. Krstic, and E. Grass, “High-Efficiency Gesture Recognition Using Multiple mmWave FMCW RADARs,” 2024 Int. Symp. Networks, Comput. Commun. ISNCC 2024, 2024, doi: 10.1109/ISNCC62547.2024.10758972.

J. T. Yu, L. Yen, and P. H. Tseng, “MmWave Radar-based Hand Gesture Recognition using Range-Angle Image,” IEEE Veh. Technol. Conf., vol. 2020-May, May 2020, doi: 10.1109/VTC2020-Spring48590.2020.9128573.

W. Jiang, Y. Ren, Y. Liu, Z. Wang, and X. Wang, “Recognition of dynamic hand gesture based on mm-wave FMCW radar micro-Doppler signatures,” ICASSP, IEEE Int. Conf. Acoust. Speech Signal Process. - Proc., vol. 2021-June, pp. 4905–4909, 2021, doi: 10.1109/ICASSP39728.2021.9414837.

Y. Zhao, V. Sark, M. Krstic, and E. Grass, “Synthetic Training Data Generator for Hand Gesture Recognition Based on FMCW RADAR,” Proc. Int. Radar Symp., vol. 2022-September, pp. 463–468, 2022, doi: 10.23919/irs54158.2022.9904997.

Zhangjin Xiong, Kaixue Ma, “Hand gesture recognition based on micro‐Doppler radar using graph neural network,” Electron. Lett., vol. 60, no. 3, 2024, doi: 10.1049/ell2.13100.

“Dual-Stream BiLSTM–Transformer Architecture for Real-Time Two-Handed Dynamic Sign Language Gesture Recognition.” Accessed: May 05, 2026. [Online]. Available: https://www.mdpi.com/2076-3417/16/6/2912

“Dynamic Gesture Recognition Based on FMCW Millimeter Wave Radar: Review of Methodologies and Results - PubMed.” Accessed: May 05, 2026. [Online]. Available: https://pubmed.ncbi.nlm.nih.gov/37687932/

A. Ślesicka and A. Kawalec, “Real-Time Hand Gesture Recognition for IoT Devices Using FMCW mmWave Radar and Continuous Wavelet Transform,” Electron. 2026, Vol. 15, Page 250, vol. 15, no. 2, p. 250, Jan. 2026, doi: 10.3390/ELECTRONICS15020250.

Y. Sun, T. Fei, X. Li, A. Warnecke, E. Warsitz, and N. Pohl, “Real-Time Radar-Based Gesture Detection and Recognition Built in an Edge-Computing Platform,” IEEE Sens. J., vol. 20, no. 18, pp. 10706–10716, Sep. 2020, doi: 10.1109/JSEN.2020.2994292.

Reda El Hail, Pouya Mehrjouseresht, “Radar-Based Human Activity Recognition: A Study on Cross-Environment Robustness,” Electronics, vol. 14, no. 5, p. 875, 2025, doi: https://doi.org/10.3390/electronics14050875.

B. Van Amsterdam, M. J. Clarkson, and D. Stoyanov, “Gesture Recognition in Robotic Surgery: A Review,” IEEE Trans. Biomed. Eng., vol. 68, no. 6, pp. 2021–2035, Jun. 2021, doi: 10.1109/TBME.2021.3054828.

D. Rodrigues and C. Li, “Hand Gesture Recognition Using FMCW Radar in Multi-Person Scenario,” 2021 IEEE Top. Conf. Wirel. Sensors Sens. Networks, WiSNeT 2021, pp. 50–52, Jan. 2021, doi: 10.1109/WISNET51848.2021.9413794.

“Micro Gesture Recognition with Multi-Dimensional Feature Fusion and CQ-MobileNetV3 Using FMCW Radar - PubMed.” Accessed: May 05, 2026. [Online]. Available: https://pubmed.ncbi.nlm.nih.gov/41305159/

“(PDF) Research Progress on Technologies and Applications of Human-Computer Interaction Gestures Based on Vision and Sensors.” Accessed: May 05, 2026. [Online]. Available: https://www.researchgate.net/publication/402496997_Research_Progress_on_Technologies_and_Applications_of_Human-Computer_Interaction_Gestures_Based_on_Vision_and_Sensors

A. Mohan, H. K. Meena, M. Wajid, and A. Srivastava, “Real-Time Dynamic Hand Gesture Recognition Using mmWave Radar on FPGA,” IEEE Embed. Syst. Lett., 2025, doi: 10.1109/LES.2025.3634571.

A. Chopde, S. Joshi, S. Surve, and S. Chavanke, “FPGA based real-time hand gesture recognition system,” 2024 15th Int. Conf. Comput. Commun. Netw. Technol. ICCCNT 2024, 2024, doi: 10.1109/ICCCNT61001.2024.10724416.