Deep Learning (DL) for Advanced Persistent Threat (APT) Detection in Cybersecurity
DOI:
https://doi.org/10.33411/IJIST/1764Keywords:
Advanced Persistent Threats (APTs), Cybersecurity, Machine Learning (ML), Deep Learning (DL), Reinforcement Learning (RL), APT Attack Life CycleAbstract
Advanced Persistent Threat (APT) is one of the most dangerous types of cyberattacks. These attacks are highly stealthy, long-term, and multi-stage in nature, typically targeting critical infrastructure, businesses, and government organizations. Conventional security solutions and classical machine learning approaches often struggle to detect such threats due to their ability to evade detection over extended periods. Recently, deep learning methods have demonstrated strong potential for APT detection by learning complex temporal and behavioral patterns from large-scale security data. This study presents a comprehensive review and comparative analysis of deep learning–based APT detection techniques reported between 2020 and 2025. The analysis covers the APT attack life cycle, taxonomy of attack types, commonly used benchmark datasets, and the performance of state-of-the-art deep learning architectures applied in modern cybersecurity systems. A quantitative synthesis of the reviewed literature shows that CNN- and LSTM-based baseline models typically achieve detection accuracies between 88% and 93%, with F1-scores ranging from 0.87 to 0.91. In comparison, more recent architectures such as transformer-based models and graph neural networks report mean detection accuracies of 94%–98%, F1-scores between 0.93 and 0.97, and recall rates above 0.92 across multiple benchmark datasets. These models demonstrate performance improvements of approximately 4%–7% in detection accuracy and 5%–8% in F1-score compared with CNN/LSTM baselines, while also achieving relative false-positive reductions in several experimental evaluations. Despite these advancements, important challenges remain, including limited availability of high-quality labeled datasets, difficulties in model interpretability, and constraints related to real-time deployment in operational environments. The study concludes with future research directions emphasizing multi-modal data fusion, explainable AI techniques, online learning frameworks, privacy-preserving detection mechanisms, and scalable deployment strategies to advance robust and practical APT detection systems.
References
Guangwu Hu, Maoqi Sun, “A High-Accuracy Advanced Persistent Threat Detection Model: Integrating Convolutional Neural Networks with Kepler-Optimized Bidirectional Gated Recurrent Units,” Electronics, vol. 14, no. 9, p. 1772, 2025, doi: https://doi.org/10.3390/electronics14091772.
Noor Hazlina Abdul Mutalib, Aznul Qalid Md Sabri, Ainuddin Wahid Abdul Wahab, Erma Rahayu Mohd Faizal Abdullah, “Explainable deep learning approach for advanced persistent threats (APTs) detection in cybersecurity: a review,” Artif. Intell. Rev., vol. 57, no. 297, 2024, [Online]. Available: https://link.springer.com/article/10.1007/s10462-024-10890-4
Duraid Thamer Salim, Manmeet Mahinderjit Singh, Pantea Keikhosrokiani, “A systematic literature review for APT detection and Effective Cyber Situational Awareness (ECSA) conceptual model,” Heliyon, vol. 9, no. 7, p. e17156, 2023, doi: https://doi.org/10.1016/j.heliyon.2023.e17156.
Shakhzod Yuldoshkhujaev, Mijin Jeon, Doowon Kim, Nick Nikiforakis, Hyungjoon Koo, “A Decade-long Landscape of Advanced Persistent Threats: Longitudinal Analysis and Global Trends,” arXiv:2509.07457, 2026, [Online]. Available: https://arxiv.org/abs/2509.07457
Almuthanna Alageel, Sergio Maffeis, Imperial College London, “Investigation of Advanced Persistent Threats Network-based Tactics, Techniques and Procedures,” arXiv:2502.08830, 2025, [Online]. Available: https://arxiv.org/abs/2502.08830
“Unicorn: Runtime Provenance-Based Detector for Advanced Persistent Threats - NDSS Symposium.” Accessed: Mar. 01, 2026. [Online]. Available: https://www.ndss-symposium.org/ndss-paper/unicorn-runtime-provenance-based-detector-for-advanced-persistent-threats/
“(PDF) The Solar Winds Cyber-Attack, the Federal and Private Sector Response, and the Recommendations and Lessons Learned.” Accessed: Mar. 30, 2026. [Online]. Available: https://www.researchgate.net/publication/365186053_The_Solar_Winds_
Cyber-Attack_the_Federal_and_Private_Sector_Response_and_the_
Recommendations _and_Lessons_Learned
Abdullateef Barakat, “Enhancing global cybersecurity: Strategies for mitigating advanced persistent threats (APTS) in a borderless digital landscape,” World J. Adv. Res. Rev., vol. 25, no. 3, pp. 829–846, Mar. 2025, doi: 10.30574/wjarr.2025.25.3.0815.
Pedro Brandao, “Advanced Persistent Threat Detection Through Multi-Layered Machine Learning: The MLADA Framework,” Preprints, 2025, [Online]. Available: https://www.preprints.org/manuscript/202507.0748
P. Dixit and S. Silakari, “Deep Learning Algorithms for Cybersecurity Applications: A Technological and Status Review,” Comput. Sci. Rev., vol. 39, Feb. 2021, doi: 10.1016/J.COSREV.2020.100317.
Abdullah Mujawib Alashjaee, “Deep learning for network security: an Attention-CNN-LSTM model for accurate intrusion detection,” Sci. Rep., vol. 15, 2025, [Online]. Available: https://www.nature.com/articles/s41598-025-07706-y
Iqbal H. Sarker, Helge Janicke, Ahmad Mohsin, Asif Gill, Leandros Maglaras, “Explainable AI for cybersecurity automation, intelligence and trustworthiness in digital twin: Methods, taxonomy, challenges and prospects,” ICT Express, vol. 10, no. 4, pp. 935–958, 2024, [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2405959524000572
G. Nalinipriya, S. Rama Sree, K. Radhika, E. Laxmi Lydia, Faten Khalid Karim, Mohamad Khairi Ishak, “Leveraging explainable artificial intelligence for early detection and mitigation of cyber threat in large-scale network environments,” Sci. Rep., vol. 15, 2025, [Online]. Available: https://www.nature.com/articles/s41598-025-08597-9
“NSFOCUS Monthly APT Insights - June 2025 - NSFOCUS, Inc., a global network and cyber security leader, protects enterprises and carriers from advanced cyber attacks.” Accessed: Mar. 01, 2026. [Online]. Available: https://nsfocusglobal.com/nsfocus-monthly-apt-insights-june/
Abdullah Said AL-Aamri, Rawad Abdulghafor, “Machine Learning for APT Detection,” Sustainabilitytain, vol. 15, no. 18, p. 13820, 2023, doi: https://doi.org/10.3390/su151813820.
Yang Hu, Roland Sandt & Robert Spatschek, “Practical feature filter strategy to machine learning for small datasets in chemistry,” Sci. Rep., 2024, [Online]. Available: https://www.nature.com/articles/s41598-024-71342-1
Mingqi Lv, HongZhe Gao, Xuebo Qiu, Tieming Chen, Tiantian Zhu, Jinyin Chen, Shouling Ji, “TREC: APT Tactic / Technique Recognition via Few-Shot Provenance Subgraph Learning,” arXiv:2402.15147, 2024, [Online]. Available: https://arxiv.org/abs/2402.15147
Animesh Singh Basnet, Mohamed Chahine Ghanem, Dipo Dunsin, Wiktor Sowinski-Mydlarz, “Advanced Persistent Threats (APT) Attribution Using Deep Reinforcement Learning,” arXiv:2410.11463, 2025, doi: https://doi.org/10.48550/arXiv.2410.11463.
Anh Tuan Le, Gregory Epiphaniou, “Automated APT Defense Using Reinforcement Learning and Attack Graph Risk-based Situation Awareness,” Auton. 2024 - Proc. Work. Auton. Cybersecurity, Co-Located with CCS 2024, 2024, [Online]. Available: https://dl.acm.org/doi/10.1145/3689933.3690834
Ameer A. Ghani, Suad A. Alasadi, “A Deep Learning Algorithm to Cybersecurity: Enhancing Intrusion Detection with a Hybrid GRU and BiLSTM Model,” Eng. Technol. Appl. Sci. Res., vol. 15, no. 3, pp. 23605–23612, 2025, doi: 10.48084/etasr.10666.
Fargana J. Abdullayeva, “Advanced Persistent Threat attack detection method in cloud computing based on autoencoder and softmax regression algorithm,” Array, vol. 10, p. 100067, 2021, doi: https://doi.org/10.1016/j.array.2021.100067.
Mohammad Mamun, Kevin Shi, “DeepTaskAPT: Insider APT detection using Task-tree based Deep Learning,” arXiv:2108.13989, 2021, [Online]. Available: https://arxiv.org/abs/2108.13989
Zefeng He, Diego Davila, “Machine Learning for Cybersecurity: A Survey of Applications, Adversarial Challenges, and Future Research Directions,” Electronics, vol. 14, no. 23, p. 4563, 2025, doi: 10.3390/electronics14234563.
Santiago Quintero-Bonilla, Angel Martín del Rey, “A New Proposal on the Advanced Persistent Threat: A Survey,” Appl. Sci., vol. 10, no. 11, p. 3874, 2020, doi: https://doi.org/10.3390/app10113874.
Anton Konev, Alexander Shelupanov, “A Survey on Threat-Modeling Techniques: Protected Objects and Classification of Threats,” Symmetry (Basel)., vol. 14, no. 3, p. 549, 2022, doi: https://doi.org/10.3390/sym14030549.
Jiajun Zhou, Jiacheng Yao, Xuanze Chen, Shanqing Yu, Qi Xuan, Xiaoniu Yang, “Lateral Movement Detection via Time-aware Subgraph Classification on Authentication Logs,” arXiv:2411.10279, 2024, [Online]. Available: https://arxiv.org/abs/2411.10279
T. Zhu et al., “APTSHIELD: A Stable, Efficient and Real-Time APT Detection System for Linux Hosts,” IEEE Trans. Dependable Secur. Comput., vol. 20, no. 6, pp. 5247–5264, Nov. 2023, doi: 10.1109/TDSC.2023.3243667.
Nur Ilzam Che Mat , Norziana Jamil , Yunus Yusoff , Miss Laiha Mat Kiah, “A systematic literature review on advanced persistent threat behaviors and its detection strategy,” J. Cybersecurity, vol. 10, no. 1, 2024, doi: https://doi.org/10.1093/cybsec/tyad023.
Bushra Sabir, Faheem Ullah, M. Ali Babar, Raj Gaire, “Machine Learning for Detecting Data Exfiltration: A Review,” arXiv:2012.09344, 2021, [Online]. Available: https://arxiv.org/abs/2012.09344
Qi Liu, Muhammad Shoaib, Mati Ur Rehman, Kaibin Bao, Veit Hagenmeyer, Wajih Ul Hassan, “Accurate and Scalable Detection and Investigation of Cyber Persistence Threats,” arXiv:2407.18832, 2024, [Online]. Available: https://arxiv.org/abs/2407.18832
Md Rayhanur Rahman, Setu Kumar Basak, Rezvan Mahdavi Hezaveh, Laurie Williams, “SoK: An empirical investigation of malware techniques in advanced persistent threat attacks,” Comput. Secur., vol. 157, p. 104618, 2025, doi: https://doi.org/10.1016/j.cose.2025.104618.
Muhammad Shofian Tsauri, “Human Vulnerabilities to Social Engineering Attacks: A Systematic Literature Review for Building a Human Firewall,” J. Appl. Informatics Comput., vol. 9, no. 4, pp. 1127–1136, 2025, doi: 10.30871/jaic.v9i4.9585.
Suleiman Y. Yerima, Mohammed K. Alzaylaee, “High Accuracy Phishing Detection Based on Convolutional Neural Networks,” arXiv:2004.03960, 2020, [Online]. Available: https://arxiv.org/abs/2004.03960
Santosh Kumar Birthriya, Priyanka Ahlawat, Ankit Kumar Jain, “Detection and prevention of spear phishing attacks: A comprehensive survey,” Comput. Secur., vol. 151, p. 104317, 2025, doi: https://doi.org/10.1016/j.cose.2025.104317.
Singamaneni Krishnapriya, Sukhvinder Singh, “A Comprehensive Survey on Advanced Persistent Threat (APT) Detection Techniques,” Comput. Mater. Contin., vol. 80, no. 2, pp. 2675–2719, 2024, doi: https://doi.org/10.32604/cmc.2024.052447.
Suresh Kumar Srinivasan, Sudalaimuthu Thalavaipillai, “Kernel rootkit detection multi class on deep learning techniques,” Bull. Electr. Eng. Informatics, vol. 13, no. 3, pp. 2000–2008, 2024, doi: 10.11591/eei.v13i3.6802.
Abdullah Al Mamun, Harith Al-Sahaf, “Detection of advanced persistent threat: A genetic programming approach,” Appl. Soft Comput., vol. 167, p. 112447, 2024, doi: https://doi.org/10.1016/j.asoc.2024.112447.
“Synthetic APT Dataset.” Accessed: Mar. 30, 2026. [Online]. Available: https://www.emergentmind.com/topics/synthetic-apt-dataset
Marcos Luengo Viñuela, Jesús Ángel Román-Gallego, “Detection of APTs by Machine Learning: A Performance Comparison,” Expert Syst., vol. 43, no. 1, 2025, doi: 10.1111/exsy.70181.
J. Liu et al., “A New Realistic Benchmark for Advanced Persistent Threats in Network Traffic,” IEEE Netw. Lett., vol. 4, no. 3, pp. 162–166, Sep. 2022, doi: 10.1109/LNET.2022.3185553.
Q. Ma and N. Rastogi, “DANTE: Predicting insider threat using LSTM on system logs,” Proc. - 2020 IEEE 19th Int. Conf. Trust. Secur. Priv. Comput. Commun. Trust. 2020, pp. 1151–1156, Dec. 2020, doi: 10.1109/TrustCom50675.2020.00153.
U. Sakthivelu, C. N.S. Vinoth Kumar, “Advanced Persistent Threat Detection and Mitigation Using Machine Learning Model,” Intell. Autom. Soft Comput., vol. 36, no. 3, pp. 3691–3707, 2023, doi: https://doi.org/10.32604/iasc.2023.036946.
H. Shadabfar, M. Dehghan, and B. Sadeghian, “DSRL-APT-2023: A New Synthetic Dataset For Advanced Persistent Threats,” Vol. 17, Issue 2, vol. 17, no. 2, pp. 107–116, Jan. 2025, doi: 10.22042/isecure.2025.214212.
C. Catal, G. Giray, B. Tekinerdogan, S. Kumar, and S. Shukla, “Applications of deep learning for phishing detection: a systematic literature review,” Knowl. Inf. Syst., vol. 64, no. 6, pp. 1457–1500, Jun. 2022, doi: 10.1007/s10115-022-01672-x.
Shuhui Zhang, Mingyu Gao, “A Malware-Detection Method Using Deep Learning to Fully Extract API Sequence Features,” Electronics, vol. 14, no. 1, p. 167, 2025, doi: https://doi.org/10.3390/electronics14010167.
Saba Aslam, Hafsa Aslam, “AntiPhishStack: LSTM-Based Stacked Generalization Model for Optimized Phishing URL Detection,” Symmetry (Basel)., vol. 16, no. 2, p. 248, 2024, doi: https://doi.org/10.3390/sym16020248.
P. Maneriker, J. W. Stokes, E. G. Lazo, D. Carutasu, F. Tajaddodianfar, and A. Gururajan, “URLTran: Improving Phishing URL Detection Using Transformers,” Proc. - IEEE Mil. Commun. Conf. MILCOM, vol. 2021-November, pp. 197–204, 2021, doi: 10.1109/MILCOM52596.2021.9653028.
A. Bensaoud, J. Kalita, and M. Bensaoud, “A survey of malware detection using deep learning,” Mach. Learn. with Appl., vol. 16, p. 100546, 2024, doi: https://doi.org/10.1016/j.mlwa.2024.100546.
Jess Hohenstein, Rene F. Kizilcec, Dominic DiFranzo, Zhila Aghajari, Hannah Mieczkowski, Karen Levy, Mor Naaman, “Artificial intelligence in communication impacts language and social relationships,” Sci. Rep., 2023, [Online]. Available: https://www.nature.com/articles/s41598-023-30938-9
M. Odusami, S. Misra, O. Abayomi-Alli, A. Abayomi-Alli, and L. Fernandez-Sanz, “A survey and meta-analysis of application-layer distributed denial-of-service attack,” Int. J. Commun. Syst., vol. 33, no. 18, p. e4603, Dec. 2020, doi: 10.1002/DAC.4603.
Yafei Song, Dandan Zhang, Jian Wang, Yanan Wang, Yang Wang, Peng Ding, “Application of deep learning in malware detection: a review,” J. Big Data, vol. 12, no. 99, 2025.
Howon Kim, Thi-Thu-Huong Le, “Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory Analysis,” Appl. Sci., vol. 13, no. 19, p. 10730, 2023, doi: https://doi.org/10.3390/app131910730.
Max Landauer, Leonhard Alton, Martina Lindorfer, Florian Skopik, Markus Wurzenberger, Wolfgang Hotwagner, “Trace of the Times: Rootkit Detection through Temporal Anomalies in Kernel Activity,” arXiv:2503.02402, 2025, [Online]. Available: https://arxiv.org/abs/2503.02402
Sihat Afnan, Mushtari Sadia, Shahrear Iqbal, Anindya Iqbal, “LogShield: A Transformer-based APT Detection System Leveraging Self-Attention,” arXiv:2311.05733, 2023, [Online]. Available: https://arxiv.org/abs/2311.05733
Zian Jia, Yun Xiong, Yuhong Nan, Yao Zhang, Jinjing Zhao, Mi Wen, “MAGIC: Detecting Advanced Persistent Threats via Masked Graph Representation Learning,” arXiv:2310.09831, 2023, [Online]. Available: https://arxiv.org/abs/2310.09831
Huynh Thai Thi, Ngo Duc Hoang Son, Phan The Duy, Nghi Hoang Khoa, Khoa Ngo-Khanh, Van-Hau Pham, “XFedHunter: An Explainable Federated Learning Framework for Advanced Persistent Threat Detection in SDN,” arXiv:2309.08485, 2023, [Online]. Available: https://arxiv.org/abs/2309.08485
Hedyeh Nazari, Abbas Yazdinejad, Ali Dehghantanha, Fattane Zarrinkalam, Gautam Srivastava, “P3GNN: A Privacy-Preserving Provenance Graph-Based Model for APT Detection in Software Defined Networking,” arXiv:2406.12003, 2024, [Online]. Available: https://arxiv.org/abs/2406.12003
Sarah Mohammed Alshehri, Sanaa Abdullah Sharaf, “Systematic Review of Graph Neural Network for Malicious Attack Detection,” Information, vol. 16, no. 6, p. 470, 2025, doi: https://doi.org/10.3390/info16060470.
Weiwu Ren, Xintong Song, Yu Hong, Ying Lei, Jinyu Yao, Yazhou Du, Wenjuan Li, “APT Attack Detection Based on Graph Convolutional Neural Networks,” Int. J. Comput. Intell. Syst., vol. 16, no. 184, 2023, [Online]. Available: https://link.springer.com/article/10.1007/s44196-023-00369-5
Cho Do Xuan, Tung Thanh Nguyen, “A novel approach for APT attack detection based on an advanced computing,” Sci. Rep., vol. 14, 2024, [Online]. Available: https://www.nature.com/articles/s41598-024-72957-0
Nan Wang, Xuezhi Wen, Dalin Zhang, Xibin Zhao, Jiahui Ma, Mengxia Luo, Fan Xu, Sen Nie, Shi Wu, Jiqiang Liu, “TBDetector:Transformer-Based Detector for Advanced Persistent Threats with Provenance Graph,” arXiv:2304.02838, 2023, [Online]. Available: https://arxiv.org/abs/2304.02838
Miracle Udurume, Vladimir Shakhov, “Comparative Analysis of Deep Convolutional Neural Network—Bidirectional Long Short-Term Memory and Machine Learning Methods in Intrusion Detection Systems,” Appl. Sci., vol. 14, no. 16, p. 6967, 2024, doi: https://doi.org/10.3390/app14166967.
M. C. Ali Hussein Ali, “Unveiling machine learning strategies and considerations in intrusion detection systems: a comprehensive survey,” Front. Comput. Sci., vol. 6, 2024, [Online]. Available: https://www.frontiersin.org/journals/computer-science/articles/10.3389/fcomp.2024.1387354/full
Sidahmed Benabderrahmane, Ngoc Hoang, Petko Valtchev, James Cheney, Talal Rahwan, “Hack Me If You Can: Aggregating AutoEncoders for Countering Persistent Access Threats Within Highly Imbalanced Data,” arXiv:2406.19220, 2024, [Online]. Available: https://arxiv.org/abs/2406.19220
Sidahmed Benabderrahmane, Petko Valtchev, James Cheney, Talal Rahwan, “APT-LLM: Embedding-Based Anomaly Detection of Cyber Advanced Persistent Threats Using Large Language Models,” arXiv:2502.09385, 2025, [Online]. Available: https://arxiv.org/abs/2502.09385
Toya Acharya, Annamalai Annamalai, “Enhancing the Network Anomaly Detection using CNN-Bidirectional LSTM Hybrid Model and Sampling Strategies for Imbalanced Network Traffic Data,” Adv. Sci. Technol. Eng. Syst. J., vol. 9, no. 1, pp. 67–78, 2024, doi: 10.25046/aj090107.
Atmane Ayoub Mansour Bahar, Kamel Soaid Ferrahi, Mohamed-Lamine Messai, Hamida Seba, Karima Amrouche, “CONTINUUM: Detecting APT Attacks through Spatial-Temporal Graph Neural Networks,” arXiv:2501.02981, 2025, [Online]. Available: https://arxiv.org/abs/2501.02981
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 50sea
This work is licensed under a Creative Commons Attribution 4.0 International License.
