Latency-Aware Comparative Evaluation of Machine Learning Classifiers for Network Intrusion Detection Using the LAAI Metric on KDD99

Ali Haider

Authors

Ali Haider School of CS & IT, Institute of Management Sciences, Peshawar, Pakistan

Keywords:

Network Intrusion Detection, Machine Learning, KDD99, LAAI, Catboost, Xgboost, Latency-Aware Evaluation, Ensemble Learning, Cybersecurity

Abstract

The increasing sophistication of cyber threats to networked infrastructure has increased the need for accurate and effective network intrusion detection systems (NIDS) (NIDS). Although machine learning (ML) methods have performed exceptionally well in classification with canonical intrusion detection datasets, existing comparative literature does not consider inference latency as an important evaluation criterion, providing model recommendations that are correct but computationally infeasible to apply in practice. This paper presents a top-down, reproducible analysis of twelve machine learning classifiers (linear, probabilistic, tree-based, ensemble, and neural paradigms) on the KDD Cup 1999 (KDD99) benchmark using the Latency-Adjusted Accuracy Index (LAAI) as the main ranking tool to combine predictive accuracy and computational efficiency. All models use an identical preprocessing pipeline and identical hardware timing protocol to ensure consistency in the methodology. Results show that CatBoost achieves the highest LAAI score of 0.9946 (accuracy 99.94), then Decision Tree (LAAI 0.9991), and XGBoost (LAAI 0.9852). More importantly, K-Nearest Neighbours (KNN) with a test accuracy of 99.87% comes with the lowest LAAI of 0.3864 with extremely high inference latency of 1.585 ms - an accuracy-latency paradox which is not captured by traditional evaluation metrics. In the Tier I cluster, the mean accuracy is 99.71% (SD = 0.61%) while the mean LAAI is 0.9857 (SD = 0.0094). In contrast, the mean accuracy across the whole dataset is 92.22% (SD = 14.81%) while the mean LAAI is 0.8497 (SD = 0.1997), which highlights the significant performance disparity between top and bottom tier classifiers. This suggests a four-level deployment classification to guide NIDS model selection for practitioners. The LAAI-based rankings are demonstrated to be generally applicable with the help of cross-validation against six independent benchmark studies.

References

A. L. Buczak and E. Guven, “A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection,” IEEE Commun. Surv. Tutorials, vol. 18, no. 2, pp. 1153–1176, 2016, doi: 10.1109/COMST.2015.2494502.

“Global Cybersecurity Index 2020.” Accessed: Apr. 13, 2026. [Online]. Available: https://www.itu.int/epublications/publication/D-STR-GCI.01-2021-HTM-E/

Iqbal H. Sarker, A. S. M. Kayes, Shahriar Badsha, Hamed Alqahtani, Paul Watters & Alex Ng, “Cybersecurity data science: an overview from machine learning perspective,” J. Big Data, vol. 7, no. 41, 2020, [Online]. Available: https://link.springer.com/article/10.1186/s40537-020-00318-5

M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis of the KDD CUP 99 data set,” IEEE Symp. Comput. Intell. Secur. Def. Appl. CISDA 2009, Dec. 2009, doi: 10.1109/CISDA.2009.5356528.

“(PDF) Reliable evaluation for the AI-enabled intrusion detection system from data perspective.” Accessed: Apr. 13, 2026. [Online]. Available: https://www.researchgate.net/publication/397105850_Reliable_evaluation_for_the_AI-enabled_intrusion_detection_system_from_data_perspective

T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting System,” Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., vol. 13-17-August-2016, pp. 785–794, Mar. 2016, doi: 10.1145/2939672.2939785.

G. Ke et al., “LightGBM: A Highly Efficient Gradient Boosting Decision Tree,” Adv. Neural Inf. Process. Syst., vol. 30, 2017, Accessed: Oct. 10, 2025. [Online]. Available: https://github.com/Microsoft/LightGBM.

Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, Andrey Gulin, “CatBoost: unbiased boosting with categorical features,” arXiv:1706.09516, 2017, [Online]. Available: https://arxiv.org/abs/1706.09516

S. K. Wanjau, G. M. Wambugu, and A. M. Oirere, “Network Intrusion Detection Systems: A Systematic Literature Review o f Hybrid Deep Learning Approaches,” Int. J. Emerg. Sci. Eng., vol. 10, no. 7, pp. 1–16, Jun. 2022, doi: 10.35940/IJESE.F2530.0610722.

“(PDF) A survey of intrusion detection techniques.” Accessed: Apr. 13, 2026. [Online]. Available: https://www.researchgate.net/publication/332665057_A_

survey_of_intrusion_detection_techniques

Ansam Khraisat, Iqbal Gondal, Peter Vamplew, “Survey of intrusion detection systems: techniques, datasets and challenges,” Cybersecurity, vol. 2, no. 20, 2019, [Online]. Available: https://link.springer.com/article/10.1186/s42400-019-0038-7

L. Breiman, “Random Forests-Random Features,” 1999.

J. H. Friedman, “Greedy function approximation: A gradient boosting machine,” Ann. Stat., vol. 29, no. 5, pp. 1189–1232, 2001, doi: 10.1214/AOS/1013203451.

Oduwole Omolara Oluwakemi, Muhammad, Umar Abdullahi, “Comparative Evaluation of Machine Learning Algorithms for Intrusion Detection,” Asian J. Res. Comput. Sci., vol. 16, no. 4, pp. 8–22, 2023, doi: 10.9734/AJRCOS/2023/v16i4366.

S. A. Almahaqeri, M. H. Almourish, A. A. Nasser, A. S. A. Alghawli, A. A. K. Elsayed, and A. N. Alhejoj, “An optimized gradient boosting framework for IoT intrusion detection: a comprehensive evaluation on the CICIoT2023 dataset,” Sci. Reports 2026, Apr. 2026, doi: 10.1038/S41598-026-47399-5.

Busra Buyuktanir, Şahsene Altinkaya, Gozde Karatas Baydogmus & Kazim Yildiz, “Federated learning in intrusion detection: advancements, applications, and future directions,” Cluster Comput., vol. 28, 2025, [Online]. Available: https://link.springer.com/article/10.1007/s10586-025-05325-w

Tae Guen Kim, Hyeon Park, “XGBoost-Based Anomaly Detection Framework for SOME/IP in In-Vehicle Networks,” Systems, vol. 14, no. 2, p. 196, 2026, doi: https://doi.org/10.3390/systems14020196.

PedregosaFabian et al., “Scikit-learn: Machine Learning in Python,” J. Mach. Learn. Res., Nov. 2011, doi: 10.5555/1953048.2078195.

Shamma Shabnam Nasim, Prashant Pranav & Sandip Dutta, “A systematic literature review on intrusion detection techniques in cloud computing,” Discov. Comput., vol. 28, no. 107, 2025, [Online]. Available: https://link.springer.com/

article/10.1007/s10791-025-09641-y

Mohammad Nassef, “Boosting Intrusion Detection Against DDoS Attacks Using a Feature Engineering-Based Fine-Tuned XGBoost Model,” Int. J. Semant. Web Inf. Syst., vol. 21, no. 1, pp. 1–39, 2025, doi: 10.4018/IJSWIS.383062.

I. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani, “Toward generating a new intrusion detection dataset and intrusion traffic characterization,” ICISSP 2018 - Proc. 4th Int. Conf. Inf. Syst. Secur. Priv., vol. 2018-January, pp. 108–116, 2018, doi: 10.5220/0006639801080116.

N. Moustafa and J. Slay, “UNSW-NB15: A comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set),” 2015 Mil. Commun. Inf. Syst. Conf. MilCIS 2015 - Proc., Dec. 2015, doi: 10.1109/MILCIS.2015.7348942.

Mohammed Nasser Al-Mhiqani, Rabiah Ahmad, “A Review of Insider Threat Detection: Classification, Machine Learning Techniques, Datasets, Open Challenges, and Recommendations,” Appl. Sci., vol. 10, no. 15, p. 5208, 2020, doi: https://doi.org/10.3390/app10155208.

A. Chouhan, N. Shahriar, and J. T. Yao, “HCL: A Hybrid CNN-LSTM Framework for Intrusion Detection in SDN-IoT Networks,” 2025 Int. Conf. Comput. Netw. Commun. ICNC 2025, pp. 254–258, 2025, doi: 10.1109/ICNC64010.2025.10994022.

I. Sharma, A. Khanna, and T. Verma, “Enhanced APT Attack Detection Using Convolutional Neural Networks and Deep Learning Models,” 2025 Int. Conf. Innov. Emerg. Technol. AI & Commun. Syst., pp. 45–50, Nov. 2025, doi: 10.1109/IETACS68750.2025.11385486.

U. Adhikari, T. H. Morris, and S. Pan, “Applying Hoeffding Adaptive Trees for Real-Time Cyber-Power Event and Intrusion Classification,” IEEE Trans. Smart Grid, vol. 9, no. 5, pp. 4049–4060, Sep. 2018, doi: 10.1109/TSG.2017.2647778.

Jieming Gu, Yue Zhong & Xiangzhan Yu, “Bridging packet and session: Cross-level dual-attention networks for encrypted traffic classification,” J. King Saud Univ. Comput. Inf. Sci., vol. 38, no. 79, 2026, [Online]. Available: https://link.springer.com/article/10.1007/s44443-026-00470-7

S. Haq and Y. Singh, “Botnet detection using machine learning,” PDGC 2018 - 2018 5th Int. Conf. Parallel, Distrib. Grid Comput., pp. 240–245, Dec. 2018, doi: 10.1109/PDGC.2018.8745912.

“(PDF) Enhanced DDoS detection using cnn1d with reciprocal points learning and attention mechanism.” Accessed: Apr. 13, 2026. [Online]. Available: https://www.researchgate.net/publication/395760043_Enhanced_DDoS_detection_using_cnn1d_with_reciprocal_points_learning_and_attention_mechanism

G. Hemanth Kumar, Sivananda Lahari Reddy Elicherla, “FL-DPCSA: Federated learning with differential privacy for cache side-channel attack detection in edge-based smart grids,” e-Prime - Adv. Electr. Eng. Electron. Energy, vol. 13, p. 101057, 2025, doi: https://doi.org/10.1016/j.prime.2025.101057.

H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1263–1284, Sep. 2009, doi: 10.1109/TKDE.2008.239.

R. Sommer and V. Paxson, “Outside the closed world: On using machine learning for network intrusion detection,” Proc. - IEEE Symp. Secur. Priv., pp. 305–316, 2010, doi: 10.1109/SP.2010.25.