AI-Based Phishing Detection System Using Random Forest with Smote and Explainabiltiy

Authors

  • Ansar Mehmood Data Science Intelligence, Canterbury Christ Church University, UK
  • Sughra Fatima Manzoor Department of Software Engineering, University of Mianwali, Pakistan
  • Altaf Khan Department of Software Engineering, University of Mianwali, Pakistan
  • Asif Raza Department of Software Engineering, University of Mianwali, Pakistan

Keywords:

Phishing emails, Random Forest, NLP, Cyber Attack, SHAP, Phishing Detection System, Email phishing, SMOTE, Feature Extraction, Social Engineering, XAI

Abstract

One of the most widespread and destructive forms of Cyber Security breach is the Phishing Attack. Phishing attacks often defeat email filtering systems due to their heightened use of social engineering techniques that by-pass traditional filtering mechanisms. This research employs a Random Forest classifier, alongside TF-IDF based feature extraction methods, to classify Emails. The major innovation of this research is the implementation of SHAP (SHapley Additive exPlanations) technology in relation to the output of the Random Forest to provide explanations of each prediction in an understandable and user-friendly manner. Users may access this system via a simple copy-and-paste interface or uploading a file to submit email content for real-time analysis. The results of the analysis will consist of prediction (Phishing/Safe), confidence rating, level of risk, and a detailed listing of each risk factor as calculated using SHAP based explanations. The Random Forest model will have an accuracy of 96.12% with 94.4% precision and a recall rate of 95.87%, significantly better than the results generated using traditional rule-based methods. Response times for Email analysis averaged 3 to 5 sec. Combining high detection rates with explainable AI improved Cyber Security Awareness, while also offering practical and easily accessible protection from Phishing attacks for individual.

References

T. Qiao, Z. Duan, L. Liu, S. Pouriyeh, T. Yadav, and M. Masum, “Explainable Multi-Agent LLM Framework for Phishing Email Detection via Role-Specialized Evidence Decomposition,” Electron. 2026, Vol. 15, Page 2606, vol. 15, no. 12, p. 2606, Jun. 2026, doi: 10.3390/ELECTRONICS15122606.

“Phishing activity trend report,” 4th Quart., 2025.

“Phishing Infographic”, [Online]. Available: https://www.cisa.gov/sites/default/files/2023-02/phishing-infographic-508c.pdf

R. Mishra and G. Varshney, “A Study of Effectiveness of Brand Domain Identification Features for Phishing Detection in 2025,” Lect. Notes Comput. Sci., vol. 15655 LNCS, pp. 89–108, 2026, doi: 10.1007/978-3-032-01823-6_6/SAVE-RESEARCH.

B. Lim, R. Huerta, A. Sotelo, A. Quintela, and P. Kumar, “EXPLICATE: Enhancing Phishing Detection through Explainable AI and LLM-Powered Interpretability,” Mar. 2025, Accessed: Jul. 25, 2026. [Online]. Available: https://arxiv.org/pdf/2503.20796

Ismi Rosia Dwianti et al., “Klasifikasi Email Phishing Menggunakan Metode TF-IDF dan Algoritma Random Forest,” ROUTERS J. Sist. dan Teknol. Inf., pp. 125–135, Jul. 2025, doi: 10.25181/RT.V3I2.4310.

T. Koide, N. Fukushi, H. Nakano, and D. Chiba, “ChatSpamDetector: Leveraging Large Language Models for Effective Phishing Email Detection,” Feb. 2024, Accessed: Apr. 20, 2025. [Online]. Available: https://arxiv.org/abs/2402.18093v2

M. A. A. Mohd Nazariee, S. P. Kumaresan, A. Haddad, and M. U. A. Ayoobkhan, “AI-Powered Threat Hunting for Email Phishing Attack Detection Using Natural Language Processing (NLP),” J. Informatics Web Eng., vol. 5, no. 2, p. 160, Jun. 2026, doi: 10.33093/JIWE.2026.5.2.10.

A. A. Akinyelu and A. O. Adewumi, “Classification of phishing email using random forest machine learning technique,” J. Appl. Math., vol. 2014, 2014, doi: 10.1155/2014/425731.

H. Fares et al., “Machine Learning Approach for Email Phishing Detection,” Procedia Comput. Sci., vol. 251, pp. 746–751, Jan. 2024, doi: 10.1016/J.PROCS.2024.11.179.

A. I. Champa, M. F. Rabbi, and M. F. Zibran, “Curated Datasets and Feature Analysis for Phishing Email Detection with Machine Learning,” 2024 IEEE 3rd Int. Conf. Comput. Mach. Intell. ICMI 2024 - Proc., 2024, doi: 10.1109/ICMI60790.2024.10585821.

J. Lee, F. Tang, P. Ye, F. Abbasi, P. Hay, and D. M. Divakaran, “D-Fence: A Flexible, Efficient, and Comprehensive Phishing Email Detection System,” Eur. Symp. Secur. Priv., pp. 578–597, Sep. 2021, doi: 10.1109/EUROSP51992.2021.00045.

M. Hina, M. Ali, A. R. Javed, F. Ghabban, L. A. Khan, and Z. Jalil, “SeFACED: Semantic-Based Forensic Analysis and Classification of E-Mail Data Using Deep Learning,” IEEE Access, vol. 9, pp. 98398–98411, 2021, doi: 10.1109/ACCESS.2021.3095730.

Firman, Tukiyat, and S. Wiharjo, “Phishing Email Classification Approach Using Machine Learning Algorithms - A Literature Review,” Data J. Inf. Syst. Manag., vol. 3, no. 3, pp. 135–145, Jul. 2025, doi: 10.61978/DATA.V3I3.692.

I. Fette, N. Sadeh, and A. Tomasic, “Learning to detect phishing emails,” 16th Int. World Wide Web Conf. WWW2007, pp. 649–656, 2007, doi: 10.1145/1242572.1242660.

A. Al Tawil, L. Almazaydeh, D. Qawasmeh, B. Qawasmeh, M. Alshinwan, and K. Elleithy, “Comparative Analysis of Machine Learning Algorithms for Email Phishing Detection Using TF-IDF, Word2Vec, and BERT,” Comput. Mater. Contin., vol. 81, no. 2, pp. 3395–3412, Nov. 2024, doi: 10.32604/CMC.2024.057279.

H. M. U. Akhtar, M. Nauman, N. A. M. Hameed, S. Hameed, and M. Z. Tareen, “Mitigating Cyber Threats: Machine Learning and Explainable AI for Phishing Detection,” VFAST Trans. Softw. Eng., vol. 13, no. 2, pp. 170–195, Apr. 2025, doi: 10.21015/VTSE.V13I2.2129.

Adwaith Atholi Thiruvoth and Pushkar Ogale, “A Phishing Detection System for Enhanced Cybersecurity Using Machine Learning,” Proc. 20th Int. Conf. Softw. Technol. (ICSOFT 2025), 2025, [Online]. Available: https://www.scitepress.org/Papers/2025/135708/135708.pdf

S. R. Abdul Samad et al., “Analysis of the Performance Impact of Fine-Tuned Machine Learning Model for Phishing URL Detection,” Electron. 2023, Vol. 12, Page 1642, vol. 12, no. 7, p. 1642, Mar. 2023, doi: 10.3390/ELECTRONICS12071642.

S. Nepal, H. Gurung, and R. Nepal, “Phishing URL Detection Using CNN-LSTM and Random Forest Classifier,” Sep. 2022, doi: 10.21203/RS.3.RS-2043842/V1.

A. Almomani, B. B. Gupta, S. Atawneh, A. Meulenberg, and E. Almomani, “A survey of phishing email filtering techniques,” IEEE Commun. Surv. Tutorials, vol. 15, no. 4, pp. 2070–2090, 2013, doi: 10.1109/SURV.2013.030713.00020.

A. Bergholz, J. De Beer, S. Glahn, M. F. Moens, G. Paaß, and S. Strobel, “New filtering approaches for phishing email,” J. Comput. Secur., vol. 18, no. 1, pp. 7–35, 2010, doi: 10.3233/JCS-2010-0371.

A. Aljofey, Q. Jiang, Q. Qu, M. Huang, and J. P. Niyigena, “An Effective Phishing Detection Model Based on Character Level Convolutional Neural Network from URL,” Electron. 2020, Vol. 9, Page 1514, vol. 9, no. 9, p. 1514, Sep. 2020, doi: 10.3390/ELECTRONICS9091514.

B. B. Gupta, A. Tewari, A. K. Jain, and D. P. Agrawal, “Fighting against phishing attacks: state of the art and future challenges,” Neural Comput. Appl. 2016 2812, vol. 28, no. 12, pp. 3629–3654, Mar. 2016, doi: 10.1007/S00521-016-2275-Y.

R. S. Rao and A. R. Pais, “Detection of phishing websites using an efficient feature-based machine learning framework,” Neural Comput. Appl. 2018 318, vol. 31, no. 8, pp. 3851–3873, Jan. 2018, doi: 10.1007/S00521-017-3305-0.

S. Salloum, T. Gaber, S. Vadera, and K. Shaalan, “A Systematic Literature Review on Phishing Email Detection Using Natural Language Processing Techniques,” IEEE Access, vol. 10, pp. 65703–65727, 2022, doi: 10.1109/ACCESS.2022.3183083.

“Phishing Email Dataset.” Accessed: Jul. 25, 2026. [Online]. Available: https://www.kaggle.com/datasets/naserabdullahalam/phishing-email-dataset

“A hierarchical framework for content-based image spam filtering | IEEE Conference Publication | IEEE Xplore.” Accessed: Aug. 19, 2026. [Online]. Available: https://ieeexplore.ieee.org/document/6269246

A. Alhuzali, A. Alloqmani, M. Aljabri, and F. Alharbi, “In-Depth Analysis of Phishing Email Detection: Evaluating the Performance of Machine Learning and Deep Learning Models Across Multiple Datasets,” Appl. Sci. 2025, Vol. 15, Page 3396, vol. 15, no. 6, p. 3396, Mar. 2025, doi: 10.3390/APP15063396.

S. Chakraborty, “Phishing email detection,” Kaggle, 2023.

S. Mahendru and T. Pandit, “SecureNet: A Comparative Study of DeBERTa and Large Language Models for Phishing Detection,” 2024 IEEE 7th Int. Conf. Big Data Artif. Intell. BDAI 2024, pp. 160–169, 2024, doi: 10.1109/BDAI62182.2024.10692765.

M. Eskandarian et al., “A lightweight defense mechanism against next-generation of phishing emails using distilled attention-augmented BiLSTM,” J. Inf. Secur. Appl., vol. 101, p. 104552, Sep. 2026, doi: 10.1016/J.JISA.2026.104552.

D. H. Kulal, C. P. Arannonu, A. Anwar, N. Rastogi, and Q. Niyaz, “Robust ML-based Detection of Conventional, LLM-Generated, and Adversarial Phishing Emails Using Advanced Text Preprocessing,” Oct. 2025, Accessed: Aug. 19, 2026. [Online]. Available: http://arxiv.org/abs/2510.11915

R. Rajoju, V. Sathvika, G. N. S. Smaran, C. Tejashwini, and G. A. Reddy, “Text Phishing Detection System using Random Forest Algorithm,” Proc. 3rd Int. Conf. Appl. Artif. Intell. Comput. ICAAIC 2024, pp. 1332–1339, 2024, doi: 10.1109/ICAAIC60222.2024.10575110.

L. F. Gutierrez, F. Abri, M. Armstrong, A. S. Namin, and K. S. Jones, “Email Embeddings for Phishing Detection,” Proc. - 2020 IEEE Int. Conf. Big Data, Big Data 2020, pp. 2087–2092, Dec. 2020, doi: 10.1109/BIGDATA50022.2020.9377821.

M. Fadli, V. Wijaya, M. R. Pribadi, and W. Widhiarso, “Effect of TF-IDF Extraction and Application of SMOTE on Model Performance in Detecting Spam Email,” Int. Conf. Electr. Eng. Comput. Sci. Informatics, pp. 637–641, 2023, doi: 10.1109/EECSI59885.2023.10295851.

R. Abdulraheem, A. Odeh, M. Al Fayoumi, and I. Keshta, “Efficient Email phishing detection using Machine learning,” 2022 IEEE 12th Annu. Comput. Commun. Work. Conf. CCWC 2022, pp. 354–358, 2022, doi: 10.1109/CCWC54503.2022.9720818.

R. Fatima, M. M. S. Fareed, S. Ullah, G. Ahmad, and S. Mahmood, “An Optimized Approach for Detection and Classification of Spam Email’s Using Ensemble Methods,” Wirel. Pers. Commun. 2024 1391, vol. 139, no. 1, pp. 347–373, Nov. 2024, doi: 10.1007/S11277-024-11628-9.

S. S. Shafin, “An explainable feature selection framework for web phishing detection with machine learning,” Data Sci. Manag., vol. 8, no. 2, pp. 127–136, Jun. 2025, doi: 10.1016/J.DSM.2024.08.004.

M. Mia, D. Derakhshan, and M. M. A. Pritom, “Can Features for Phishing URL Detection Be Trusted Across Diverse Datasets? A Case Study with Explainable AI,” Proc. 2024 11th Int. Conf. Networking, Syst. Secur. NSysS 2024, vol. 1, pp. 137–145, Jan. 2025, doi: 10.1145/3704522.3704532;CSUBTYPE:STRING:CONFERENCE.

F. Jáñez-Martino, R. Alaiz-Rodríguez, V. González-Castro, E. Fidalgo, and E. Alegre, “Classifying spam emails using agglomerative hierarchical clustering and a topic-based approach,” Appl. Soft Comput., vol. 139, p. 110226, May 2023, doi: 10.1016/J.ASOC.2023.110226.

S. Salloum, T. Gaber, S. Vadera, and K. Shaalan, “A New English/Arabic Parallel Corpus for Phishing Emails,” ACM Trans. Asian Low-Resource Lang. Inf. Process., vol. 22, no. 7, Jul. 2023, doi: 10.1145/3606031;

Downloads

Published

2026-08-30

How to Cite

Mehmood, A., Manzoor , S. F., Altaf Khan, & Asif Raza. (2026). AI-Based Phishing Detection System Using Random Forest with Smote and Explainabiltiy. International Journal of Innovations in Science & Technology, 8(5), 2371–2390. Retrieved from https://journal.50sea.com/index.php/IJIST/article/view/1964