AI Transparency in Network Intrusion Detection: Utilizing Explainable Machine Learning for Identifying Security Threats

Dawlat Mustafa Sulaiman

doi:10.65542/djei.v2i3.87

Authors

Dawlat Mustafa Sulaiman Department of Information Technology, Technical College of Duhok, Duhok Polytechnic University, Duhok, Kurdistan Region, Iraq https://orcid.org/0009-0004-8306-5834

DOI:

https://doi.org/10.65542/djei.v2i3.87

Keywords:

Intrusion Detection Systems, Machine Learning, Random Forest, Cybersecurity, NSL-KDD, Explainable AI

Abstract

Machine-learning intrusion detectors often report near-perfect performance on familiar benchmarks, yet their generalization and decision transparency remain uncertain. This study evaluates a reproducible and explainable binary network-intrusion-detection workflow using the NSL-KDD KDDTrain+_20Percent file (25,192 records and 41 input features). Six classifiers were compared: Random Forest (RF), Logistic Regression, Decision Tree (DT), Support Vector Machine, k-Nearest Neighbors, and Gaussian Naive Bayes. The protocol used a stratified 70:30 hold-out split, five-fold stratified cross-validation, training-only preprocessing, a fixed random seed, and independent evaluation on KDDTest+. On the internal hold-out set, RF achieved 99.64% accuracy, 99.40% attack recall, 99.85% specificity, and a ROC-AUC of 0.99994; its five-fold mean accuracy was 99.74% with a standard deviation of 0.16%. On KDDTest+, RF accuracy decreased to 77.82%, whereas DT achieved the highest independent accuracy of 81.53%. Global and case-based SHAP analyses identified influential network features and explained correct, false-positive, and false-negative predictions. These findings show that near-perfect internal NSL-KDD performance does not establish deployment readiness without independent testing and transparent error analysis.

References

1. Gunduz, M.Z.; Das, R. Cyber-security on smart grid: Threats and potential solutions. Comput. Netw. 2020, 169, 107094. https://doi.org/10.1016/j.comnet.2019.107094 DOI: https://doi.org/10.1016/j.comnet.2019.107094

2. Berman, D.S.; Buczak, A.L.; Chavis, J.S.; Corbett, C.L. A survey of deep learning methods for cyber security. Information 2019, 10, 122. https://doi.org/10.3390/info10040122 DOI: https://doi.org/10.3390/info10040122

3. International Telecommunication Union. Measuring Digital Development: Facts and Figures 2021; ITU: Geneva, Switzerland, 2021.

4. Check Point Research. Cyber Attack Trends: 2021 Mid-Year Report; Check Point Software Technologies: Tel Aviv, Israel, 2021.

5. Zeadally, S.; Adi, E.; Baig, Z.; Khan, I.A. Harnessing artificial intelligence capabilities to improve cybersecurity. IEEE Access 2020, 8, 23817–23837. https://doi.org/10.1109/ACCESS.2020.2968045 DOI: https://doi.org/10.1109/ACCESS.2020.2968045

6. Wang, M.; Zheng, K.; Yang, Y.; Wang, X. An explainable machine learning framework for intrusion detection systems. IEEE Access 2020, 8, 73127–73141. https://doi.org/10.1109/ACCESS.2020.2988359 DOI: https://doi.org/10.1109/ACCESS.2020.2988359

7. Marino, D.L.; Wickramasinghe, C.S.; Manic, M. An adversarial approach for explainable AI in intrusion detection systems. In Proceedings of the IECON 2018—44th Annual Conference of the IEEE Industrial Electronics Society, Washington, DC, USA, 21–23 October 2018; pp. 3237–3243. https://doi.org/10.1109/IECON.2018.8591457 DOI: https://doi.org/10.1109/IECON.2018.8591457

8. Ribeiro, M.T.; Singh, S.; Guestrin, C. “Why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 1135–1144. https://doi.org/10.1145/2939672.2939778 DOI: https://doi.org/10.1145/2939672.2939778

9. Sahu, S.K.; Sarangi, S.; Jena, S.K. A detail analysis on intrusion detection datasets. In Proceedings of the 2014 IEEE International Advance Computing Conference (IACC), Gurgaon, India, 21–22 February 2014; pp. 1348–1353. https://doi.org/10.1109/IAdCC.2014.6779523 DOI: https://doi.org/10.1109/IAdCC.2014.6779523

10. Gaspar, D.; Silva, P.; Silva, C. Explainable AI for intrusion detection systems: LIME and SHAP applicability on multi-layer perceptron. IEEE Access 2024, 12, 30164–30175. https://doi.org/10.1109/ACCESS.2024.3368377 DOI: https://doi.org/10.1109/ACCESS.2024.3368377

11. Tavallaee, M.; Bagheri, E.; Lu, W.; Ghorbani, A.A. A detailed analysis of the KDD CUP 99 data set. In Proceedings of the 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, Ottawa, ON, Canada, 8–10 July 2009; pp. 1–6. https://doi.org/10.1109/CISDA.2009.5356528 DOI: https://doi.org/10.1109/CISDA.2009.5356528

12. Canadian Institute for Cybersecurity, University of New Brunswick. NSL-KDD Dataset. Available online: https://www.unb.ca/cic/datasets/nsl.html (accessed on 7 August 2026).

13. Awotunde, J.B.; Misra, S. Feature extraction and artificial intelligence-based intrusion detection model for a secure Internet of Things networks. In Illumination of Artificial Intelligence in Cybersecurity and Forensics; Misra, S., Arumugam, C., Eds.; Lecture Notes on Data Engineering and Communications Technologies; Springer: Cham, Switzerland, 2022; Volume 109, pp. 21–44. https://doi.org/10.1007/978-3-030-93453-8_2 DOI: https://doi.org/10.1007/978-3-030-93453-8_2

14. Zebin, T.; Rezvy, S.; Luo, Y. An explainable AI-based intrusion detection system for DNS over HTTPS (DoH) attacks. IEEE Trans. Inf. Forensics Secur. 2022, 17, 2339–2349. https://doi.org/10.1109/TIFS.2022.3183390 DOI: https://doi.org/10.1109/TIFS.2022.3183390

15. Saheed, Y.K.; Abiodun, A.I.; Misra, S.; Holone, M.K.; Colomo-Palacios, R. A machine learning-based intrusion detection for detecting Internet of Things network attacks. Alex. Eng. J. 2022, 61, 9395–9409. https://doi.org/10.1016/j.aej.2022.02.063 DOI: https://doi.org/10.1016/j.aej.2022.02.063

16. Musleh, D.; Alotaibi, M.; Alhaidari, F.; Rahman, A.; Mohammad, R.M. Intrusion detection system using feature extraction with machine learning algorithms in IoT. J. Sens. Actuator Netw. 2023, 12, 29. https://doi.org/10.3390/jsan12020029 DOI: https://doi.org/10.3390/jsan12020029

17. Mane, S.; Rao, D. Explaining network intrusion detection system using explainable AI framework. arXiv 2021, arXiv:2103.07110. https://doi.org/10.48550/arXiv.2103.07110

18. Logeswari, G.; Bose, S.; Anitha, T. An intrusion detection system for SDN using machine learning. Intell. Autom. Soft Comput. 2023, 35, 867–880. https://doi.org/10.32604/iasc.2023.026769 DOI: https://doi.org/10.32604/iasc.2023.026769

19. Wali, S.; Farrukh, Y.A.; Khan, I. Explainable AI and Random Forest based reliable intrusion detection system. Comput. Secur. 2025, 157, 104542. https://doi.org/10.1016/j.cose.2025.104542 DOI: https://doi.org/10.1016/j.cose.2025.104542

20. Karakaya, A. A hybrid approach for IoT security: Combining ensemble learning with fuzzy logic. Sensors 2025, 25, 5668. https://doi.org/10.3390/s25185668 DOI: https://doi.org/10.3390/s25185668

21. Bikila, D.D.; Capek, J. Machine learning-based attack detection for the Internet of Things. Future Gener. Comput. Syst. 2025, 166, 107630. https://doi.org/10.1016/j.future.2024.107630 DOI: https://doi.org/10.1016/j.future.2024.107630

22. Wahib, H.A.; Abdullah, M.Z.; Mohammad, A.S. Enhancing intrusion detection system for software-defined networks based on machine learning. Ing. Syst. Inf. 2025, 30, 1015–1025. https://doi.org/10.18280/isi.300418 DOI: https://doi.org/10.18280/isi.300418

23. Almufti, S.M.; Hani, A.A.; Zeebaree, S.R.M.; Asaad, R.R.; Majeed, D.A.; Sallow, A.B.; Ahmad, H.B. Intelligent home IoT devices: An exploration of machine learning-based networked traffic investigation. J. Ilm. Ilmu Terap. Univ. Jambi 2024, 8, 1–10. https://doi.org/10.22437/jiituj.v8i1.32767 DOI: https://doi.org/10.22437/jiituj.v8i1.32767

24. Mohammed, M.A.; Lakhan, A.; Zebari, D.A.; Abdulkareem, K.H.; Nedoma, J.; Martinek, R.; Tariq, U.; Alhaisoni, M.; Tiwari, P. Adaptive secure malware efficient machine learning algorithm for healthcare data. CAAI Trans. Intell. Technol. 2023. https://doi.org/10.1049/cit2.12200 DOI: https://doi.org/10.1049/cit2.12200

25. Zebari, D.A.; Saleh, M.D.; Ahmed, F.Y.H.; Asaad, R.R. An explainable machine learning framework for Parkinson’s disease classification using high-dimensional speech features. Qubahan Acad. J. 2026, 5, 14–35. https://doi.org/10.48161/qaj.v5n2a84 DOI: https://doi.org/10.48161/qtj.v5n2a84

26. Cover, T.M.; Hart, P.E. Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 1967, 13, 21–27. https://doi.org/10.1109/TIT.1967.1053964 DOI: https://doi.org/10.1109/TIT.1967.1053964

27. Cortes, C.; Vapnik, V. Support-vector networks. Mach. Learn. 1995, 20, 273–297. https://doi.org/10.1007/BF00994018 DOI: https://doi.org/10.1007/BF00994018

28. Rokach, L.; Maimon, O. Decision trees. In Data Mining and Knowledge Discovery Handbook; Maimon, O., Rokach, L., Eds.; Springer: New York, NY, USA, 2005; pp. 165–192. DOI: https://doi.org/10.1007/0-387-25465-X_9

29. Breiman, L. Bagging predictors. Mach. Learn. 1996, 24, 123–140. https://doi.org/10.1007/BF00058655 DOI: https://doi.org/10.1007/BF00058655

30. Breiman, L. Random forests. Mach. Learn. 2001, 45, 5–32. https://doi.org/10.1023/A:1010933404324 DOI: https://doi.org/10.1023/A:1010933404324

31. Lewis, D.D. Naive (Bayes) at forty: The independence assumption in information retrieval. In Machine Learning: ECML-98; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 1998; Volume 1398, pp. 4–15. https://doi.org/10.1007/BFb0026666 DOI: https://doi.org/10.1007/BFb0026666

32. Hosmer, D.W., Jr.; Lemeshow, S.; Sturdivant, R.X. Applied Logistic Regression, 3rd ed.; Wiley: Hoboken, NJ, USA, 2013. https://doi.org/10.1002/9781118548387 DOI: https://doi.org/10.1002/9781118548387

33. Lundberg, S.M.; Lee, S.-I. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30, Proceedings of the Annual Conference on Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; pp. 4765–4774.

34. Saabas, A. treeinterpreter: Package for interpreting scikit-learn’s decision tree and random forest predictions. Python package. Available online: https://pypi.org/project/treeinterpreter/ (accessed on 21 August 2026).