A Hybrid Machine Learning Approach for Enhancing Intrusion Detection Systems Using CICIDS2017 Dataset

Ara Zozan Miran; Govand Salih Kadir

doi:10.65542/djei.v2i2.49

Authors

Ara Zozan Miran 1 Department of Information Technology, Technical College of Duhok, Duhok Polytechnic University, Duhok, Kurdistan Region, Iraq, 2 Department of Information Technology, Technical College of Informatics- Akre, Akre University for Applied Science. https://orcid.org/0000-0002-9019-0068
Govand Salih Kadir University of Kurdistan Hewlêr (UKH), Erbil City, Iraq https://orcid.org/0000-0001-8013-8947

DOI:

https://doi.org/10.65542/djei.v2i2.49

Keywords:

Intrusion Detection System (IDS), Machine Learning (ML), Supervised Learning, Unsupervised Learning, Anomaly Detection

Abstract

Traditional Intrusion Detection Systems (IDSs) are often ineffective because they rely on signature-based methods, resulting in high false-positive rates. With the expansion of new Artificial Intelligence (AI), especially Machine Learning (ML) algorithms, used in IDS systems, it is possible to achieve high performance in detecting anomalous and known threats. For that reason, this study aims to use Machine Learning (ML) algorithms to learn different patterns of known and anomalous threats across the Data Link (Layer 2) and Network (Layer 3) characteristics of the OSI model, as a basis for adaptive systems for IDS frameworks. Two supervised models (Random Forest, XGBoost) and two unsupervised models (Isolation Forest, One-Class SVM) were compared on the CICIDS2017 online dataset. The Results indicate that XGBoost achieved 99.3% accuracy on known threats, while One-Class SVM achieved 92.1% accuracy for unknown threats. Later, model performance was evaluated using standard classification metrics with a paired t-test. The findings of this study show the importance of combining supervised and unsupervised ML algorithms as a hybrid system to detect, classify, and learn from known and anomalous threats on different layers of network traffic. As a result, the findings can serve as a base for the adaptive IDS systems that can learn features and achieve higher performance.

References

Ahmad, Z.; Khan, A.S.; Shiang, C.W.; Abdullah, J.; Ahmad, F. Network intrusion detection system: A systematic study of machine learning and deep learning approaches. Trans. Emerg. Telecommun. Technol. 2021, 32, e4150. https://doi.org/10.1002/ett.4150. DOI: https://doi.org/10.1002/ett.4150

Alhayali, R.A.I.; Aljanabi, M.; Ali, A.H.; Mohammed, M.A.; Sutikno, T. Optimized machine learning algorithm for intrusion detection. Indones. J. Electr. Eng. Comput. Sci. 2021, 24, 590–599. https://doi.org/10.11591/ijeecs.v24.i1.pp590-599 DOI: https://doi.org/10.11591/ijeecs.v24.i1.pp590-599

Chua, T.H.; Salam, I. Evaluation of machine learning algorithms in network-based intrusion detection using progressive dataset. Symmetry 2023, 15, 1251. https://doi.org/10.3390/sym15061251 DOI: https://doi.org/10.3390/sym15061251

Crespo-Martínez, I.S.; Campazas-Vega, A.; Guerrero-Higueras, Á.M.; Riego-DelCastillo, V.; Álvarez-Aparicio, C.; Fernández-Llamas, C. SQL injection attack detection in network flow data. Comput. Secur. 2023, 127, 103093. https://doi.org/10.1016/j.cose.2023.103093 DOI: https://doi.org/10.1016/j.cose.2023.103093

Ileberi, E.; Sun, Y.; Wang, Z. Performance evaluation of machine learning methods for credit card fraud detection using SMOTE and AdaBoost. IEEE Access 2021, 9, 165286–165294. https://doi.org/10.1109/ACCESS.2021.3134330 DOI: https://doi.org/10.1109/ACCESS.2021.3134330

Khan, M.M. Developing an AI-powered intrusion detection system for cloud infrastructure. J. Artif. Intell. Mach. Learn. Data Sci. 2024, 2, 1074–1080. https://doi.org/10.51219/JAIMLD/mohammed-mustafa-khan/255 DOI: https://doi.org/10.51219/JAIMLD/mohammed-mustafa-khan/255

Lew, J.; Shah, D.A.; Pati, S.; Cattell, S.; Zhang, M.; Sandhupatla, A.; Ng, C.; Goli, N.; Sinclair, M.D.; Rogers, T.G.; Aamodt, T.M. Analyzing machine learning workloads using a detailed GPU simulator. In Proceedings of the 2019 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS); IEEE: Madison, WI, USA, 2019; pp. 151–152. https://doi.org/10.1109/ISPASS.2019.00028 DOI: https://doi.org/10.1109/ISPASS.2019.00028

Lifandali, O.; Abghour, N.; Chiba, Z. Feature selection using a combination of ant colony optimization and random forest algorithms applied to an isolation forest-based intrusion detection system. Procedia Comput. Sci. 2023, 220, 796–805. https://doi.org/10.1016/j.procs.2023.03.106 DOI: https://doi.org/10.1016/j.procs.2023.03.106

Mahbooba, B.; Timilsina, M.; Sahal, R.; Serrano, M. Explainable artificial intelligence (XAI) to enhance trust management in intrusion detection systems using a decision tree model. Complexity 2021, 2021, 6634811. https://doi.org/10.1155/2021/6634811 DOI: https://doi.org/10.1155/2021/6634811

Maseer, Z.K.; Yusof, R.; Bahaman, N.; Mostafa, S.A.; Foozy, C.F.M. Benchmarking of machine learning for anomaly-based intrusion detection systems in the CICIDS2017 dataset. IEEE Access 2021, 9, 22351–22370. https://doi.org/10.1109/ACCESS.2021.3056614 DOI: https://doi.org/10.1109/ACCESS.2021.3056614

Mhamdi, L.; Isa, M.M. Securing SDN: Hybrid autoencoder-random forest for intrusion detection and attack mitigation. J. Netw. Comput. Appl. 2024, 225, 103868. https://doi.org/10.1016/j.jnca.2024.103868 DOI: https://doi.org/10.1016/j.jnca.2024.103868

Ozkan-Okay, M.; Samet, R.; Aslan, O.; Gupta, D. A comprehensive systematic literature review on intrusion detection systems. IEEE Access 2021, 9, 157727–157760. https://doi.org/10.1109/ACCESS.2021.3129336 DOI: https://doi.org/10.1109/ACCESS.2021.3129336

Parveen Sultana, H.; Shrivastava, N.; Dominic, D.D.; Nalini, N.; Balajee, J.M. Comparison of machine learning algorithms to build an optimized network intrusion detection system. J. Comput. Theor. Nanosci. 2019, 16, 2541–2549. https://doi.org/10.1166/jctn.2019.7929 DOI: https://doi.org/10.1166/jctn.2019.7929

Sadiq, S.; Eesa, A.S. Optimization algorithms for intrusion detection system: A review. Int. J. Res. Granthaalayah 2020, 8, 217–225. https://doi.org/10.29121/granthaalayah.v8.i8.2020.1031 DOI: https://doi.org/10.29121/granthaalayah.v8.i8.2020.1031

Silivery, A.K.; Rao Kovvur, R.M.; Solleti, R.; Kumar, L.S.; Madhu, B. A model for multi-attack classification to improve intrusion detection performance using deep learning approaches. Meas. Sens. 2023, 30, 100924. https://doi.org/10.1016/j.measen.2023.100924 DOI: https://doi.org/10.1016/j.measen.2023.100924

Singh Chinthalapudi, S. Detecting and mitigating SQL injection in .NET applications using AI-based anomaly detection. Int. J. Innov. Sci. Res. Technol. 2025, 10, 2582–2595. https://doi.org/10.38124/ijisrt/25mar1676 DOI: https://doi.org/10.38124/ijisrt/25mar1676

Smith, J.; Kevin, E. AI-powered intrusion detection systems for next-generation cloud. ResearchGate 2025. Available online: https://www.researchgate.net/publication/390448273

Sulaiman, N.S.; Nasir, A.; Othman, W.R.W.; Wahab, S.F.A.; Aziz, N.S.; Yacob, A.; Samsudin, N. Intrusion detection system techniques: A review. J. Phys. Conf. Ser. 2021, 1874, 012042. https://doi.org/10.1088/1742-6596/1874/1/012042 DOI: https://doi.org/10.1088/1742-6596/1874/1/012042

Wang, B.X.; Chen, J.L.; Yu, C.L. An AI-powered network threat detection system. IEEE Access 2022, 10, 54029–54037. https://doi.org/10.1109/ACCESS.2022.3175886 DOI: https://doi.org/10.1109/ACCESS.2022.3175886

Wang, C.; Sun, Y.; Lv, S.; Wang, C.; Liu, H.; Wang, B. Intrusion detection system based on one-class support vector machine and Gaussian mixture model. Electronics 2023, 12, 930. https://doi.org/10.3390/electronics12040930 DOI: https://doi.org/10.3390/electronics12040930

Waskle, S.; Parashar, L.; Singh, U. Intrusion detection system using PCA with random forest approach. In Proceedings of the 2020 IEEE International Conference on Electronics and Sustainable Communication Systems (ICESC); IEEE, 2020. https://doi.org/10.1109/ICESC48915.2020.9155656 DOI: https://doi.org/10.1109/ICESC48915.2020.9155656

Xu, W.; Fan, Y. Intrusion detection systems based on logarithmic autoencoder and XGBoost. Secur. Commun. Netw. 2022, 2022, 9068724. https://doi.org/10.1155/2022/9068724 DOI: https://doi.org/10.1155/2022/9068724

Al Hasan, R.A.; Hamza, E.K. An improved intrusion detection system using machine learning with singular value decomposition and principal component analysis. Int. J. Intell. Eng. Syst. 2023, 16, 25–38. https://doi.org/10.22266/ijies2023.0831.03 DOI: https://doi.org/10.22266/ijies2023.0831.03

Lanz, S.; Pignol, S.L.R.; Schmitt, P.; Wang, H.; Papaioannou, M.; Choudhary, G.; Dragoni, N. Optimizing Internet of Things honeypots with machine learning: A review. Appl. Sci. 2025, 15, 5251. https://doi.org/10.3390/app15105251 DOI: https://doi.org/10.3390/app15105251

Onyebueke, A.E.; David, A.; Munu, S. Network intrusion detection system using XGBoost and random forest algorithms. Asian J. Pure Appl. Math. 2023, 5, 1–?. https://doi.org/10.54254/2753-8818/31/20241171 DOI: https://doi.org/10.54254/2753-8818/31/20241171

Pashaei, A.; Akbari, M.E.; Zolfy Lighvan, M.; Charmin, A. Early intrusion detection system using a honeypot for industrial control networks. Results Eng. 2022, 16, 100576. https://doi.org/10.1016/j.rineng.2022.100576 DOI: https://doi.org/10.1016/j.rineng.2022.100576

Saranya, T.; Sridevi, S.; Deisy, C.; Chung, T.D.; Khan, M.K.A.A. Performance analysis of machine learning algorithms in intrusion detection systems: A review. Procedia Compute. Sci. 2020, 171, 1251–1260. https://doi.org/10.1016/j.procs.2020.04.133 DOI: https://doi.org/10.1016/j.procs.2020.04.133