Deep Reinforcement Learning for Adaptive Intrusion Detection in Software-Defined Networks
DOI:
https://doi.org/10.59746/733r3q77الكلمات المفتاحية:
Software-defined networks، intrusion detection system، deep reinforcement learning، dueling double dqn، prioritized experience replayالملخص
Cybersecurity threats against Software-Defined Networks (SDN) are continuously emerging, demanding improved intrusion detection capabilities that can quickly adapt to new threats. In this paper, we design a Deep Reinforcement Learning-based Intrusion Detection System (DRL-IDS) that leverages reinforcement learning to model network intrusion detection as a Markov decision process. We utilize Dueling Double Deep Q-Network (D3QN) with Prioritized Experience Replay to enable our IDS agent to perform online traffic classification in SDNs. Our proposed approach is evaluated on the comprehensive InSDN dataset in both binary and multi-class settings. We achieved 99.99% accuracy, precision, recall, and F1-score on binary detection tasks, with an AUC-ROC of 0.9988. In a multi-classification setting with seven different attack types (DDoS, DoS, Probe, BFA, BOTNET, Web-Attack, and U2R), we achieve an accuracy of 99.83% and F1-score of 99.97%. Our approach is compared with Random Forest, Decision Tree, K-Nearest Neighbors, and Multi-Layer Perceptron algorithms and exhibits comparable or better performance in all evaluation metrics. Adaptive reward with bonus rewards for detecting attacks and prioritized experience sampling allows tackling class imbalance problems. Our method can be implemented for intrusion detection systems (IDS) in software-defined networking (SDN) environments that demand real-time detection with low false-positive rates.
المراجع
[1] Farhan, M., Waheed ud din, H., Ullah, S., Hussain, M. S., Khan, M. A., Mazhar, T., et al. (2025). Network-based intrusion detection using deep learning technique. Scientific Reports, 15(1), 25550, https://doi.org/10.1038/s41598-025-08770-0
[2] Halbouni, A., Gunawan, T. S., Habaebi, M. H., Halbouni, M., Kartiwi, M., & Ahmad, R. (2022). CNN-LSTM: Hybrid Deep Neural Network for Network Intrusion Detection System. IEEE Access, 10, 99837-99849, https://doi.org/10.1109/ACCESS.2022.3206425
[3] Sewak, M., Sahay, S. K., & Rathore, H. (2023). Deep Reinforcement Learning in the Advanced Cybersecurity Threat Detection and Protection. Information Systems Frontiers, 25(2), 589-611, https://doi.org/10.1007/s10796-022-10333-x
[4] Altunay, H. C., & Albayrak, Z. (2023). A hybrid CNN+LSTM-based intrusion detection system for industrial IoT networks. Engineering Science and Technology, an International Journal, 38, 101322, https://doi.org/10.1016/j.jestch.2022.101322
[5] Liu, Y., Tsang, K. F., Wu, C. K., Wei, Y., Wang, H., & Zhu, H. (2023). IEEE P2668-Compliant Multi-Layer IoT-DDoS Defense System Using Deep Reinforcement Learning. IEEE Transactions on Consumer Electronics, 69(1), 49-64, https://doi.org/10.1109/TCE.2022.3213872
[6] Nazir, A., He, J., Zhu, N., Qureshi, S. S., Qureshi, S. U., Ullah, F., et al. (2024). A deep learning-based novel hybrid CNN-LSTM architecture for efficient detection of threats in the IoT ecosystem. Ain Shams Engineering Journal, 15(7), 102777, https://doi.org/10.1016/j.asej.2024.102777
[7] Mohamed, S., & Ejbali, R. (2023). Deep SARSA-based reinforcement learning approach for anomaly network intrusion detection system. International Journal of Information Security, 22(1), 235-247, https://doi.org/10.1007/s10207-022-00634-2
[8] Santos, R. R. D., Viegas, E. K., Santin, A. O., & Cogo, V. V. (2023). Reinforcement Learning for Intrusion Detection: More Model Longness and Fewer Updates. IEEE Transactions on Network and Service Management, 20(2), 2040-2055, https://doi.org/10.1109/TNSM.2022.3207094
[9] Al-Omar, B., & Trabelsi, Z. Intrusion Detection Using Attention-Based CNN-LSTM Model. In I. Maglogiannis, L. Iliadis, J. MacIntyre, & M. Dominguez (Eds.), Artificial Intelligence Applications and Innovations, Cham, 2023 (pp. 515-526): Springer Nature Switzerland. doi:10.1007/978-3-031-34111-3_43.
[10] Benaddi, H., Ibrahimi, K., Benslimane, A., Jouhari, M., & Qadir, J. (2022). Robust Enhancement of Intrusion Detection Systems Using Deep Reinforcement Learning and Stochastic Game. IEEE Transactions on Vehicular Technology, 71(10), 11089-11102, https://doi.org/10.1109/TVT.2022.3186834
[11] Dunmore, A., Jang-Jaccard, J., Sabrina, F., & Kwak, J. (2023). A Comprehensive Survey of Generative Adversarial Networks (GANs) in Cybersecurity Intrusion Detection. IEEE Access, 11, 76071-76094, https://doi.org/10.1109/ACCESS.2023.3296707
[12] Nguyen, T. T., & Reddi, V. J. (2023). Deep Reinforcement Learning for Cyber Security. IEEE Transactions on Neural Networks and Learning Systems, 34(8), 3779-3795, https://doi.org/10.1109/TNNLS.2021.3121870
[13] Hammar, K., & Stadler, R. (2024). Learning Near-Optimal Intrusion Responses Against Dynamic Attackers. IEEE Transactions on Network and Service Management, 21(1), 1158-1177, https://doi.org/10.1109/TNSM.2023.3293413
[14] Zhao, W., Mahmoud, Q. H., & Alwidian, S. (2023). Evaluation of GAN-Based Model for Adversarial Training. Sensors, 23(5), 2697, https://doi.org/10.3390/s23052697
[15] Nakıp, M., & Gelenbe, E. (2024). Online Self-Supervised Deep Learning for Intrusion Detection Systems. IEEE Transactions on Information Forensics and Security, 19, 5668-5683, https://doi.org/10.1109/TIFS.2024.3402148
[16] Kandhro, I. A., Alanazi, S. M., Ali, F., Kehar, A., Fatima, K., Uddin, M., et al. (2023). Detection of Real-Time Malicious Intrusions and Attacks in IoT Empowered Cybersecurity Infrastructures. IEEE Access, 11, 9136-9148, https://doi.org/10.1109/ACCESS.2023.3238664
[17] Chemmakha, M., Habibi, O., & Lazaar, M. (2024). Towards a Deep Learning Approach for IoT Attack Detection Based on a New Generative Adversarial Network Architecture and Gated Recurrent Unit. Journal of Network and Systems Management, 32(4), 96, https://doi.org/10.1007/s10922-024-09873-1
[18] Randhawa, R. H., Aslam, N., Alauthman, M., Khalid, M., & Rafiq, H. (2024). Deep reinforcement learning based Evasion Generative Adversarial Network for botnet detection. Future Gener. Comput. Syst., 150(C), 294–302, https://doi.org/10.1016/j.future.2023.09.011
[19] Kanimozhi, R., & Ramesh, P. S. (2025). Deep reinforcement learning-based intrusion detection scheme for software-defined networking. Sci Rep, 15(1), 38827, https://doi.org/10.1038/s41598-025-24869-w
[20] Ma, Y., Li, C., Wang, Y., & Wang, Y. (2025). Application of deep reinforcement learning algorithms for automatic threat detection and response in dynamic network environments to improve cybersecurity. Journal of Computational Methods in Sciences and Engineering, 25(3), 2112-2125, https://doi.org/10.1177/14727978241309550
[21] Shaikh, J. A., Wang, C., Sima, M. W. U., Arshad, M., Owais, M., Hassan, D. S. M., et al. (2025). A deep Reinforcement learning-based robust Intrusion Detection System for securing IoMT Healthcare Networks. [Original Research]. Frontiers in Medicine, Volume 12 - 2025, https://doi.org/10.3389/fmed.2025.1524286
[22] Jue, C., Hongyu, T., Meng, C., Haidong, P., & Xihe, Q. (2026). GCB-PPO2: A Hybrid Deep Reinforcement Learning Intrusion Detection System for Under-Represented Attack Categories in SDN. IEEE Transactions on Network Science and Engineering, 13, 84-101, https://doi.org/10.1109/TNSE.2025.3581689
