A Survey on Network Security Traffic Analysis and Anomaly Detection Techniques
DOI:
https://doi.org/10.62677/IJETAA.2404117Keywords:
Network security, Anomaly detection, Machine learning, Deep learning, Hybrid methodsAbstract
With the increasingly severe network security situation, advanced network traffic anomaly detection techniques are urgently needed. This paper provides a comprehensive survey of the research status and latest progress in the field of network anomaly detection. Firstly, we introduce the basic concepts, common methods, and challenges of network traffic analysis, which lays the foundation for anomaly detection. Then, we systematically summarize the mainstream techniques in the anomaly detection field, including statistical methods, machine learning methods, deep learning methods, and behavior analysis methods, analyzing their basic principles, representative works, advantages and disadvantages, and applicable scenarios. Next, we focus on discussing the hybrid methods in the anomaly detection field, elaborating on the motivations, common strategies, and representative works of hybrid methods, and pointing out that hybrid methods are an important development direction for anomaly detection. In addition, the paper also summarizes the application effects of several types of methods in practical network security tasks and makes a quantitative comparison in tabular form. Finally, we prospect the future development trends of network anomaly detection techniques, proposing goals such as intelligentization, automation, federalization, and interpretability, while analyzing the challenges faced by anomaly detection, including data heterogeneity, complexity of security threats, model robustness, privacy protection, and interpretability. We argue that network anomaly detection requires interdisciplinary integration, strengthening of security big data governance, and a shift from passive defense to active immunity. As the strategic position of cyberspace security becomes increasingly prominent, driven by disruptive technologies such as big data, artificial intelligence, and blockchain, network anomaly detection will surely usher in new development opportunities and challenges.
Downloads
References
Verizon, “2023 Data Breach Investigations Report,” 2023.
Cybersecurity Ventures, “Cybercrime To Cost The World $10.5 Trillion Annually By 2025,” https://cybersecurityventures.com/cybercrimedamages-6-trillion-by-2021/, 2023-01-19.
Cisco, “Cisco Annual Internet Report (2018–2023) White Paper,” 2023.
W. Wei, L. Xie, L. Yang, et al., “A DDoS attack detection algorithm based on dynamic threshold,” Journal on Communications, vol. 35, no.11, pp. 37-45, 2014.
M. H. Bhuyan, D. K. Bhattacharyya, and J. K. Kalita, “Network Anomaly Detection: Methods, Systems and Tools,” IEEE Communications Surveys & Tutorials, vol. 16, no. 1, pp. 303-336, 2014.
M. Tan, Z. Nie, and C. Jin, “A Survey of Intelligent Network Anomaly Traffic Detection Techniques,” Journal of Software, vol. 29, no. 8, pp.2437-2468, 2018.
M. F. Umer, M. Sher, and Y. Bi, “Flow-based intrusion detection:Techniques and challenges,” Computers Security, vol. 70, pp. 238-254,2017.
X. Zhao, L. Tian, D. Cheng, et al., “A Botnet Detection Method Based on Statistical Features of Network Traffic,” Journal of Electronics Information Technology, vol. 34, no. 7, pp. 1519-1525, 2012.
J. Wang and I. C. Paschalidis, “Botnet Detection Based on Anomaly and Community Detection,” IEEE Transactions on Control of Network Systems, vol. 4, no. 2, pp. 392-404, 2017.
V. Chandola, A. Banerjee, and V. Kumar, “Anomaly Detection: A Survey,” ACM Computing Surveys, vol. 41, no. 3, pp. 1-58, 2009.
D. Cheng, X. Zhao, X. Yang, et al., “An Anomaly Network Traffic Detection Method Based on Multi-dimensional Features and Integrated Learning,” Journal on Communications, vol. 36, no. 8, pp. 74-84, 2015.
D. Kwon, H. Kim, J. Kim, et al., “A Survey of Deep Learning-Based Network Anomaly Detection,” Cluster Computing, vol. 22, no. 1, pp.949-961, 2019.
M. Wei, X. Wang, and G. Liu, “A Survey of Anomaly Detection in Massive Network Traffic,” Chinese Journal of Computers, vol. 43, no. 6, pp. 1145-1171, 2020.
M. Ahmed, A. Naser Mahmood, and J. Hu, “A Survey of Network Anomaly Detection Techniques,” Journal of Network and Computer Applications, vol. 60, pp. 19-31, 2016.
E. De la Hoz, E. De La Hoz, A. Ortiz, et al., “PCA filtering and probabilistic SOM for network intrusion detection,” Neurocomputing, vol.164, pp. 71-81, 2015.
J. Camacho, A. P´erez-Villegas, P. Garc´ıa-Teodoro, et al., “PCA-based multivariate statistical network monitoring for anomaly detection,” Computers Security, vol. 59, pp. 118-137, 2016.
G. Fernandes, J. J. P. C. Rodrigues, L. F. Carvalho, et al., “Network anomaly detection using IP flows with principal component analysis and ant colony optimization,” Journal of Network and Computer Applications, vol. 64, pp. 1-11, 2018.
R. J. Hyndman, E. Wang, and N. Laptev, “Large-Scale Unusual Time Series Detection,” IEEE International Conference on Data Mining Workshop, pp. 1616-1619, 2016.
F. Meng, Y. Fu, F. Lou, et al., “A Novel Unsupervised Anomaly Detection Approach for Intrusion Detection System,” IEEE Third International Conference on Data Science in Cyberspace, pp. 9-14, 2018.
S. Yadahalli and M. K. Nighot, “Adaboost based parameterized methods for wireless sensor network,” Procedia Computer Science, vol. 125, pp.470-476, 2018.
Z. Zhou, Machine Learning, Tsinghua University Press, Beijing, 2016.
S. Agrawal and J. Agrawal, “Survey on Anomaly Detection using Data Mining Techniques,” Procedia Computer Science, vol. 60, pp. 708-713,2015.
A. L. Buczak and E. Guven, “A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection,” IEEE Communications Surveys Tutorials, vol. 18, no. 2, pp. 1153-1176, 2016.
D. J. Weller-Fahy, B. J. Borghetti, and A. A. Sodemann, “A Survey of Distance and Similarity Measures Used Within Network Intrusion Anomaly Detection,” IEEE Communications Surveys Tutorials, vol. 17,no. 1, pp. 70-91, 2015.
S. Omar, A. Ngadi, and H. H. Jebur, “Machine Learning Techniques for Anomaly Detection: An Overview,” International Journal of Computer Applications, vol. 79, no. 2, pp. 33-41, 2013.
C. Xu, B. Wang, and D. Feng, “A Survey of Machine Learning Methods for Network Anomaly Detection,” Computer Science, vol. 45, no. 1, pp. 14-23, 2018.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436-444, 2015.
H. Yu, X. Wang, J. Ma, et al., “A Survey of Network Anomaly Detection Based on Deep Learning,” Journal of Computer Research and Development, vol. 39, no. 1, pp. 8-14, 2018.
L. Li, L. Zhang, W. Zhao, et al., “Network Anomaly Traffic Detection Method Based on Auto-encoder Neural Network,” Journal on Communications, vol. 38, no. 10, pp. 42-50, 2017.
N. Gao, L. Gao, Q. Gao, et al., “An Intrusion Detection Model Based on Deep Belief Networks,” 2014 Second International Conference on Advanced Cloud and Big Data, pp. 247-252, 2014.
A. H. Muna, N. Moustafa, and E. Sitnikova, “Identification of malicious activities in industrial internet of things based on deep learning models,” Journal of Information Security and Applications, vol. 41, pp. 1-11, 2018.
T. A. Tang, L. Mhamdi, D. McLernon, et al., “Deep learning approach for Network Intrusion Detection in Software Defined Networking,” 2016 International Conference on Wireless Networks and Mobile Communications, pp. 258-263, 2016.
B. Yu, D. L. Gray, J. Pan, et al., “Inline DGA Detection with Deep Networks,” IEEE International Conference on Data Mining Workshops, pp. 683-692, 2018.
Z. Qin, T. Li, Y. Wang, et al., “Network Traffic Analysis Using Refined Petri Nets: A Survey,” IEEE Access, vol. 6, pp. 54800-54826, 2018.
S. Garc´ıa, M. Grill, J. Stiborek, et al., “An empirical comparison of botnet detection methods,” Computers Security, vol. 45, pp. 100-123, 2014.
H. Wang, J. Gu, and G. Zhang, “FlowSym: Symmetric Behavioral Sequence Analysis for Botnet Detection,” 2018 17th IEEE International Conference on Trust, Security and Privacy in Computing and Communications/12th IEEE International Conference on Big Data Science and Engineering, pp. 422-429, 2018.
Y. Yang, J. Luo, Q. Dong, et al., “An Anomaly Detection Method Based on Hidden Markov Model,” Journal of Software, vol. 24, no. 2, pp. 243-255, 2013.
S. Nagaraja, P. Mittal, C. Y. Hong, et al., “BotGrep: Finding P2P Bots with Structured Graph Analysis,” Usenix Security Symposium, pp. 95-110, 2010.
Q. Yan, F. R. Yu, Q. Gong, et al., “Software-Defined Networking (SDN) and Distributed Denial of Service (DDoS) Attacks in Cloud Computing Environments: A Survey, Some Research Issues, and Challenges,” IEEE Communications Surveys Tutorials, vol. 18, no. 1, pp. 602-622, 2016.
Y. Xiang, K. Li, and W. Zhou, “Low-Rate DDoS Attacks Detection and Traceback by Using New Information Metrics,” IEEE Transactions on Information Forensics and Security, vol. 6, no. 2, pp. 426-437, 2011.
W. Meng, W. Li, and L. F. Kwok, “EFM: Enhancing the performance of signature-based network intrusion detection systems using enhanced filter mechanism,” Computers Security, vol. 43, pp. 189-204, 2014.
S. Wang, A. Dehghani, L. Li, et al., “BoT-CLEAN: A Proactive Scheme for Securing Proxy-Based Command and Control (C2) Channels of Botnets,” 2019 IEEE Conference on Communications and Network Security, pp. 1-9, 2019.
M. Zhang, B. Xu, S. Bai, et al., “A Deep Learning Method for Detecting Web Attacks Using a Specially Designed CNN,” Neural Information Processing, pp. 828-836, 2017.
A. Javaid, Q. Niyaz, W. Sun, et al., “A Deep Learning Approach for Network Intrusion Detection System,” Eai International Conference on Bio-Inspired Information and Communications Technologies, pp. 21-26, 2016.
N. Shone, T. N. Ngoc, V. D. Phai, et al., “A Deep Learning Approach o Network Intrusion Detection,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 2, no. 1, pp. 41-50, 2018.
Q. Yang, Y. Liu, T. Chen, et al., “Federated Machine Learning: Concept and Applications,” ACM Transactions on Intelligent Systems and Technology, vol. 10, no. 2, pp. 1-19, 2019.
W. Samek, T. Wiegand, and K. R. M¨uller, “Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models,” ITU Journal: ICT Discoveries, vol. 1, no. 1, pp. 39-48, 2017.
Q. Niyaz, W. Sun, and A. Y. Javaid, “A Deep Learning Based DDoS Detection System in Software-Defined Networking (SDN),” EAI Endorsed Transactions on Security and Safety, vol. 4, no. 12, pp. 1-12, 2017.
J. Li, B. Zhao, and C. Zhang, “Fuzzing: a survey,” Cybersecurity, vol. 1, no. 1, pp. 1-13, 2018.
M. Ring, S. Wunderlich, D. Grudl, et al., “Flow-based benchmark data sets for intrusion detection,” Proceedings of the 16th Euro
Downloads
Published
Issue
Section
Categories
License
Copyright (c) 2024 Weibao Zhang, Joan P. Lazaro (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
How to Cite
