AI-IDS: Application of Deep Learning to Real-Time Web Intrusion Detection

Abstract

Deep Learning has been widely applied to problems in detecting various network attacks. However, no cases on network security have shown applications of various deep learning algorithms in real-time services beyond experimental conditions. Moreover, owing to the integration of high-performance computing, it is necessary to apply systems that can handle large-scale traffic. Given the rapid evolution of web-attacks, we implemented and applied our Artificial Intelligence-based Intrusion Detection System (AI-IDS). We propose an optimal convolutional neural network and long short-term memory network (CNN-LSTM) model, normalized UTF-8 character encoding for Spatial Feature Learning (SFL) to adequately extract the characteristics of real-time HTTP traffic without encryption, calculating entropy, and compression. We demonstrated its excellence through repeated experiments on two public datasets (CSIC-2010, CICIDS2017) and fixed real-time data. By training payloads that analyzed true or false positives with a labeling tool, AI-IDS distinguishes sophisticated attacks, such as unknown patterns, encoded or obfuscated attacks from benign traffic. It is a flexible and scalable system that is implemented based on Docker images, separating user-defined functions by independent images. It also helps to write and improve Snort rules for signature-based IDS based on newly identified patterns. As the model calculates the malicious probability by continuous training, it could accurately analyze unknown web-attacks.