An Automatic Portscan Detection System with Adaptive Threshold Setting

Abstract

For the purpose of compromising hosts, attackers including infected hosts initially perform a portscan using IP addresses in order to find vulnerable hosts. Considerable research related to portscan detection has been done and many algorithms have been proposed and implemented in the network intrusion detection system (NIDS). In order to distinguish portscanners from remote hosts, most portscan detection algorithms use a fixed threshold that is manually managed by the network manager. Because the threshold is a constant, even though the network environment or the characteristics of traffic can change, many false positives and false negatives are generated by NIDS. This reduces the efficiency of NIDS and imposes a high processing burden on a network management system (NMS). In this paper, in order to address this problem, we propose an automatic portscan detection system using an fast increase slow decrease (FISD) scheme, that will automatically and adaptively set the threshold based on statistical data for traffic during prior time periods. In particular, we focus on reducing false positives rather than false negatives, while the threshold is adaptively set within a range between minimum and maximum values. We also propose a new portscan detection algorithm, rate of increase in the number of failed connection request (RINF), which is much more suitable for our system and shows better performance than other existing algorithms. In terms of the implementation, we compare our scheme with other two simple threshold estimation methods for an adaptive threshold setting scheme. Also, we compare our detection algorithm with other three existing approaches for portscan detection using a real traffic trace. In summary, we show that FISD results in less false positives than other schemes and RINF can fast and accurately detect portscanners. We also show that the proposed system, including our scheme and algorithm, provides good performance in terms of the rate of false positives.