Perhaps in place of Intelligent Intrusion Detection System (IDS) I should be talking about Cyber-security AI, but that is a very broad topic that cannot fit in this short article. Therefore, as time and space has allowed me today, I will narrow down to the specific subject of intrusion detection systems, their evolution to intelligent intrusion detection Systems, and why every organization should deploy one on its network and IT infrastructure.
Intrusion Detection Systems (IDS)
In the simplest terms, an Intrusion Detection Systems or just IDS helps in monitoring and identifying suspicious activities on a network (such as a failed login attempt, unauthorized system access, spam/phishing mails or malware threats). Traditionally, at the highest level, an intrusion detection system falls into one of the two categories, Host-based Intrusion Detection Systems (HIDS) and Network-based Intrusion Detection Systems (NIDS). The former being an individual device detecting a compromise and the latter detecting a compromise in transit over a network. NIDS can be further categorized into Anomaly and Signature based systems. Signature based systems form the mainstay of commercial network intrusion detection systems with anomaly based still largely a research concept. By signature based I mean that these systems follow a well crafted and laid down rules in the form of white-lists and blacklists to determine what constitutes an allowed (and harmless) network activity and what constitutes a suspicious or undesired activity. A human eye is less likely to register everything that happens on our networks, making IDS the only solution.
Now, these systems are well capable of dealing with known attacks and threats. However, times have changed. Today, more than ever, we are facing system and network threats that are way more sophisticated. With the attack base widening due to such emerging technologies as cloud computing, Internet of Things (IoT), Artificial Intelligence and machine learning just to mention a few, we need to look closely at our conventional network intrusion detection systems, and the question begs; Are our signature based NIDS still efficient when faced with new attack vectors? Or when the known attacks are modified purposely to get around the rules? The answer is a NO.
Apart from not being able to deal with unknown attacks, conventional signature based NIDS have one more drawback. According to Neustar (a cyber-security and risks research firm), on average, 26% of alerts raised by these systems in organizations are false positives. Actually, most of the alerts raised by the conventional IDSs require no action at all; others can be easily fixed, and only a handful need IT intervention. That is the case with security tools that produce large quantities of data to be analyzed without contextualizing potential threats, therefore contributing to data overload, alert fatigue and burnout among the IT security team.
“Cyber-security teams are increasingly drowning in data and are overwhelmed by the massive volume of alerts, many of them false positives. To ensure these high-value employees in mission critical roles are well-equipped to separate the signal from the noise, enterprises need a curated approach to security data that provides timely, actionable insights that are hyper relevant to their own organization and industry.” – Rodney Joffe, chairman of NISC and SVP and Fellow at Neustar.
Intelligent Intrusion Detection Systems
Intelligent Intrusion Detection Systems offer an alternative, and a solution to the above highlighted IDS drawbacks. Instead of burdening the organization’s IT personnel with the arduous task of processing, analyzing and responding to every alert raised, this work is taken up by these new evolving systems. This will give the team ample time to work on other projects that can be done better by humans. Most Intelligent IDSs are non-signature based and they run on machine learning algorithms and artificial neural networks. They don’t rely on rules, and are therefore way more efficient and accurate in classifying network traffic. That malicious shell code aimed at obtaining unauthorized command-line access to your computer systems and servers will not go undetected (Shell codes are frequently used as a payload in system penetration tools due to the enhanced access and further leverage they offer to attackers).
A Shell code can be extremely difficult to distinguish from benign network traffic. One study published in a UK science journal found out that signatures designed to match shell codes frequently also matched with other non shell code binaries e.g. DLLs as well as jpg image files. This problem of false positives on shell codes and signature based systems has been so prevalent that Microsoft had to discuss it at length in their patent of methods to detect malicious shell codes with the goal of reducing false positives. With an Intelligent IDS, this is well taken care of, and even though they don’t guarantee totally false positive free alerts, they reduce them to nearly unrecognizable levels.
Having said so, you need to boost your network and systems security by deploying an Intelligent IDS if you still haven’t. Though these systems are still very new and at the testing stages, some good examples to look out for are OWASP Intelligent Intrusion Detection System and Darktrace’s Enterprise Immune System