Tech Topics

Improve Security Analytics with the Elastic Stack, Wazuh, and IDS

By
Santiago Bassett
•
Mike Paquette

The Elastic Stack delivers security analytics capabilities that are widely used for threat detection, visibility, and incident response. The speed and scale at which Elasticsearch can index and search security-related information enable security analysts to work more efficiently, while Kibana dashboards provide wide visibility and enable interactive threat hunting. And the machine learning engine can automate the analysis of complex datasets, making it possible to spot intruders that otherwise would’ve gone unnoticed.

Popular Intrusion Detection Systems (IDS), such as Wazuh or Suricata, use a signature-based approach to threat detection. That is, they compare patterns found in files, logs, and network traffic against a database of patterns known to be associated with malicious activity, alerting when a match is found. They provide useful rulesets to analyze and correlate data, usually generating thousands or millions of alerts per day in a production environment.

Casting a wide net can ensure that all potential security events are caught, but it also adds the work of sifting through thousands (or millions) of alerts every day. Elastic machine learning features help reduce the noise by automatically identifying unusual behaviors. This is a clear use case where anomaly-based and signature-based technologies complement each other, making threat detection easier and investigations more efficient.

Wazuh, commonly deployed along with the Elastic Stack, is an open source host-based intrusion detection system (HIDS). It provides log analysis, file integrity monitoring, rootkit and vulnerability detection, configuration assessment and incident response capabilities. The Wazuh solution architecture is based on multi-platform lightweight agents that run on monitored systems, reporting to a centralized server where data analysis is done. In addition, it provides a complete Kibana plugin for configuration management, status monitoring, querying and alert data visualization.

On the other hand, Suricata is a free and open source network threat detection engine, capable of real time network intrusion detection (NIDS), inline intrusion prevention (NIPS), network security monitoring (NSM), and offline pcap processing. Suricata inspects the network traffic using its rules and signature language to match known threats, policy violations and malicious behaviors, and supports scripting for detection of complex threats.

In this blog post, we will provide an overview of how to detect intrusions using an integration of Wazuh and Suricata in combination with Elastic machine learning jobs to help prioritize investigations.

Integrating Wazuh, Suricata, and the Elastic Stack

For the purpose of this blog post, we have set up a lab environment where Wazuh agents have been deployed to several Internet-facing servers, monitoring system and application log data, file integrity, and system calls.

In addition, we are running a Suricata sensor that takes care of monitoring the network traffic. This sensor is often configured to monitor traffic via a network tap, mirroring port, or a SPAN (Switched Port Analyzer) port, but can also can be deployed directly to your servers.

In order to get the most out of these two tools, we decided to analyze Suricata alerts using Wazuh rules, unifying the alerts format and allowing us to do correlation (for example with threat intelligence sources) and trigger automatic responses.

This integration was done by configuring a Wazuh agent to read Suricata JSON output. This agent acts as a collector that forwards the Suricata NIDS alerts to the Wazuh server, where they are processed through Wazuh log analysis rules, resulting in new and enriched security events.

Now, as result of this work, both NIDS and HIDS alerts are sent to Elasticsearch via Filebeat (configured to read Wazuh alerts) and Logstash (also used for geolocation enrichment), where we will use machine learning jobs to detect anomalies and unusual behaviors.

Below is an example of a deployment of host-based and network-based IDS technologies, integrated together with Elastic Stack:

Security analytics with the Elastic Stack and Wazuh

Detecting Malicious Actors using Machine Learning Jobs

In our lab environment, we enabled all rules, and found out that, for just one of our Internet-facing web servers, our Wazuh agent is reporting somewhere from 4 to 10 thousand IDS alerts per day. They are mostly related to web attacks, authentication failures, configuration issues (detected using hardening checks), file integrity changes, or vulnerable software.

To make it easier for the security analysts, Wazuh IDS alerts are enriched with metadata, such as a level value or group, making it possible to filter by priority or type. In addition, the Wazuh Kibana plugin provides pre-configured dashboards with useful information regarding the agent status, configuration, and alerts. See screenshot below:

The information provided by Wazuh is certainly useful, but it still does not tell us about unusual behaviors. Here is where an Elastic machine learning job can help us.

Elastic machine learning allows us to create several type of “jobs.” The job is the basic element of machine learning analysis. In our case, we decided to create what is known as a “population analysis,” where we instructed the machine learning engine to build a baseline model of IP address typical behavior over a period of time, identifying the ones that are behaving abnormally compared to the rest of population.

More specifically, in our case we wanted to look for source IP addresses that, when compared to all other IP addresses, are responsible for an unusually high number of alert types. This can’t be done just by creating aggregation tables, as the anomalies occurred at a certain point in time, not known by our security analyst, and we have over 60 days of alert data (we detected some attacks for which their duration was no more than two minutes).

As a result of our population analysis, we spot several abnormal behaviors (in the form of a list of source IP addresses), which we decided to investigate. We used the “Anomaly Explorer” to learn when each one of those possible malicious actors were attacking our environment.

Investigating an Intrusion Attempt

As an example of a security analysis, we highlight the investigation of one of the IP addresses identified by the machine learning job. This particular IP address generated several NIDS and HIDS alerts in less than a minute, triggering an automated Wazuh action that blocked the IP address in the local firewall of our web server.

As shown in the screenshot above, Suricata NIDS detected malicious traffic from the indicated source IP address. Four rules matched incoming traffic, generating the following alerts:

  • ET DROP Dshield Block Listed Source group 1
  • ET CINS Active Threat Intelligence Poor Reputation IP group 77
  • SURICATA HTTP URI terminated by non-compliant character
  • SURICATA HTTP METHOD terminated by non-compliant character

The first two rules, part of Emerging Threats ruleset, indicate that the source IP address is known to have a poor reputation according to Threat Intelligence (TI) sources: Dshield and Active Threat Intelligence.

In addition, two other signatures detected anomalous HTTP activity coming from the same source IP address, most likely part of a scanning phase where the attacker gathers information looking for potential vulnerabilities.

The Wazuh HIDS component also triggered different alerts based on the analysis of the web server access logs. This approach, totally different from network packet inspection, ended up generating the following alerts:

  • IP address found in AlienVault reputation database
  • Host Blocked by firewall-drop.sh Active Response
  • Host Unblocked by firewall-drop.sh Active Response

The first alert indicates that the source IP address is also known to be part of yet another Threat Intelligence (TI) source: AlienVault OTX IP reputation database.

The second two alerts are triggered by the Wazuh Active Response module that, as result of the previously mentioned alerts, automatically added a firewall rule to block traffic from that particular source IP address for exactly 60 seconds (this is configurable). This action was enough to block scanning activities and caused the malicious actor desist from the intrusion attempt.

Conclusion

Employing both signature-based and anomaly-based intrusion detection, using technologies such as Wazuh, Suricata and Elastic machine learning, can make threat detection easier and investigations more efficient.

In addition, integrating host-based IDS (used to monitor systems at a host level) with network-based IDS (used to inspect network traffic) can also increase threat detection and security visibility. Wazuh makes this easy, as it can be used to integrate host and network IDS systems with the Elastic Stack, and can provide mechanism to run automated responses and block attacks in real-time.