December 15, 2016 Engineering

Integrating the Elastic Stack with ArcSight SIEM - Part 1

By Samir Bennacer

In this blog series we will provide an overview of how to extend and complement the capabilities of your existing SIEM to create an effective security analytics solution for your organization. For the purposes of example, we will demonstrate the use of an  X-Pack enabled Elastic Stack with one of the SIEM solutions...ArcSight.

The following demonstrates an example of Elasticsearch with the ArcSight SIEM. The existing ArcSight connector can be used to send data to Elasticsearch, with two possible approaches to configuration. The simplest solution is to add a CEF syslog destination to the ArcSight connector allowing it to send data to the Logstash TCP input.

In this blog we will cover the simpler former solution utilising the tcp output.  The supporting files for this example are provided here. These should be downloaded prior to following the steps below:

Setup the Elastic Stack

  1. Download and install Elasticsearch.
  2. Download and install Kibana
  3. Install X-Pack
  4. Download and install Logstash (version 5.1.1 or higher)
  5. Start Logstash using the config file and the template you downloaded

Setup the Elastic Stack with Docker

For a quick setup you can download an example docker-compose.yml definition to help you to install all the elastic stack with x-plugin ( step 1 to 5 ), then issue:

$ docker-compose up

But First, ensure that:

Importing ArcSight Data

  1. Configure ArcSight connectors to send data to Logstash
    1. Run the command ..<installdir>\current\bin\arcsight agentsetup
    2. Choose yes to start the ‘wizardmode’
    3. Choose ‘I want to add/remove/modify ArcSight Manager destinations’
    4. Choose ‘add new destination’
    5. Choose ‘CEF syslog’
    6. Add the information of the logstash host and port 5000 you prepared and choose the TCP protocol.
  1. Point your web browser at http://localhost:5601/ to open Kibana. You should be prompted to log in to Kibana. To log in, you can use the built-in ‘elastic’ user and the password ‘changeme’. NOTE: These are the same credentials used in the logstash.conf download from above. When you change them ensure you update your logstash configuration and restart the pipeline.
  2. Configure the index pattern cef-* and select the @timestamp and check the data in the discovery
  3. An example dashboard, provided here, is shown below.

The visualizations and the dashboard can be imported into Kibana through the Management > Saved Objects tab.

Whilst the above focuses on ArcSight, any device that supports the CEF data format and a TCP output can be configured to send the data directly to a Logstash instance ( via the TCP input ).  The reader is referred to device vendor specific documentation e.g. Example guide for F5.

In this blog series, we will also cover using ArcSight with Kafka, and X-Pack Alerting to notify when security events occur.

If you want to check out more about security analytics presentations, we suggest Security Analytics @ USAA, Tapping Out Security with FireEye, Hunting the Hackers by Cisco's Talos, Tinder: Keeping Your Data From Getting Swiped Right Away , and Cyber Security Log Analytics with Decision Lab.

Editor's Note: Like what you read? There's more where that came from. Part 2 continues the story with how to proactively monitor security data in Elasticsearch using X-Pack. Part 3 walks you through how to scale the architecture.