Elastic SIEM for home and small business: Beats on CentOS

Note: The “SIEM for home and small business” blog series contains configurations relevant to the beta release of Elastic SIEM using Elastic Stack 7.4. We recommend using Elastic Stack 7.6 and newer, as Elastic SIEM was made generally available in 7.6. Please also note the Elastic SIEM solution mentioned in this post is now referred to as Elastic Security.

Hey, there. This is part five of the Elastic SIEM for home and small business blog series. If you haven’t read the first, second, and third blogs, you may want to before going any further. In the Getting started blog, we created our Elasticsearch Service deployment and started collecting data from one of our computers using Winlogbeat. In the Securing cluster access blog, we secured access to our cluster by restricting privileges for users and Beats. In the GeoIP data and Beats config blog, we created an ingest pipeline for GeoIP data and reviewed our Beats configurations.

Identifying our data collection needs on CentOS

In the first blog, we determined that we need to use Auditbeat, Filebeat, Packetbeat, and Winlogbeat to collect log files, activities of users and processes, and network data. We do not need to install all of those Beats applications on our Linux devices, only the Beats we need to collect data relevant to us. In this blog, we will install and configure Auditbeat, Filebeat, and Packetbeat on CentOS.

For data collection from our Linux systems, we will focus on activities of users and processes, log files, and network data. We will use Auditbeat to collect file integrity and system information, Filebeat to collect auditd and log data, and Packetbeat to collect system-specific network data. In this blog, we will review the Repositories for YUM guide in order to install Beats with Yum. In order to get started we need administrative access to our CentOS system, then we need to add the Beats repository for Yum, and a copy of our Beats common configuration settings.

Determining the event data we want to collect

We will primarily collect system information from Auditbeat since the System module for Auditbeat is fully implemented for Linux — meaning most of our information will come from Auditbeat. Also, while we have the option to use the Auditd module for Auditbeat to collect audit events, we will instead use the Auditd module for Filebeat as an example of how to configure modules in Filebeat. Regarding log collection with Filebeat, we can review the log files that exist in /var/log on our CentOS system and decide if there are any additional logs we want to collect.

The CentOS system we will collect data from is primarily used for Pi-hole, so we will leverage Packetbeat to capture DNS traffic. While there are many different ways to configure event collection from Pi-hole (a few examples are using Logstash with elk-hole or DNS-ELK, or by using an ECS compliant ingest pipeline), this blog will focus on data collection from CentOS while highlighting some example configuration options.

Using the GeoIP ingest pipeline

Before we start installing and configuring Beats on our CentOS system, I want to cover the GeoIP ingest processor issue in more detail. There is a workaround that will allow us to enrich data with GeoIP information, but it depends on many variables. Essentially, you will have to enable logging for each respective Beat, then look at the logs to confirm data is being sent to your Elasticsearch Service deployment.

In the GeoIP data and Beats config blog, we created an ingest pipeline for GeoIP data, then we commented out the output.elasticsearch.pipeline: geoip-info setting. The reason I disabled use of the GeoIP ingest processor was to reduce (or prevent) frustration. Quite simply, you can read through (and follow) Elasticsearch GitHub issue 46193 for more detail around why I have currently chosen to disable these settings.

At a high level, the GeoIP ingest processor does not allow an array of IP addresses. This is a problem for systems running both IPv4 and IPv6, as well as systems with multiple network adapters (both physical and virtual). Any time Beats passes an array of IPs to Elasticsearch while using the GeoIP ingest processor a Failed to publish events: temporary bulk send failure error message will be generated when netinfo.enabled: true is set on devices with multiple interfaces or addresses, Beats will include IP addresses and MAC addresses as fields host.ip and host.mac. This means the respective Beat will attempt to send an array of values for host.ip that causes the error.

While the example Beats common configurations have netinfo.enabled: false (instead of true) and commented out the output.elasticsearch.pipeline: geoip-info setting, the way to tell Beats to use the GeoIP ingest pipeline is to uncomment the output.elasticsearch.pipeline: geoip-info setting in your Beats configuration file.

If you decide to use the GeoIP ingest pipeline, you should also configure logging for each Beat. The reason we need to configure logging is that the Beat will run even though data is not being sent to Elasticsearch. For this blog, we will store our logs in the /var/log/elastic directory, but you will need to look for an Failed to publish events: temporary bulk send failure error message that will look similar to this:

ERROR    pipeline/output.go:121    Failed to publish events: temporary bulk send failure

Again, this has to do with passing an array of IPs to Elasticsearch using the GeoIP ingest processor. With that said, feel free to follow Elasticsearch GitHub issue 46193.

Configuring the Beats repository for YUM

The Beats repository for YUM guide provides instructions on how to add the Beats repository for YUM on our CentOS server. The guide tells us we need to download and install the public signing key, then create an elastic.repo file in our /etc/yum.repos.d/ directory.

[myuser@centos7pi ~]$ sudo rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
[sudo] password for myuser:
[myuser@centos7pi ~]$ echo "Now we will create the elastic.repo file as defined in the YUM repo guide"
# Now we will create the elastic.repo file as defined in the YUM repo guide
[myuser@centos7pi ~]$ sudo vi /etc/yum.repos.d/elastic.repo
[myuser@centos7pi ~]$ echo "Now we will validate the elastic.repo file"
# Now we will validate the elastic.repo file
[myuser@centos7pi ~]$ sudo cat /etc/yum.repos.d/elastic.repo
[elastic-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
[myuser@centos7pi ~]$

We can continue now that the repository is ready to use.

Auditbeat on CentOS

Installing Auditbeat

Since we have our Beats repository for YUM added, we will download and install Auditbeat by issuing the sudo yum install auditbeat command. Once we have our configuration file ready, we will also configure Auditbeat to start automatically during boot. After installing Auditbeat, do not start the service until we finish initial configuration.

[myuser@centos7pi ~]$ sudo yum install auditbeat
[sudo] password for myuser: 
Is this ok [y/d/N]: y
Downloading packages:
auditbeat-7.4.0-x86_64.rpm                                   |  21 MB  00:00:03
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : auditbeat-7.4.0-1.x86_64                                    1/1
  Verifying  : auditbeat-7.4.0-1.x86_64                                    1/1
Installed:
  auditbeat.x86_64 0:7.4.0-1
Complete!
[myuser@centos7pi ~]$

Remember, do not start the Auditbeat service until we finish initial configuration.

Configuring Auditbeat

Now that Auditbeat is installed, we need to update our auditbeat.yml configuration file. Since we have already determined our common Beats configurations, we only need to finalize our Auditbeat-specific options, which will be stored in the first section of the file. Make sure to read through the auditbeat.reference.yml page or the auditbeat.reference.yml file in the /etc/auditbeat directory. For clarity, the Beats common configuration sections will be stored below the Auditbeat-specific options.

If you plan on using the GeoIP ingest processor, you should configure logging for Auditbeat. Since we will store our log files in the /var/log/elastic directory, our logging configuration will look like this:

#=== Auditbeat logging ===
# Initially use `info` for the logging.level, set logging.level to `debug` if you see
# an `Failed to publish events: temporary bulk send failure` error message in the logs
logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/elastic
  name: auditbeat
  keepfiles: 7
  permissions: 0644

We will add the Auditbeat logging section to our configuration, but leave it commented out for now. With that said, after issuing sudo cat /etc/auditbeat/auditbeat.yml here is what I have set in auditbeat.yml:

##=== Auditbeat specific options ===
#===  Modules configuration ===
auditbeat.modules:
- module: file_integrity
  paths:
  - /bin
  - /usr/bin
  - /sbin
  - /usr/sbin
  - /etc
- module: system
  datasets:
    - user
    - login
  user.detect_password_changes: true
  period: 10s 
  state.period: 12h
- module: system
  datasets:
    - package
    - host
  period: 30m
  state.period: 12h
- module: system
  datasets:
    - socket
  socket.include_localhost: false
  period: 3s
- module: system
  datasets:
    - process 
  processors:
    - drop_event.when:
         or:
          - contains.event.action: "existing_process"
          - contains.event.action: "process_error"
    - add_process_metadata:
        match_pids: [process.ppid]
        target: process.parent
  period: 5s
#=== Auditbeat logging ===
# Configure logging for Auditbeat if you plan on using the GeoIP ingest processor
# Initially use `info` for the logging.level, set logging.level to `debug` if you see
# an `Failed to publish events: temporary bulk send failure` error message in the logs
#logging.level: info
#logging.to_files: true
#logging.files:
#  path: /var/log/elastic
#  name: auditbeat
#  keepfiles: 7
#  permissions: 0644
#=== Beats Common Configs Here ===

You may have noticed I have added two drop_event.when conditions as an example using the drop_event processor. You may have also noticed I specified state.period and period in the system module section. The period parameter controls the poll frequency while state.period controls the frequency of state updates. I also included the add process metadata processor to enrich events collected by the system module. For reference, this example configuration can be found in the examples GitHub repo under Security Analytics/SIEM-at-Home.

Next, we will add the settings from our Beats common configuration settings to our auditbeat.yml configuration file. We will use the settings from the General, Top Level Processor, Elastic Cloud, Xpack Monitoring, and Queue sections and add them to the end of our configuration file. Our completed auditbeat.yml file now looks like this:

#=== Auditbeat specific options ===
#===  Modules configuration ===
auditbeat.modules:
- module: file_integrity
  paths:
  - /bin
  - /usr/bin
  - /sbin
  - /usr/sbin
  - /etc
- module: system
  datasets:
    - user
    - login
  user.detect_password_changes: true
  period: 10s 
  state.period: 12h
- module: system
  datasets:
    - package
    - host
  period: 30m
  state.period: 12h
- module: system
  datasets:
    - socket
  socket.include_localhost: false
  period: 3s
- module: system
  datasets:
    - process 
  processors:
    - drop_event.when:
         or:
          - contains.event.action: "existing_process"
          - contains.event.action: "process_error"
    - add_process_metadata:
        match_pids: [process.ppid]
        target: process.parent
  period: 5s
#=== Auditbeat logging ===
# Configure logging for Auditbeat if you plan on using the GeoIP ingest processor
# Initially use `info` for the logging.level, set logging.level to `debug` if you see
# an `Failed to publish events: temporary bulk send failure` error message in the logs
#logging.level: info
#logging.to_files: true
#logging.files:
#  path: /var/log/elastic
#  name: auditbeat
#  keepfiles: 7
#  permissions: 0644
# Configuration applicable for all beats on a specific device
#=== General ===
name: C7PiHole
tags: ["myTag", "C7PiHole"]
fields:
  env: myEnv
  version: 12-18-2019
#=== Top Level Processor ===
processors:
  - add_host_metadata:
      # netinfo.enabled should be set to `false` until GitHub issue
      # https://github.com/elastic/elasticsearch/issues/46193 is resolved
      netinfo.enabled: false
      Geo: # These Geo configurations are optional
        location: 40.7128, -74.0060
        continent_name: North America
        country_iso_code: US
        region_name: New York
        region_iso_code: US-NY
        city_name: New York City
        name: myHomeLocation
  - add_locale: ~
#  - add_cloud_metadata: ~
  - add_fields:
      #when.network.source.ip: 10.101.101.0/24
      when.network.source.ip: private
      fields:
        source.geo.location:
          lat: 40.7128
          lon: -74.0060
        source.geo.continent_name: North America
        source.geo.country_iso_code: US
        source.geo.region_name: New York
        source.geo.region_iso_code: US-NY
        source.geo.city_name: New York City
        source.geo.name: myHomeLocation
      target: ''
  - add_fields:
      #when.network.destination.ip: 10.101.101.0/24
      when.network.destination.ip: private
      fields:
        destination.geo.location:
          lat: 40.7128
          lon: -74.0060
        destination.geo.continent_name: North America
        destination.geo.country_iso_code: US
        destination.geo.region_name: New York
        destination.geo.region_iso_code: US-NY
        destination.geo.city_name: New York City
        destination.geo.name: myHomeLocation
      target: ''
#=== Elastic Cloud ===
# These settings simplify using beats with the Elastic Cloud (https://cloud.elastic.co/).
cloud.id: "My_Elastic_Cloud_Deployment:abcdefghijklmnopqrstuvwxyz1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ"
cloud.auth: "home_beats:0987654321abcDEF"
# The geoip-info pipeline is used to enrich GeoIP information in Elasticsearch
# You must configure the pipeline in Elasticsearch before enabling the pipeline in Beats.
# The `output.elasticsearch.pipeline: geoip-info` setting should be commented out until
# until GitHub issue https://github.com/elastic/elasticsearch/issues/46193 is resolved
#output.elasticsearch.pipeline: geoip-info
# The `max_retries` setting is the number of times to retry publishing an event after 
# a publishing failure. After the specified number of retries, the events are typically dropped.
output.elasticsearch.max_retries: 5
# When deploying beats to multiple systems or locations, uncomment the following 
# setup.template.* and setup.ilm.* configurations after running the beats setup command
# Otherwise, uncomment the following configurations for beats that will only publish data
#setup.template.enabled: false
#setup.ilm.check_exists: false
#setup.ilm.overwrite: false
#=== Xpack Monitoring ===
# When monitoring is enabled, the beat will ship monitoring data to the cluster
monitoring.enabled: true
#=== Queue ===
# See the 'Configure the internal queue' documentation for each Beat before 
# configuring the queue. Note that only one queue type can be configured.
# You need to uncomment the specific queue type you decide to use.
# The `queue.mem` settings will cache events to memory in case access to the
# Elasticsearch cluster, via the internet, is unavailable (internet outage)
#queue.mem:
#  events: 4096
#  flush.min_events: 512
#  flush.timeout: 1s
# The `queue.spool` settings will cache events to disk in case the system is offline
# NOTE: The file spool queue is a beta functionality in 7.4
queue.spool:
  file:
    size: 512MiB
    page_size: 16KiB
    prealloc: ~
  write:
    buffer_size: 10MiB
    flush.timeout: 1s
    flush.min.events: 1024

Once the file is updated, don’t forget to save the changes made to auditbeat.yml before proceeding.

Starting Auditbeat

Now that the configuration is updated, it's time to set up Auditbeat with our Elasticsearch Service deployment. Running the setup command only needs to be performed once for each beat (for instance only once for Auditbeat 7.4.0). In our console, we will navigate to the /usr/share/auditbeat directory, then issue the sudo bin/auditbeat -c /etc/auditbeat/auditbeat.yml -path.home /usr/share/auditbeat setup command. Then we will confirm we receive “Index setup finished” and “Loaded dashboards” messages (shown below).

[myuser@centos7pi ~]$ cd /usr/share/auditbeat
[myuser@centos7pi auditbeat]$ sudo bin/auditbeat -c /etc/auditbeat/auditbeat.yml -path.home /usr/share/auditbeat setup
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
[myuser@centos7pi auditbeat]$

Now that we’ve set up the index and loaded the dashboards, we'll enable the Auditbeat service then start the Auditbeat service.

[myuser@centos7pi auditbeat]$ sudo systemctl enable auditbeat
Created symlink from /etc/systemd/system/multi-user.target.wants/auditbeat.service to /usr/lib/systemd/system/auditbeat.service.
[myuser@centos7pi auditbeat]$ sudo systemctl start auditbeat
[myuser@centos7pi auditbeat]$ sudo systemctl status auditbeat
● auditbeat.service - Audit the activities of users and processes on your system.
   Loaded: loaded (/usr/lib/systemd/system/auditbeat.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2019-12-11 16:39:34 CST; 22s ago
     Docs: https://www.elastic.co/products/beats/auditbeat
 Main PID: 12558 (auditbeat)
   CGroup: /system.slice/auditbeat.service
           └─12558 /usr/share/auditbeat/bin/auditbeat -e -c /etc/auditbeat/auditbeat.yml -path.home /...
Dec 11 16:39:54 centos7pi.my.domain auditbeat[12558]: 2019-12-11T16:39:54.338-0600        IN...se
Hint: Some lines were ellipsized, use -l to show in full.
[myuser@centos7pi auditbeat]$ cd ~
[myuser@centos7pi ~]$

We will navigate back to our home directory in our console so we can install and configure Filebeat and Packetbeat on this CentOS system. If you are using the GeoIP ingest pipeline and do not see data coming from Auditbeat for this device in your Elastic cluster, you should uncomment the logging section and restart Auditbeat.

Filebeat on CentOS

Installing Filebeat

Since we have our Beats repository for YUM added, we will download and install Filebeat by issuing the sudo yum install filebeat command. Once we have our configuration file ready, we will also configure Filebeat to start automatically during boot. After installing Filebeat, do not start the service until we finish initial configuration.

[myuser@centos7pi ~]$ sudo yum install filebeat
[sudo] password for myuser: 
Is this ok [y/d/N]: y
Downloading packages:
filebeat-7.4.0-x86_64.rpm                                   |  23 MB  00:00:04
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : filebeat-7.4.0-1.x86_64                                    1/1
  Verifying  : filebeat-7.4.0-1.x86_64                                    1/1
Installed:
  filebeat.x86_64 0:7.4.0-1
Complete!
[myuser@centos7pi ~]$

Remember, do not start the Filebeat service until we finish initial configuration.

Configuring Filebeat

Now that Filebeat is installed, we need to update our filebeat.yml configuration file. Since we have already determined our common Beats configurations, we only need to finalize our Filebeat-specific options, which will be stored in the first section of the file. Make sure to read through the filebeat.reference.yml page or the filebeat.reference.yml file in the /etc/filebeat directory. For clarity, the Beats common configuration sections will be stored below the Filebeat specific options.

If you plan on using the GeoIP ingest processor, you should configure logging for Filebeat. Since we will store our log files in the /var/log/elastic directory, our logging configuration will look like this:

#=== Filebeat logging ===
# Initially use `info` for the logging.level, set logging.level to `debug` if you see
# an `Failed to publish events: temporary bulk send failure` error message in the logs
logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/elastic
  name: filebeat
  keepfiles: 7
  permissions: 0644

We will add the Filebeat logging section to our configuration but leave it commented out for now. With that said, after issuing sudo cat /etc/filebeat/filebeat.yml here is what I have set in filebeat.yml:

#=== Filebeat specific options ===
#=== Filebeat modules ===
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
  reload.period: 30s
#=== Filebeat inputs ===
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/yum.log
  close_inactive: 1m
- type: log
  enabled: true
  paths:
    - /var/log/pihole.log
  fields:
    app: pihole
    name: pihole
- type: log
  enabled: true
  paths:
    - /var/log/pihole-FTL.log
  fields:
    app: pihole
    name: pihole-FTL
- type: log
  enabled: true
  paths:
    - /var/log/pihole_updateGravity.log
  fields:
    app: pihole
    name: pihole-updateGravity
- type: log
  enabled: true
  paths:
    - /var/log/lighttpd/*.log
  fields:
    app: pihole
    name: pihole-lighttpd
#=== Filebeat logging ===
# Configure logging for Filebeat if you plan on using the GeoIP ingest processor
# Initially use `info` for the logging.level, set logging.level to `debug` if you see
# an `Failed to publish events: temporary bulk send failure` error message in the logs
#logging.level: info
#logging.to_files: true
#logging.files:
#  path: /var/log/elastic
#  name: filebeat
#  keepfiles: 7
#  permissions: 0644
#=== Beats Common Configs Here ===

You may have noticed the entry for Yum logs contains a modified close_inactive setting — this configuration example tells Filebeat to close the file handle if the yum log has not been harvested for one minute. You may also have noticed the four entries for Pi-hole logs that will enrich our Pi-hole data with the app and name fields for the individual log inputs. For reference, this example configuration can be found in the examples GitHub repo under Security Analytics/SIEM-at-Home.

Next, we will add the settings from our Beats common configuration settings to our filebeat.yml configuration file. We will use the settings from the General, Top Level Processor, Elastic Cloud, Xpack Monitoring, and Queue sections and add them to the end of our configuration file. Our completed filebeat.yml file now looks like this:

#=== Filebeat specific options ===
#=== Filebeat modules ===
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
  reload.period: 30s
#=== Filebeat inputs ===
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/yum.log
  close_inactive: 1m
- type: log
  enabled: true
  paths:
    - /var/log/pihole.log
  fields:
    app: pihole
    name: pihole
- type: log
  enabled: true
  paths:
    - /var/log/pihole-FTL.log
  fields:
    app: pihole
    name: pihole-FTL
- type: log
  enabled: true
  paths:
    - /var/log/pihole_updateGravity.log
  fields:
    app: pihole
    name: pihole-updateGravity
- type: log
  enabled: true
  paths:
    - /var/log/lighttpd/*.log
  fields:
    app: pihole
    name: pihole-lighttpd
#=== Filebeat logging ===
# Configure logging for Filebeat if you plan on using the GeoIP ingest processor
# Initially use `info` for the logging.level, set logging.level to `debug` if you see
# an `Failed to publish events: temporary bulk send failure` error message in the logs
#logging.level: info
#logging.to_files: true
#logging.files:
#  path: /var/log/elastic
#  name: filebeat
#  keepfiles: 7
#  permissions: 0644
# Configuration applicable for all beats on a specific device
#=== General ===
name: C7PiHole
tags: ["myTag", "C7PiHole"]
fields:
  env: myEnv
  version: 12-18-2019
#=== Top Level Processor ===
processors:
  - add_host_metadata:
      # netinfo.enabled should be set to `false` until GitHub issue
      # https://github.com/elastic/elasticsearch/issues/46193 is resolved
      netinfo.enabled: false
      Geo: # These Geo configurations are optional
        location: 40.7128, -74.0060
        continent_name: North America
        country_iso_code: US
        region_name: New York
        region_iso_code: US-NY
        city_name: New York City
        name: myHomeLocation
  - add_locale: ~
#  - add_cloud_metadata: ~
  - add_fields:
      #when.network.source.ip: 10.101.101.0/24
      when.network.source.ip: private
      fields:
        source.geo.location:
          lat: 40.7128
          lon: -74.0060
        source.geo.continent_name: North America
        source.geo.country_iso_code: US
        source.geo.region_name: New York
        source.geo.region_iso_code: US-NY
        source.geo.city_name: New York City
        source.geo.name: myHomeLocation
      target: ''
  - add_fields:
      #when.network.destination.ip: 10.101.101.0/24
      when.network.destination.ip: private
      fields:
        destination.geo.location:
          lat: 40.7128
          lon: -74.0060
        destination.geo.continent_name: North America
        destination.geo.country_iso_code: US
        destination.geo.region_name: New York
        destination.geo.region_iso_code: US-NY
        destination.geo.city_name: New York City
        destination.geo.name: myHomeLocation
      target: ''
#=== Elastic Cloud ===
# These settings simplify using beats with the Elastic Cloud (https://cloud.elastic.co/).
cloud.id: "My_Elastic_Cloud_Deployment:abcdefghijklmnopqrstuvwxyz1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ"
cloud.auth: "home_beats:0987654321abcDEF"
# The geoip-info pipeline is used to enrich GeoIP information in Elasticsearch
# You must configure the pipeline in Elasticsearch before enabling the pipeline in Beats.
# The `output.elasticsearch.pipeline: geoip-info` setting should be commented out until
# until GitHub issue https://github.com/elastic/elasticsearch/issues/46193 is resolved
#output.elasticsearch.pipeline: geoip-info
# The `max_retries` setting is the number of times to retry publishing an event after 
# a publishing failure. After the specified number of retries, the events are typically dropped.
output.elasticsearch.max_retries: 5
# When deploying beats to multiple systems or locations, uncomment the following 
# setup.template.* and setup.ilm.* configurations after running the beats setup command
# Otherwise, uncomment the following configurations for beats that will only publish data
#setup.template.enabled: false
#setup.ilm.check_exists: false
#setup.ilm.overwrite: false
#=== Xpack Monitoring ===
# When monitoring is enabled, the beat will ship monitoring data to the cluster
monitoring.enabled: true
#=== Queue ===
# See the 'Configure the internal queue' documentation for each Beat before 
# configuring the queue. Note that only one queue type can be configured.
# You need to uncomment the specific queue type you decide to use.
# The `queue.mem` settings will cache events to memory in case access to the
# Elasticsearch cluster, via the internet, is unavailable (internet outage)
#queue.mem:
#  events: 4096
#  flush.min_events: 512
#  flush.timeout: 1s
# The `queue.spool` settings will cache events to disk in case the system is offline
# NOTE: The file spool queue is a beta functionality in 7.4
queue.spool:
  file:
    size: 512MiB
    page_size: 16KiB
    prealloc: ~
  write:
    buffer_size: 10MiB
    flush.timeout: 1s
    flush.min.events: 1024

Once the file is updated, don’t forget to save the changes made to filebeat.yml before proceeding.

Configuring Filebeat Modules

Next, we need to enable a few Filebeat Modules, which will simplify the collection, parsing, and visualization of common log formats. Since we installed Filebeat with YUM, the default modules are disabled but available in the /etc/filebeat/modules.d/ directory. For your awareness, the file name of the modules that are disabled end in .disabled. On this system we will use the auditd and system modules.

[myuser@centos7pi ~]$ sudo filebeat modules list
Enabled:
Disabled:
#[...]
[myuser@centos7pi ~]$ sudo filebeat modules enable auditd
Enabled auditd
[myuser@centos7pi ~]$ sudo filebeat modules enable system
Enabled system
[myuser@centos7pi ~]$ sudo filebeat modules list
Enabled:
auditd
system
Disabled:
#[...]
[myuser@centos7pi ~]$

We can continue now that the auditd and system modules are enabled.

Starting Filebeat

Now that the configuration is updated, it is time to set up Filebeat with our Elasticsearch Service deployment. Running the setup command only needs to be performed once for each beat (for instance, only once for Filebeat 7.4.0). In our console, we will navigate to the /usr/share/filebeat directory then issue the sudo bin/filebeat -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat setup command. Then we will confirm we receive “Index setup finished”, “Loaded dashboards”, “Loaded machine learning job configurations”, and “Loaded Ingest pipelines” messages (shown below).

[myuser@centos7pi ~]$ cd /usr/share/filebeat
[myuser@centos7pi filebeat]$ sudo bin/filebeat -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat setup
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
Loaded machine learning job configurations
Loaded Ingest pipelines
[myuser@centos7pi filebeat]$

Now that we’ve set up the index and loaded the dashboards, we can enable the Filebeat service and then start the Filebeat service.

[myuser@centos7pi filebeat]$ sudo systemctl enable filebeat
Created symlink from /etc/systemd/system/multi-user.target.wants/filebeat.service to /usr/lib/systemd/system/filebeat.service.
[myuser@centos7pi filebeat]$ sudo systemctl start filebeat
[myuser@centos7pi filebeat]$ sudo systemctl status filebeat
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
   Loaded: loaded (/usr/lib/systemd/system/filebeat.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2019-12-11 22:34:36 CST; 5s ago
     Docs: https://www.elastic.co/products/beats/filebeat
 Main PID: 12869 (filebeat)
   CGroup: /system.slice/filebeat.service
           └─12869 /usr/share/filebeat/bin/filebeat -e -c /etc/filebeat/filebeat.yml -path.home /usr/...
Dec 11 22:34:40 centos7pi.my.domain filebeat[12869]: 2019-12-11T22:34:40.351-0600        INF...ed
Hint: Some lines were ellipsized, use -l to show in full.
[myuser@centos7pi filebeat]$ cd ~
[myuser@centos7pi ~]$

We will navigate back to our home directory in our console so that we can install and configure Packetbeat on this CentOS system. If you are using the GeoIP ingest pipeline and do not see data coming from Filebeat for this device in your Elastic cluster, you should uncomment the logging section and restart Filebeat.

Packetbeat on CentOS

While it is ideal to deploy Packetbeat to either existing application servers or dedicated servers (to get traffic from mirror ports or tap devices), we will leverage Packetbeat to collect DNS traffic since we are running Pi-hole on this system.

Installing Packetbeat

Since we have our Beats repository for YUM added, we will download and install Packetbeat by issuing the sudo yum install packetbeat command. Once we have our configuration file ready, we will also configure Packetbeat to start automatically during boot. After installing Packetbeat, do not start the service until we finish initial configuration.

[myuser@centos7pi ~]$ sudo yum install packetbeat
[sudo] password for myuser: 
Is this ok [y/d/N]: y
Downloading packages:
packetbeat-7.4.0-x86_64.rpm                                   |  20 MB  00:00:04
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : packetbeat-7.4.0-1.x86_64                                    1/1
  Verifying  : packetbeat-7.4.0-1.x86_64                                    1/1
Installed:
  packetbeat.x86_64 0:7.4.0-1
Complete!
[myuser@centos7pi ~]$

Remember, do not start the Packetbeat service until we finish initial configuration.

Configuring Packetbeat

Now that Packetbeat is installed, we need to update our packetbeat.yml configuration file. Since we have already determined our common Beats configurations, we only need to finalize our Packetbeat-specific options, which will be stored in the first section of the file. Make sure to read through the packetbeat.reference.yml page or the packetbeat.reference.yml file in the /etc/packetbeat directory. For clarity, the Beats common configuration sections will be stored below the Packetbeat-specific options.

If you plan on using the GeoIP ingest processor, you should configure logging for Packetbeat. Since we will store our log files in the /var/log/elastic directory, our logging configuration will look like this:

#=== Packetbeat logging ===
# Initially use `info` for the logging.level, set logging.level to `debug` if you see
# an `Failed to publish events: temporary bulk send failure` error message in the logs
logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/elastic
  name: packetbeat
  keepfiles: 7
  permissions: 0644

We will add the Packetbeat logging section to our configuration but leave it commented out for now. With that said, after issuing sudo cat /etc/packetbeat/packetbeat.yml here is what I have set in packetbeat.yml:

#=== Packetbeat specific options ===
#=== Network device ===
# Select the network interface to sniff the data. On Linux, you can use the
# "any" keyword to sniff on all connected interfaces.
packetbeat.interfaces.device: any
#=== Flows ===
packetbeat.flows:
  timeout: 30s
  period: 10s
#=== Transaction protocols ===
# For more information on the transaction protocols, see
# https://www.elastic.co/guide/en/beats/packetbeat/7.4/configuration-protocols.html
packetbeat.protocols:
- type: icmp
  # Enable ICMPv4 and ICMPv6 monitoring. Default: false
  enabled: true
- type: dhcpv4
  # Configure the DHCP for IPv4 ports.
  ports: [67, 68]
  send_request: true
  send_response: true
- type: dns
  # Configure the ports where to listen for DNS traffic. You can disable
  # the DNS protocol by commenting out the list of ports.
  ports: [53]
  include_authorities: true
  include_additionals: true
  send_request: true
  send_response: true
- type: http
  # Configure the ports where to listen for HTTP traffic. You can disable
  # the HTTP protocol by commenting out the list of ports.
  ports: [80, 8080, 8000, 5000, 8002]
- type: tls
  # Configure the ports where to listen for TLS traffic. You can disable
  # the TLS protocol by commenting out the list of ports.
  ports:
    - 443   # HTTPS
    - 993   # IMAPS
    - 995   # POP3S
    - 5223  # XMPP over SSL
    - 8443
    - 8883  # Secure MQTT
    - 9243  # Elasticsearch
#=== Packetbeat logging ===
# Configure logging for Packetbeat if you plan on using the GeoIP ingest processor
# Initially use `info` for the logging.level, set logging.level to `debug` if you see
# an `Failed to publish events: temporary bulk send failure` error message in the logs
#logging.level: info
#logging.to_files: true
#logging.files:
#  path: /var/log/elastic
#  name: packetbeat
#  keepfiles: 7
#  permissions: 0644
#=== Beats Common Configs Here ===

You may have noticed the include_authorities: true and include_additionals: true entries in the DNS transaction protocol section, which is an example using DNS-specific configuration options. You may have also noticed the send_request: true and send_response: true entries in both the DHCP and DNS transaction protocol sections, which are examples using common protocol options. For reference, this example configuration can be found in the examples GitHub repo under Security Analytics/SIEM-at-Home.

Next, we will add the settings from our Beats common configuration settings to our packetbeat.yml configuration file. We will use the settings from the General, Top Level Processor, Elastic Cloud, Xpack Monitoring, and Queue sections and add them to the end of our configuration file. Our completed packetbeat.yml file now looks like this:

#=== Packetbeat specific options ===
#=== Network device ===
# Select the network interface to sniff the data. On Linux, you can use the
# "any" keyword to sniff on all connected interfaces.
packetbeat.interfaces.device: any
#=== Flows ===
packetbeat.flows:
  timeout: 30s
  period: 10s
#=== Transaction protocols ===
# For more information on the transaction protocols, see
# https://www.elastic.co/guide/en/beats/packetbeat/7.4/configuration-protocols.html
packetbeat.protocols:
- type: icmp
  # Enable ICMPv4 and ICMPv6 monitoring. Default: false
  enabled: true
- type: dhcpv4
  # Configure the DHCP for IPv4 ports.
  ports: [67, 68]
  send_request: true
  send_response: true
- type: dns
  # Configure the ports where to listen for DNS traffic. You can disable
  # the DNS protocol by commenting out the list of ports.
  ports: [53]
  include_authorities: true
  include_additionals: true
  send_request: true
  send_response: true
- type: http
  # Configure the ports where to listen for HTTP traffic. You can disable
  # the HTTP protocol by commenting out the list of ports.
  ports: [80, 8080, 8000, 5000, 8002]
- type: tls
  # Configure the ports where to listen for TLS traffic. You can disable
  # the TLS protocol by commenting out the list of ports.
  ports:
    - 443   # HTTPS
    - 993   # IMAPS
    - 995   # POP3S
    - 5223  # XMPP over SSL
    - 8443
    - 8883  # Secure MQTT
    - 9243  # Elasticsearch
#=== Packetbeat logging ===
# Configure logging for Packetbeat if you plan on using the GeoIP ingest processor
# Initially use `info` for the logging.level, set logging.level to `debug` if you see
# an `Failed to publish events: temporary bulk send failure` error message in the logs
#logging.level: info
#logging.to_files: true
#logging.files:
#  path: /var/log/elastic
#  name: packetbeat
#  keepfiles: 7
#  permissions: 0644
# Configuration applicable for all beats on a specific device
#=== General ===
name: C7PiHole
tags: ["myTag", "C7PiHole"]
fields:
  env: myEnv
  version: 12-18-2019
#=== Top Level Processor ===
processors:
  - add_host_metadata:
      # netinfo.enabled should be set to `false` until GitHub issue
      # https://github.com/elastic/elasticsearch/issues/46193 is resolved
      netinfo.enabled: false
      Geo: # These Geo configurations are optional
        location: 40.7128, -74.0060
        continent_name: North America
        country_iso_code: US
        region_name: New York
        region_iso_code: US-NY
        city_name: New York City
        name: myHomeLocation
  - add_locale: ~
#  - add_cloud_metadata: ~
  - add_fields:
      #when.network.source.ip: 10.101.101.0/24
      when.network.source.ip: private
      fields:
        source.geo.location:
          lat: 40.7128
          lon: -74.0060
        source.geo.continent_name: North America
        source.geo.country_iso_code: US
        source.geo.region_name: New York
        source.geo.region_iso_code: US-NY
        source.geo.city_name: New York City
        source.geo.name: myHomeLocation
      target: ''
  - add_fields:
      #when.network.destination.ip: 10.101.101.0/24
      when.network.destination.ip: private
      fields:
        destination.geo.location:
          lat: 40.7128
          lon: -74.0060
        destination.geo.continent_name: North America
        destination.geo.country_iso_code: US
        destination.geo.region_name: New York
        destination.geo.region_iso_code: US-NY
        destination.geo.city_name: New York City
        destination.geo.name: myHomeLocation
      target: ''
#=== Elastic Cloud ===
# These settings simplify using beats with the Elastic Cloud (https://cloud.elastic.co/).
cloud.id: "My_Elastic_Cloud_Deployment:abcdefghijklmnopqrstuvwxyz1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ"
cloud.auth: "home_beats:0987654321abcDEF"
# The geoip-info pipeline is used to enrich GeoIP information in Elasticsearch
# You must configure the pipeline in Elasticsearch before enabling the pipeline in Beats.
# The `output.elasticsearch.pipeline: geoip-info` setting should be commented out until
# until GitHub issue https://github.com/elastic/elasticsearch/issues/46193 is resolved
#output.elasticsearch.pipeline: geoip-info
# The `max_retries` setting is the number of times to retry publishing an event after 
# a publishing failure. After the specified number of retries, the events are typically dropped.
output.elasticsearch.max_retries: 5
# When deploying beats to multiple systems or locations, uncomment the following 
# setup.template.* and setup.ilm.* configurations after running the beats setup command
# Otherwise, uncomment the following configurations for beats that will only publish data
#setup.template.enabled: false
#setup.ilm.check_exists: false
#setup.ilm.overwrite: false
#=== Xpack Monitoring ===
# When monitoring is enabled, the beat will ship monitoring data to the cluster
monitoring.enabled: true
#=== Queue ===
# See the 'Configure the internal queue' documentation for each Beat before 
# configuring the queue. Note that only one queue type can be configured.
# You need to uncomment the specific queue type you decide to use.
# The `queue.mem` settings will cache events to memory in case access to the
# Elasticsearch cluster, via the internet, is unavailable (internet outage)
#queue.mem:
#  events: 4096
#  flush.min_events: 512
#  flush.timeout: 1s
# The `queue.spool` settings will cache events to disk in case the system is offline
# NOTE: The file spool queue is a beta functionality in 7.4
queue.spool:
  file:
    size: 512MiB
    page_size: 16KiB
    prealloc: ~
  write:
    buffer_size: 10MiB
    flush.timeout: 1s
    flush.min.events: 1024

Once the file is updated, don’t forget to save the changes made to packetbeat.yml before proceeding.

Starting Packetbeat

Now that the configuration is updated, it's time to set up Packetbeat with our Elasticsearch Service deployment. Running the setup command only needs to be performed once for each beat (for instance, only once for Packetbeat 7.4.0). In our console, we will navigate to the /usr/share/packetbeat directory, then issue the sudo bin/packetbeat -c /etc/packetbeat/packetbeat.yml -path.home /usr/share/packetbeat setup command. Then we will confirm we receive “Index setup finished” and “Loaded dashboards” messages (shown below).

[myuser@centos7pi ~]$ cd /usr/share/packetbeat
[myuser@centos7pi packetbeat]$ sudo bin/packetbeat -c /etc/packetbeat/packetbeat.yml -path.home /usr/share/packetbeat setup
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
[myuser@centos7pi packetbeat]$

Now that we’ve set up the index and loaded the dashboards, it is time to enable the Packetbeat service, then start the Packetbeat service.

[myuser@centos7pi packetbeat]$ sudo systemctl enable packetbeat
Created symlink from /etc/systemd/system/multi-user.target.wants/packetbeat.service to /usr/lib/systemd/system/packetbeat.service.
[myuser@centos7pi packetbeat]$ sudo systemctl start packetbeat
[myuser@centos7pi packetbeat]$ sudo systemctl status packetbeat
● packetbeat.service - Packetbeat analyzes network traffic and sends the data to Elasticsearch.
   Loaded: loaded (/usr/lib/systemd/system/packetbeat.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2019-12-11 23:01:16 CST; 5s ago
     Docs: https://www.elastic.co/products/beats/packetbeat
 Main PID: 12990 (packetbeat)
   CGroup: /system.slice/packetbeat.service
           └─12990 /usr/share/packetbeat/bin/packetbeat -e -c /etc/packetbeat/packetbeat.yml -path.ho...
Dec 11 23:01:20 centos7pi.my.domain packetbeat[12990]: 2019-12-11T23:01:20.733-0600        I...ed
Hint: Some lines were ellipsized, use -l to show in full.
[myuser@centos7pi packetbeat]$ cd ~
[myuser@centos7pi ~]$

If you are using the GeoIP ingest pipeline and do not see data coming from Packetbeat for this device in your Elastic cluster, uncomment the logging section and restart Packetbeat.

Wrapping up with Beats on CentOS

Hurray! Now we have finished installing and configuring Auditbeat, Filebeat, and Packetbeat on this CentOS system. If you run into any issues, make sure to enable logging for the respective Beat (uncomment the logging options in the configuration file) then look through the log files in the /var/log/elastic directory.

Next steps: Data collection with Beats

In the next blog, we will install and configure Auditbeat, Filebeat, and Packetbeat on MacOS. While the next few blogs will focus on data collection from different systems, we will also cover some other use cases, including visualizing GeoIP data in Elastic Maps.

Follow along with this Elastic SIEM for home and small business blog series as we develop a powerful, yet simple, security solution at home (or for your small business). Remember that once we install and configure Beats on our systems, we can go to the SIEM app to see what data is available.

A few last things...

If you run into any issues, the first place we’d recommend turning is to our documentation. It can help with many common issues. If you still have outstanding questions, check out our Elastic forums for additional help. Or, if you want to talk to the Elastic Support team directly, you have direct access to a team of experts if you’ve deployed on Elasticsearch Service. If you are self-hosting, you can start an Elastic subscription today and have direct access to a team of experts. Be safe out there!