Configure Logstash

The Elastic Stack is very popular for handling logs and it can be used to send logs to Elastic Cloud. If you have an Elasticsearch cluster running on Elastic Cloud, you can run Logstash outside of Elastic Cloud, on the cloud or in your own data center, and configure Logstash to send log events to your Elasticsearch cluster on Elastic Cloud.

To send logs to Elasticsearch, you use the elasticsearch-output plugin. Specify the following options in the configuration file:

  1. Point Logstash to a cluster in Elastic Cloud with the hosts parameter. Logstash versions before 2.x might also require that you specify values for protocol and port.
  2. Include credentials for user and password, if required. For Elasticsearch version 2.x, we recommend that you always enable Shield, which will require user credentials. For Elasticsearch version 5.0 and later, X-Pack security features are always enabled on Elastic Cloud and you must include user credentials to be able to connect to your cluster.
  3. Logstash versions before 2.x might require that you set ssl to be true. We recommend you always use SSL.
  4. Specify any additional options that you want to include, such as the index to use and document type.

To make it easier for you to get started, we have included a couple of examples for you. The parameters for Logstash configuration files changed between versions, so make sure that you adapt the example that applies to your version of Logstash.

Logstash and the Elasticsearch cluster receiving the logs do not have to be of the same version, but not all versions are compatible with each other. To learn more about supported Logstash versions, see Support Matrix.

For production systems, these examples need to be modified further. The examples are meant to help you get started quickly, but they specify users created by X-Pack or Shield that should not be used for production systems, because they have too many permissions. If you plan to put Logstash into production, create a dedicated user that has only the minimum required permissions. To learn more, see Configuring Logstash to use Basic Authentication with X-Pack for Elasticsearch version 5.0 and later or Creating a user with Shield for Elasticsearch versions before 5.0.

Example: Sending Logs to Elasticsearch (Logstash Version 2.0 and Later)

To send logs to the myindex index with the document type mylogs, specify output options similar to the following in your configuration file:

output {
  elasticsearch {
    hosts => "https://6f881bxxxxxxxxxxxxxxxxxxx.us-east-1.aws.found.io:9243"
    user => "elastic"
    password => "xxxxxxxxxxxxxxxxxxx"
    index => "myindex"
    document_type => "mylogs"
        }
}

To learn more about configuration options, see Configuring Logstash.

Example: Sending Logs to Elasticsearch (Logstash Versions Before 2.0)

To send logs to the myindex index with the document type mylogs, specify output options similar to the following in your configuration file:

output {
  elasticsearch {
    hosts => "1923a7xxxxxxxxxxxxxxxxxxx.eu-west-1.aws.found.io"
    user => "admin"
    password => "xxxxxxxxxxxxxxxxxxx"
    protocol => http
    ssl => true
    port => "9243"
    index => "myindex"
    document_type => "mylogs"
  }
}

To learn more about configuration options, see Configuring Logstash.