Settings Fileedit

You can set options in the Logstash settings file, logstash.yml, to control Logstash execution. For example, you can specify pipeline settings, the location of configuration files, logging options, and other settings. Most of the settings in the logstash.yml file are also available as command-line flags when you run Logstash. Any flags that you set at the command line override the corresponding settings in the logstash.yml file.

The logstash.yml file, which is written in YAML, is located in LOGSTASH_HOME/config. You can specify settings in hierarchical form or use flat keys. For example, to use hierarchical form to set the pipeline batch size and batch delay, you specify:

pipeline:
  batch:
    size: 125
    delay: 5

To express the same values as flat keys, you specify:

pipeline.batch.size: 125
pipeline.batch.delay: 5

The logstash.yml file includes the following settings:

Setting Description Default value

node.name

A descriptive name for the node.

Machine’s hostname

path.data

The directory that Logstash and its plugins use for any persistent needs.

LOGSTASH_HOME/data

pipeline.workers

The number of workers that will, in parallel, execute the filter and output stages of the pipeline. If you find that events are backing up, or that the CPU is not saturated, consider increasing this number to better utilize machine processing power.

Number of the host’s CPU cores

pipeline.output.workers

The number of workers to use per output plugin instance.

1

pipeline.batch.size

The maximum number of events an individual worker thread will collect from inputs before attempting to execute its filters and outputs. Larger batch sizes are generally more efficient, but come at the cost of increased memory overhead. You may have to increase the JVM heap size by setting the LS_HEAP_SIZE variable to effectively use the option.

125

pipeline.batch.delay

When creating pipeline event batches, how long in milliseconds to wait before dispatching an undersized batch to filters and workers.

5

pipeline.unsafe_shutdown

When set to true, forces Logstash to exit during shutdown even if there are still inflight events in memory. By default, Logstash will refuse to quit until all received events have been pushed to the outputs. Enabling this option can lead to data loss during shutdown.

false

path.config

The path to the Logstash config for the main pipeline. If you specify a directory or wildcard, config files are read from the directory in alphabetical order.

Platform-specific. See Logstash Directory Layout.

config.string

A string that contains the pipeline configuration to use for the main pipeline. Use the same syntax as the config file.

None

config.test_and_exit

When set to true, checks that the configuration is valid and then exits. Note that grok patterns are not checked for correctness with this setting. Logstash can read multiple config files from a directory. If you combine this setting with log.level: debug, Logstash will log the combined config file, annotating each config block with the source file it came from.

false

config.reload.automatic

When set to true, periodically checks if the configuration has changed and reloads the configuration whenever it is changed. This can also be triggered manually through the SIGHUP signal.

false

config.reload.interval

How often in seconds Logstash checks the config files for changes.

3

config.debug

When set to true, shows the fully compiled configuration as a debug log message. You must also set log.level: debug. WARNING: The log message will include any password options passed to plugin configs as plaintext, and may result in plaintext passwords appearing in your logs!

false

queue.type

The internal queuing model to use for event buffering. Specify memory for legacy in-memory based queuing, or persisted for disk-based ACKed queueing (persistent queues).

memory

path.queue

The directory path where the data files will be stored when persistent queues are enabled (queue.type: persisted).

path.data/queue

queue.page_capacity

The size of the page data files used when persistent queues are enabled (queue.type: persisted). The queue data consists of append-only data files separated into pages.

250mb

queue.max_events

The maximum number of unread events in the queue when persistent queues are enabled (queue.type: persisted).

0 (unlimited)

queue.max_bytes

The total capacity of the queue in number of bytes. Make sure the capacity of your disk drive is greater than the value you specify here. If both queue.max_events and queue.max_bytes are specified, Logstash uses whichever criteria is reached first.

1024mb (1g)

queue.checkpoint.acks

The maximum number of ACKed events before forcing a checkpoint when persistent queues are enabled (queue.type: persisted). Specify queue.checkpoint.acks: 0 to set this value to unlimited.

1024

queue.checkpoint.writes

The maximum number of written events before forcing a checkpoint when persistent queues are enabled (queue.type: persisted). Specify queue.checkpoint.writes: 0 to set this value to unlimited.

1024

queue.checkpoint.interval

The interval in milliseconds when a checkpoint is forced on the head page when persistent queues are enabled (queue.type: persisted). Specify queue.checkpoint.interval: 0 for no periodic checkpoint.

1000

http.host

The bind address for the metrics REST endpoint.

"127.0.0.1"

http.port

The bind port for the metrics REST endpoint.

9600

log.level

The log level. Valid options are:

  • fatal
  • error
  • warn
  • info
  • debug
  • trace

info

log.format

The log format. Set to json to log in JSON format, or plain to use Object#.inspect.

plain

path.logs

The directory where Logstash will write its log to.

`LOGSTASH_HOME/logs

path.plugins

Where to find custom plugins. You can specify this setting multiple times to include multiple paths. Plugins are expected to be in a specific directory hierarchy: PATH/logstash/TYPE/NAME.rb where TYPE is inputs, filters, outputs, or codecs, and NAME is the name of the plugin.

Platform-specific. See Logstash Directory Layout.