Processors Configurationedit

You can define processors in your configuration to process events before they are sent to the configured output. The libbeat library provides processors for reducing the number of exported fields, and processors for enhancing events with additional metadata. Each processor receives an event, applies a defined action to the event, and returns the event. If you define a list of processors, they are executed in the order they are defined in the configuration file.

event -> processor 1 -> event1 -> processor 2 -> event2 ...

The processors are defined in the Filebeat configuration file.

Each processor has associated an action with a set of parameters and optionally a condition. If the condition is present, then the action is executed only if the condition is fulfilled. If no condition is passed then the action is always executed.

processors:
 - <action>:
     when:
        <condition>
     <parameters>
 - <action>:
     when:
        <condition>
     <parameters>
...

where <action> can be a way to select the fields that are exported or a way to add meta data to the event , <condition> contains the definition of the condition. and <parameters> is the list of parameters passed along the <action>.

See Filtering and Enhancing the Exported Data for specific Filebeat examples.

Conditionedit

Each condition receives a field to compare or multiple fields under the same condition and then AND is used between them. You can see a list of the exported fields.

For each field, you can specify a simple field name or a nested map, for example dns.question.name.

A condition can be:

equalsedit

With the equals condition, you can compare if a field has a certain value. The condition accepts only an integer or a string value.

For example, the following condition checks if the response code of the HTTP transaction is 200:

equals:
  http.response.code: 200

containsedit

The contains condition checks if a value is part of a field. The field can be a string or an array of strings. The condition accepts only a string value.

For example, the following condition checks if an error is part of the transaction status:

contains:
  status: "Specific error"

regexpedit

The regexp condition checks the field against a regular expression. The condition accepts only strings.

For example, the following condition checks if the process name starts with foo:

reqexp:
  system.process.name: "foo.*"

rangeedit

The range condition checks if the field is in a certain range of values. The condition supports lt, lte, gt and gte. The condition accepts only integer or float values.

For example, the following condition checks for failed HTTP transaction by comparing the http.response.code field with 400.

range:
    http.response.code:
        gte: 400

that can be also translated to:

range:
    http.response.code.gte: 400

For example, the following condition checks if the CPU usage in percentage has a value between 0.5 and 0.8.

range:
    system.cpu.user.pct.gte: 0.5
    system.cpu.user.pct.lt: 0.8

The or operator receives a list of conditions.

or:
  - <condition1>
  - <condition2>
  - <condition3>
  ...

For example the condition http.response.code = 304 OR http.response.code = 404 translates to:

or:
  - equals:
      http.response.code: 304
  - equals:
      http.response.code: 404

ANDedit

The and operator receives a list of conditions.

and:
  - <condition1>
  - <condition2>
  - <condition3>
  ...

For example the condition http.response.code = 200 AND status = OK translates to:

and:
  - equals:
      http.response.code: 200
  - equals:
      status: OK

To configure a condition like <condition1> OR <condition2> AND <condition3>:

or:
 - <condition1>
 - and:
    - <condition2>
    - <condition3>

NOTedit

The not operator receives the condition to negate.

not:
  <condition>

For example the condition NOT status = OK translates to:

not:
  equals:
    status: OK

Actionsedit

The supported filter actions are:

See Exported Fields for the full list of possible fields.

include_fieldsedit

The include_fields action specifies what fields to export if a certain condition is fulfilled. The condition is optional and if it’s missing then the defined fields are always exported. The @timestamp and type fields are always exported, even if they are not defined in the include_fields list.

processors:
 - include_fields:
     when:
        condition
     fields: ["field1", "field2", ...]

You can specify multiple include_fields actions under the processors section.

Note

If you define an empty list of fields under include_fields, then only the required fields, @timestamp and type, are exported.

drop_fieldsedit

The drop_fields action specifies what fields to drop if a certain condition is fulfilled. The condition is optional and if it’s missing then the defined fields are always dropped. The @timestamp and type fields cannot be dropped, even if they show up in the drop_fields list.

processors:
 - drop_fields:
     when:
        condition
     fields: ["field1", "field2", ...]
Note

If you define an empty list of fields under drop_fields, then no fields are dropped.

drop_eventedit

The drop_event action drops the entire event if the associated condition is fulfilled. The condition is mandatory, as without one all the events are dropped.

processors:
 - drop_event:
     when:
        condition