Defining field mappings

edit

You must define the fields used by your Beat, along with their mapping details, in _meta/fields.yml. After editing this file, run make update.

Define the field mappings in the fields array:

- key: mybeat
  title: mybeat
  description: These are the fields used by mybeat.
  fields:
    - name: last_name 
      type: keyword 
      required: true 
      description: > 
        The last name.
    - name: first_name
      type: keyword
      required: true
      description: >
        The first name.
    - name: comment
      type: text
      required: false
      description: >
        Comment made by the user.

name: The field name

type: The field type. The value of type can be any datatype available in Elasticsearch. If no value is specified, the default type is keyword.

required: Whether or not a field value is required

description: Some information about the field contents

Mapping parameters

edit

You can specify other mapping parameters for each field. See the Elasticsearch Reference for more details about each parameter.

format

Specify a custom date format used by the field.

multi_fields

For text or keyword fields, use multi_fields to define multi-field mappings.

enabled

Whether or not the field is enabled.

analyzer

Which analyzer to use when indexing.

search_analyzer

Which analyzer to use when searching.

norms

Applies to text and keyword fields. Default is false.

dynamic

Dynamic field control. Can be one of true (default), false, or strict.

index

Whether or not the field should be indexed.

doc_values

Whether or not the field should have doc values generated.

copy_to

Which field to copy the field value into.

ignore_above

Elasticsearch ignores (does not index) strings that are longer than the specified value. When this property value is missing or 0, the libbeat default value of 1024 characters is used. If the value is -1, the Elasticsearch default value is used.

For example, you can use the copy_to mapping parameter to copy the last_name and first_name fields into the full_name field at index time:

- key: mybeat
  title: mybeat
  description: These are the fields used by mybeat.
  fields:
    - name: last_name
      type: text
      required: true
      copy_to: full_name 
      description: >
        The last name.
    - name: first_name
      type: text
      required: true
      copy_to: full_name 
      description: >
        The first name.
    - name: full_name
      type: text
      required: false
      description: >
        The last_name and first_name combined into one field for easy searchability.

Copy the value of last_name into full_name

Copy the value of first_name into full_name

There are also some Kibana-specific properties, not detailed here. These are: analyzed, count, searchable, aggregatable, and script. Kibana parameters can also be described using pattern, input_format, output_format, output_precision, label_template, url_template, and open_link_in_current_tab.

Defining text multi-fields

edit

There are various options that you can apply when using text fields. You can define a simple text field using the default analyzer without any other options, as in the example shown earlier.

To keep the original keyword value when using text mappings, for instance to use in aggregations or ordering, you can use a multi-field mapping:

- key: mybeat
  title: mybeat
  description: These are the fields used by mybeat.
  fields:
    - name: city
      type: text
      multi_fields: 
        - name: keyword 
          type: keyword 

multi_fields: Define the multi_fields mapping parameter.

name: This is a conventional name for a multi-field. It can be anything (raw is another common option) but the convention is to use keyword.

type: Specify the keyword type to use the field in aggregations or to order documents.

For more information, see the Elasticsearch documentation about multi-fields.