IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

› ›

Defining field mappings

edit

Defining field mappings

edit

You must define the fields used by your Beat, along with their mapping details, in _meta/fields.yml. After editing this file, run make update.

Define the field mappings in the fields array:

- key: mybeat
  title: mybeat
  description: These are the fields used by mybeat.
  fields:
    - name: last_name 
      type: keyword 
      required: true 
      description: > 
        The last name.
    - name: first_name
      type: keyword
      required: true
      description: >
        The first name.
    - name: comment
      type: text
      required: false
      description: >
        Comment made by the user.

	`name`: The field name
	`type`: The field type. The value of `type` can be any datatype available in Elasticsearch. If no value is specified, the default type is `keyword`.
	`required`: Whether or not a field value is required
	`description`: Some information about the field contents

Mapping parameters

edit

You can specify other mapping parameters for each field. See the Elasticsearch Reference for more details about each parameter.

`format`	Specify a custom date format used by the field.
`multi_fields`	For `text` or `keyword` fields, use `multi_fields` to define multi-field mappings.
`enabled`	Whether or not the field is enabled.
`analyzer`	Which analyzer to use when indexing.
`search_analyzer`	Which analyzer to use when searching.
`norms`	Applies to `text` and `keyword` fields. Default is `false`.
`dynamic`	Dynamic field control. Can be one of `true` (default), `false`, or `strict`.
`index`	Whether or not the field should be indexed.
`doc_values`	Whether or not the field should have doc values generated.
`copy_to`	Which field to copy the field value into.
`ignore_above`	Elasticsearch ignores (does not index) strings that are longer than the specified value. When this property value is missing or `0`, the `libbeat` default value of `1024` characters is used. If the value is `-1`, the Elasticsearch default value is used.

For example, you can use the copy_to mapping parameter to copy the last_name and first_name fields into the full_name field at index time:

- key: mybeat
  title: mybeat
  description: These are the fields used by mybeat.
  fields:
    - name: last_name
      type: text
      required: true
      copy_to: full_name 
      description: >
        The last name.
    - name: first_name
      type: text
      required: true
      copy_to: full_name 
      description: >
        The first name.
    - name: full_name
      type: text
      required: false
      description: >
        The last_name and first_name combined into one field for easy searchability.

	Copy the value of `last_name` into `full_name`
	Copy the value of `first_name` into `full_name`

There are also some Kibana-specific properties, not detailed here. These are: analyzed, count, searchable, aggregatable, and script. Kibana parameters can also be described using pattern, input_format, output_format, output_precision, label_template, url_template, and open_link_in_current_tab.

Defining text multi-fields

edit

There are various options that you can apply when using text fields. You can define a simple text field using the default analyzer without any other options, as in the example shown earlier.

To keep the original keyword value when using text mappings, for instance to use in aggregations or ordering, you can use a multi-field mapping:

- key: mybeat
  title: mybeat
  description: These are the fields used by mybeat.
  fields:
    - name: city
      type: text
      multi_fields: 
        - name: keyword 
          type: keyword

	`multi_fields`: Define the `multi_fields` mapping parameter.
	`name`: This is a conventional name for a multi-field. It can be anything (`raw` is another common option) but the convention is to use `keyword`.
	`type`: Specify the `keyword` type to use the field in aggregations or to order documents.

For more information, see the Elasticsearch documentation about multi-fields.

Defining a text analyzer in-line

edit

It is possible to define a new text analyzer or search analyzer in-line with the field definition in the field’s mapping parameters.

For example, you can define a new text analyzer that does not break hyphenated names:

- key: mybeat
  title: mybeat
  description: These are the fields used by mybeat.
  fields:
    - name: last_name
      type: text
      required: true
      description: >
        The last name.
      analyzer:
        mybeat_hyphenated_name: 
          type: pattern 
          pattern: "[\\W&&[^-]]+" 
      search_analyzer:
        mybeat_hyphenated_name: 
          type: pattern
          pattern: "[\\W&&[^-]]+"

	Use a newly defined text analyzer
	Define the custome analyzer type
	Specify the analyzer behaviour
	Use the same analyzer for the search

The names of custom analyzers that are defined in-line may not be reused for a different text analyzer. If a text analyzer name is reused it is checked for matching existing instances of the analyzer. It is recommended that the analyzer name is prefixed with the beat name to avoid name clashes.

For more information, see Elasticsearch documentation about defining custom text analyzers.

« Community Beats Naming Conventions »