Truncate token filteredit

Truncates tokens that exceed a specified character limit. This limit defaults to 10 but can be customized using the length parameter.

For example, you can use the truncate filter to shorten all tokens to 3 characters or fewer, changing jumping fox to jum fox.

This filter uses Lucene’s TruncateTokenFilter.

Exampleedit

The following analyze API request uses the truncate filter to shorten tokens that exceed 10 characters in the quinquennial extravaganza carried on:

GET _analyze
{
  "tokenizer" : "whitespace",
  "filter" : ["truncate"],
  "text" : "the quinquennial extravaganza carried on"
}

The filter produces the following tokens:

[ the, quinquenni, extravagan, carried, on ]

Add to an analyzeredit

The following create index API request uses the truncate filter to configure a new custom analyzer.

PUT custom_truncate_example
{
  "settings" : {
    "analysis" : {
      "analyzer" : {
        "standard_truncate" : {
        "tokenizer" : "standard",
        "filter" : ["truncate"]
        }
      }
    }
  }
}

Configurable parametersedit

length
(Optional, integer) Character limit for each token. Tokens exceeding this limit are truncated. Defaults to 10.

Customizeedit

To customize the truncate filter, duplicate it to create the basis for a new custom token filter. You can modify the filter using its configurable parameters.

For example, the following request creates a custom truncate filter, 5_char_trunc, that shortens tokens to a length of 5 or fewer characters:

PUT 5_char_words_example
{
  "settings": {
    "analysis": {
      "analyzer": {
        "lowercase_5_char": {
          "tokenizer": "lowercase",
          "filter": [ "5_char_trunc" ]
        }
      },
      "filter": {
        "5_char_trunc": {
          "type": "truncate",
          "length": 5
        }
      }
    }
  }
}