WARNING: This documentation covers Elasticsearch 2.x. The 2.x versions of Elasticsearch have passed their EOL dates. If you are running a 2.x version, we strongly advise you to upgrade.
This documentation is no longer maintained and may be removed. For the latest information, see the current Elasticsearch documentation.
Breaking text into tokens is only half the job. To make those
tokens more easily searchable, they need to go through a normalization
process to remove insignificant differences between otherwise identical words,
such as uppercase versus lowercase. Perhaps we also need to remove significant
differences, to make
está all searchable as the same
word. Would you search for
déjà vu, or just for
This is the job of the token filters, which receive a stream of tokens from the tokenizer. You can have multiple token filters, each doing its particular job. Each receives the new token stream as output by the token filter before it.