Search templatesedit
A search template is a stored search you can run with different variables.
If you use Elasticsearch as a search backend, you can pass user input from a search bar as parameters for a search template. This lets you run searches without exposing Elasticsearch’s query syntax to your users.
If you use Elasticsearch for a custom application, search templates let you change your searches without modifying your app’s code.
Create a search templateedit
To create or update a search template, use the create stored script API.
The request’s source supports the same parameters as the search API's request body.source also accepts Mustache variables, from an open source project mustache.java.
Typically Mustache variables are enclosed in
double curly brackets: {{my-var}}. When you run a templated search, Elasticsearch
replaces these variables with values from params. To learn more about mustache syntax - see Mustache.js manual Search templates must use a lang of mustache.
The following request creates a search template with an id of
my-search-template.
response = client.put_script(
id: 'my-search-template',
body: {
script: {
lang: 'mustache',
source: {
query: {
match: {
message: '{{query_string}}'
}
},
from: '{{from}}',
size: '{{size}}'
},
params: {
query_string: 'My query string'
}
}
}
)
puts response
PUT _scripts/my-search-template
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
},
"params": {
"query_string": "My query string"
}
}
}
Elasticsearch stores search templates as Mustache scripts in the
cluster state. Elasticsearch compiles search templates in the template script context.
Settings that limit or disable scripts also affect search templates.
Validate a search templateedit
To test a template with different params, use the
render search template API.
response = client.render_search_template(
body: {
id: 'my-search-template',
params: {
query_string: 'hello world',
from: 20,
size: 10
}
}
)
puts response
POST _render/template
{
"id": "my-search-template",
"params": {
"query_string": "hello world",
"from": 20,
"size": 10
}
}
When rendered, the template outputs a search request body.
{
"template_output": {
"query": {
"match": {
"message": "hello world"
}
},
"from": "20",
"size": "10"
}
}
You can also use the API to test inline templates.
response = client.render_search_template(
body: {
source: {
query: {
match: {
message: '{{query_string}}'
}
},
from: '{{from}}',
size: '{{size}}'
},
params: {
query_string: 'hello world',
from: 20,
size: 10
}
}
)
puts response
POST _render/template
{
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
},
"params": {
"query_string": "hello world",
"from": 20,
"size": 10
}
}
Run a templated searchedit
To run a search with a search template, use the search
template API. You can specify different params with each request.
response = client.search_template(
index: 'my-index',
body: {
id: 'my-search-template',
params: {
query_string: 'hello world',
from: 0,
size: 10
}
}
)
puts response
GET my-index/_search/template
{
"id": "my-search-template",
"params": {
"query_string": "hello world",
"from": 0,
"size": 10
}
}
The response uses the same properties as the search API's response.
{
"took": 36,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 0.5753642,
"hits": [
{
"_index": "my-index",
"_id": "1",
"_score": 0.5753642,
"_source": {
"message": "hello world"
}
}
]
}
}
Run multiple templated searchesedit
To run multiple templated searches with a single request, use the multi search template API. These requests often have less overhead and faster speeds than multiple individual searches.
response = client.msearch_template(
index: 'my-index',
body: [
{},
{
id: 'my-search-template',
params: {
query_string: 'hello world',
from: 0,
size: 10
}
},
{},
{
id: 'my-other-search-template',
params: {
query_type: 'match_all'
}
}
]
)
puts response
GET my-index/_msearch/template
{ }
{ "id": "my-search-template", "params": { "query_string": "hello world", "from": 0, "size": 10 }}
{ }
{ "id": "my-other-search-template", "params": { "query_type": "match_all" }}
Get search templatesedit
To retrieve a search template, use the get stored script API.
response = client.get_script( id: 'my-search-template' ) puts response
GET _scripts/my-search-template
To get a list of all search templates and other stored scripts, use the cluster state API.
response = client.cluster.state( metric: 'metadata', pretty: true, filter_path: 'metadata.stored_scripts' ) puts response
GET _cluster/state/metadata?pretty&filter_path=metadata.stored_scripts
Delete a search templateedit
To delete a search template, use the delete stored script API.
response = client.delete_script( id: 'my-search-template' ) puts response
DELETE _scripts/my-search-template
Set default valuesedit
To set a default value for a variable, use the following syntax:
{{my-var}}{{^my-var}}default value{{/my-var}}
If a templated search doesn’t specify a value in its params, the search uses
the default value instead. For example, the following template sets defaults
for from and size.
response = client.render_search_template(
body: {
source: {
query: {
match: {
message: '{{query_string}}'
}
},
from: '{{from}}{{^from}}0{{/from}}',
size: '{{size}}{{^size}}10{{/size}}'
},
params: {
query_string: 'hello world'
}
}
)
puts response
POST _render/template
{
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}{{^from}}0{{/from}}",
"size": "{{size}}{{^size}}10{{/size}}"
},
"params": {
"query_string": "hello world"
}
}
URL encode stringsedit
Use the {{#url}} function to URL encode a string.
response = client.render_search_template(
body: {
source: {
query: {
term: {
'url.full' => '{{#url}}{{host}}/{{page}}{{/url}}'
}
}
},
params: {
host: 'http://example.com',
page: 'hello-world'
}
}
)
puts response
POST _render/template
{
"source": {
"query": {
"term": {
"url.full": "{{#url}}{{host}}/{{page}}{{/url}}"
}
}
},
"params": {
"host": "http://example.com",
"page": "hello-world"
}
}
The template renders as:
{
"template_output": {
"query": {
"term": {
"url.full": "http%3A%2F%2Fexample.com%2Fhello-world"
}
}
}
}
Concatenate valuesedit
Use the {{#join}} function to concatenate array values as a comma-delimited
string. For example, the following template concatenates two email addresses.
response = client.render_search_template(
body: {
source: {
query: {
match: {
'user.group.emails' => '{{#join}}emails{{/join}}'
}
}
},
params: {
emails: [
'user1@example.com',
'user_one@example.com'
]
}
}
)
puts response
POST _render/template
{
"source": {
"query": {
"match": {
"user.group.emails": "{{#join}}emails{{/join}}"
}
}
},
"params": {
"emails": [ "user1@example.com", "user_one@example.com" ]
}
}
The template renders as:
{
"template_output": {
"query": {
"match": {
"user.group.emails": "user1@example.com,user_one@example.com"
}
}
}
}
You can also specify a custom delimiter.
response = client.render_search_template(
body: {
source: {
query: {
range: {
'user.effective.date' => {
gte: '{{date.min}}',
lte: '{{date.max}}',
format: "{{#join delimiter='||'}}date.formats{{/join delimiter='||'}}"
}
}
}
},
params: {
date: {
min: '2098',
max: '06/05/2099',
formats: [
'dd/MM/yyyy',
'yyyy'
]
}
}
}
)
puts response
POST _render/template
{
"source": {
"query": {
"range": {
"user.effective.date": {
"gte": "{{date.min}}",
"lte": "{{date.max}}",
"format": "{{#join delimiter='||'}}date.formats{{/join delimiter='||'}}"
}
}
}
},
"params": {
"date": {
"min": "2098",
"max": "06/05/2099",
"formats": ["dd/MM/yyyy", "yyyy"]
}
}
}
The template renders as:
{
"template_output": {
"query": {
"range": {
"user.effective.date": {
"gte": "2098",
"lte": "06/05/2099",
"format": "dd/MM/yyyy||yyyy"
}
}
}
}
}
Convert to JSONedit
Use the {{#toJson}} function to convert a variable value to its JSON
representation.
For example, the following template uses {{#toJson}} to pass an array. To
ensure the request body is valid JSON, the source is written in the string
format.
response = client.render_search_template(
body: {
source: '{ "query": { "terms": { "tags": {{#toJson}}tags{{/toJson}} }}}',
params: {
tags: [
'prod',
'es01'
]
}
}
)
puts response
POST _render/template
{
"source": "{ \"query\": { \"terms\": { \"tags\": {{#toJson}}tags{{/toJson}} }}}",
"params": {
"tags": [
"prod",
"es01"
]
}
}
The template renders as:
{
"template_output": {
"query": {
"terms": {
"tags": [
"prod",
"es01"
]
}
}
}
}
You can also use {{#toJson}} to pass objects.
response = client.render_search_template(
body: {
source: '{ "query": {{#toJson}}my_query{{/toJson}} }',
params: {
my_query: {
match_all: {}
}
}
}
)
puts response
POST _render/template
{
"source": "{ \"query\": {{#toJson}}my_query{{/toJson}} }",
"params": {
"my_query": {
"match_all": { }
}
}
}
The template renders as:
{
"template_output" : {
"query" : {
"match_all" : { }
}
}
}
You can also pass an array of objects.
response = client.render_search_template(
body: {
source: '{ "query": { "bool": { "must": {{#toJson}}clauses{{/toJson}} }}}',
params: {
clauses: [
{
term: {
'user.id' => 'kimchy'
}
},
{
term: {
'url.domain' => 'example.com'
}
}
]
}
}
)
puts response
POST _render/template
{
"source": "{ \"query\": { \"bool\": { \"must\": {{#toJson}}clauses{{/toJson}} }}}",
"params": {
"clauses": [
{
"term": {
"user.id": "kimchy"
}
},
{
"term": {
"url.domain": "example.com"
}
}
]
}
}
The template renders as:
{
"template_output": {
"query": {
"bool": {
"must": [
{
"term": {
"user.id": "kimchy"
}
},
{
"term": {
"url.domain": "example.com"
}
}
]
}
}
}
}
Use conditionsedit
To create if conditions, use the following syntax:
{{#condition}}content{{/condition}}
If the condition variable is true, Elasticsearch displays its content. For example, the
following template searches data from the past year if year_scope is true.
response = client.render_search_template(
body: {
source: '{ "query": { "bool": { "filter": [ {{#year_scope}} { "range": { "@timestamp": { "gte": "now-1y/d", "lt": "now/d" } } }, {{/year_scope}} { "term": { "user.id": "{{user_id}}" }}]}}}',
params: {
year_scope: true,
user_id: 'kimchy'
}
}
)
puts response
POST _render/template
{
"source": "{ \"query\": { \"bool\": { \"filter\": [ {{#year_scope}} { \"range\": { \"@timestamp\": { \"gte\": \"now-1y/d\", \"lt\": \"now/d\" } } }, {{/year_scope}} { \"term\": { \"user.id\": \"{{user_id}}\" }}]}}}",
"params": {
"year_scope": true,
"user_id": "kimchy"
}
}
The template renders as:
{
"template_output" : {
"query" : {
"bool" : {
"filter" : [
{
"range" : {
"@timestamp" : {
"gte" : "now-1y/d",
"lt" : "now/d"
}
}
},
{
"term" : {
"user.id" : "kimchy"
}
}
]
}
}
}
}
If year_scope is false, the template searches data from any time period.
response = client.render_search_template(
body: {
source: '{ "query": { "bool": { "filter": [ {{#year_scope}} { "range": { "@timestamp": { "gte": "now-1y/d", "lt": "now/d" } } }, {{/year_scope}} { "term": { "user.id": "{{user_id}}" }}]}}}',
params: {
year_scope: false,
user_id: 'kimchy'
}
}
)
puts response
POST _render/template
{
"source": "{ \"query\": { \"bool\": { \"filter\": [ {{#year_scope}} { \"range\": { \"@timestamp\": { \"gte\": \"now-1y/d\", \"lt\": \"now/d\" } } }, {{/year_scope}} { \"term\": { \"user.id\": \"{{user_id}}\" }}]}}}",
"params": {
"year_scope": false,
"user_id": "kimchy"
}
}
The template renders as:
{
"template_output" : {
"query" : {
"bool" : {
"filter" : [
{
"term" : {
"user.id" : "kimchy"
}
}
]
}
}
}
}
To create if-else conditions, use the following syntax:
{{#condition}}if content{{/condition}} {{^condition}}else content{{/condition}}
For example, the following template searches data from the past year if
year_scope is true. Otherwise, it searches data from the past day.
response = client.render_search_template(
body: {
source: '{ "query": { "bool": { "filter": [ { "range": { "@timestamp": { "gte": {{#year_scope}} "now-1y/d" {{/year_scope}} {{^year_scope}} "now-1d/d" {{/year_scope}} , "lt": "now/d" }}}, { "term": { "user.id": "{{user_id}}" }}]}}}',
params: {
year_scope: true,
user_id: 'kimchy'
}
}
)
puts response
POST _render/template
{
"source": "{ \"query\": { \"bool\": { \"filter\": [ { \"range\": { \"@timestamp\": { \"gte\": {{#year_scope}} \"now-1y/d\" {{/year_scope}} {{^year_scope}} \"now-1d/d\" {{/year_scope}} , \"lt\": \"now/d\" }}}, { \"term\": { \"user.id\": \"{{user_id}}\" }}]}}}",
"params": {
"year_scope": true,
"user_id": "kimchy"
}
}
Search template examples with Mustacheedit
The mustache templating language defines various tag types you can use within templates. The following sections describe some of these tag types and provide examples of using them in Elasticsearch search templates.
Mustache variablesedit
Mustache tags are typically enclosed in double curly brackets. A mustache variable: {{my-variable}} is a type of mustache tag. When you run a templated search, Elasticsearch replaces these variables with values from params.
For example, consider the following search template:
response = client.put_script(
id: 'my-search-template',
body: {
script: {
lang: 'mustache',
source: {
query: {
match: {
message: '{{query_string}}'
}
},
from: '{{from}}',
size: '{{size}}'
}
}
}
)
puts response
PUT _scripts/my-search-template
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
}
}
}
Testing the above search template with params:
response = client.render_search_template(
body: {
id: 'my-search-template',
params: {
query_string: 'hello world',
from: 20,
size: 10
}
}
)
puts response
POST _render/template
{
"id": "my-search-template",
"params": {
"query_string": "hello world",
"from": 20,
"size": 10
}
}
When rendered, the {{query_string}} in message is replaced with hello world passed in params.
{
"template_output": {
"query": {
"match": {
"message": "hello world"
}
},
"from": "20",
"size": "10"
}
}
If your search template doesn’t pass a value to your query_string the message would be replaced with a empty string.
For example:
response = client.render_search_template(
body: {
id: 'my-search-template',
params: {
from: 20,
size: 10
}
}
)
puts response
POST _render/template
{
"id": "my-search-template",
"params": {
"from": 20,
"size": 10
}
}
When rendered, template outputs as:
{
"template_output": {
"query": {
"match": {
"message": ""
}
},
"from": "20",
"size": "10"
}
}
Sectionsedit
Sections are also a type of Mustache tags. You can use sections in your search template with a nested or unnested object. A section begins with {{#my-section-variable}} and ends with {{/my-section-variable}}.
The following search template shows an example using sections with nested objects:
response = client.render_search_template(
body: {
source: "\n {\n \"query\": {\n \"match\": {\n {{#query_message}}\n {{#query_string}}\n \"message\": \"Hello {{#first_name_section}}{{first_name}}{{/first_name_section}} {{#last_name_section}}{{last_name}}{{/last_name_section}}\"\n {{/query_string}}\n {{/query_message}}\n }\n }\n }\n ",
params: {
query_message: {
query_string: {
first_name_section: {
first_name: 'John'
},
last_name_section: {
last_name: 'kimchy'
}
}
}
}
}
)
puts response
POST _render/template
{
"source":
"""
{
"query": {
"match": {
{{#query_message}}
{{#query_string}}
"message": "Hello {{#first_name_section}}{{first_name}}{{/first_name_section}} {{#last_name_section}}{{last_name}}{{/last_name_section}}"
{{/query_string}}
{{/query_message}}
}
}
}
""",
"params": {
"query_message": {
"query_string": {
"first_name_section": {"first_name": "John"},
"last_name_section": {"last_name": "kimchy"}
}
}
}
}
The template renders as:
{
"template_output": {
"query": {
"match": {
"message": "Hello John kimchy"
}
}
}
}
Listsedit
You can pass a list of objects and loop over each item in your search template.
For example, following search template combines sections and matches all the usernames:
response = client.put_script(
id: 'my-search-template',
body: {
script: {
lang: 'mustache',
source: {
query: {
multi_match: {
query: '{{query_string}}',
fields: '[{{#text_fields}}{{user_name}},{{/text_fields}}]'
}
}
}
}
}
)
puts response
PUT _scripts/my-search-template
{
"script": {
"lang": "mustache",
"source": {
"query":{
"multi_match":{
"query": "{{query_string}}",
"fields": """[{{#text_fields}}{{user_name}},{{/text_fields}}]"""
}
}
}
}
}
Testing the template:
response = client.render_search_template(
body: {
id: 'my-search-template',
params: {
query_string: 'My string',
text_fields: [
{
user_name: 'John'
},
{
user_name: 'kimchy'
}
]
}
}
)
puts response
POST _render/template
{
"id": "my-search-template",
"params": {
"query_string": "My string",
"text_fields": [
{
"user_name": "John"
},
{
"user_name": "kimchy"
}
]
}
}
When rendered, template outputs:
{
"template_output": {
"query": {
"multi_match": {
"query": "My string",
"fields": "[John,kimchy,]"
}
}
}
}
The above will cause a trailing comma issue, which causes invalid JSON. A workaround would be to include an inverted section and adding a variable to make sure it’s the last item in the array.
For example:
response = client.put_script(
id: 'my-search-template',
body: {
script: {
lang: 'mustache',
source: {
query: {
multi_match: {
query: '{{query_string}}',
fields: '[{{#text_fields}}{{user_name}}{{^last}},{{/last}}{{/text_fields}}]'
}
}
}
}
}
)
puts response
PUT _scripts/my-search-template
{
"script": {
"lang": "mustache",
"source": {
"query":{
"multi_match":{
"query": "{{query_string}}",
"fields": """[{{#text_fields}}{{user_name}}{{^last}},{{/last}}{{/text_fields}}]"""
}
}
}
}
}
Testing the my-search-template again with a variable: last to determine it’s the last item in the array:
response = client.render_search_template(
body: {
id: 'my-search-template',
params: {
query_string: 'My string',
text_fields: [
{
user_name: 'John',
last: false
},
{
user_name: 'kimchy',
last: true
}
]
}
}
)
puts response
POST _render/template
{
"id": "my-search-template",
"params": {
"query_string": "My string",
"text_fields": [
{
"user_name": "John",
"last": false
},
{
"user_name": "kimchy",
"last": true
}
]
}
}
When rendered the template outputs:
{
"template_output": {
"query": {
"multi_match": {
"query": "My string",
"fields": "[John,kimchy]"
}
}
}
}
Lambdasedit
Elasticsearch has pre-built custom functions to support converting the text into a specific format.
To Learn more about usage of mustache lambdas, check out the examples in Url encode strings, Concatenate values, and Convert to json.
Inverted sectionsedit
Inverted sections are useful when you want to set a value once.
To use inverted sections use following syntax:
{{^my-variable}} content {{/my-variable}}
For example, in the following search template if name_exists is false, message is set with Hello World, else it is set to empty string.
response = client.render_search_template(
body: {
source: {
query: {
match: {
message: '{{^name_exists}}Hello World{{/name_exists}}'
}
}
},
params: {
name_exists: false
}
}
)
puts response
POST _render/template
{
"source": {
"query": {
"match": {
"message": "{{^name_exists}}Hello World{{/name_exists}}"
}
}
},
"params": {
"name_exists": false
}
}
They can also be combined with conditions and default values.
For example, in the following search template, if name_exists is true, the value of {{query_string}} is replaced. If name_exists is false, it is set to the default value World.
response = client.render_search_template(
body: {
source: {
query: {
match: {
message: 'Hello {{#name_exists}}{{query_string}}{{/name_exists}}{{^name_exists}}World{{/name_exists}}'
}
}
},
params: {
query_string: 'Kimchy',
name_exists: true
}
}
)
puts response
POST _render/template
{
"source": {
"query": {
"match": {
"message": "Hello {{#name_exists}}{{query_string}}{{/name_exists}}{{^name_exists}}World{{/name_exists}}"
}
}
},
"params": {
"query_string": "Kimchy",
"name_exists": true
}
}
When rendered, template output:
{
"template_output": {
"query": {
"match": {
"message": "Hello Kimchy"
}
}
}
}
Set delimiteredit
You can change the default delimiter: double curly brackets {{my-variable}} to any custom delimiter in your search template.
For example, the following search template changes the default delimiter to a single round bracket (query_string).
response = client.put_script(
id: 'my-search-template',
body: {
script: {
lang: 'mustache',
source: "\n {\n \"query\": {\n \"match\": {\n {{=( )=}}\n \"message\": \"(query_string)\"\n (={{ }}=)\n }\n }\n }\n "
}
}
)
puts response
PUT _scripts/my-search-template
{
"script": {
"lang": "mustache",
"source":
"""
{
"query": {
"match": {
{{=( )=}}
"message": "(query_string)"
(={{ }}=)
}
}
}
"""
}
}
Testing the template with new delimiter:
response = client.render_search_template(
body: {
id: 'my-search-template',
params: {
query_string: 'hello world'
}
}
)
puts response
POST _render/template
{
"id": "my-search-template",
"params": {
"query_string": "hello world"
}
}
When rendered, template outputs:
{
"template_output": {
"query": {
"match": {
"message": "hello world"
}
}
}
}
Unsupported featuresedit
The following mustache features are not supported in Elasticsearch search templates:
- Partials