Loading

ES|QL Views

A view is a virtual index with fields that are produced from the output of an ES|QL query. Each view has a name and a definition. The definition is a complete ES|QL query, with source commands and processing commands. The view name can be used within an index pattern in the FROM command of any normal ES|QL query, as well as within another view definition. When the main query is executed, all referenced views are also executed independently. This ensures up-to-date results from all source indexes referenced by both the main query and the view definitions themselves. The final output combines all these results into a single list, including any duplicate rows.

Views are a good fit when you want to:

  • Reuse a named query. Wrap a frequently used ES|QL pipeline as a view and reference it by name, instead of repeating the same query in every request.
  • Abstract common transformations. Centralize renames, type conversions, or derived fields so consumers see a consistent set of columns without needing to know the underlying source structure.
  • Combine pre-processed data sources. Define one view per source, each with its own filters or aggregations, and query them together in a single FROM clause.
  • Simplify queries for downstream tools. Dashboards, alerts, or ad-hoc analysts can query FROM my_view without needing to know the indices or processing commands behind it.

Define a view using the REST API:

For example, consider the following ES|QL query:

FROM addresses
| RENAME city.country.name AS country
| EVAL country = CASE(country == "United States of America", "United States", country)
| STATS count=COUNT() BY country
		
count:long country:keyword
1 Japan
1 Netherlands
1 United States

We can define a view called country_addresses using that query:

				PUT /_query/view/country_addresses
					{
    "query": """
        FROM addresses
        | RENAME city.country.name AS country
        | EVAL country = CASE(country == "United States of America", "United States", country)
        | STATS count=COUNT() BY country
        """
}
		

Now a query like FROM country_addresses will produce the same output:

FROM country_addresses
		
count:long country:keyword
1 Japan
1 Netherlands
1 United States

Use views as if they were ordinary indices:

FROM index_pattern
		

Where index_pattern is a comma-separated list of index or view names, including wildcards and date-math.

Much like subqueries, views enable you to combine results from multiple independently processed data sources within a single query. Each view runs its own pipeline of processing commands (such as WHERE, EVAL, STATS, or SORT) and the results are combined together with results from other index patterns, views or subqueries in the FROM clause.

Fields that exist in one source but not another are filled with null values.

If a view definition contains a reference to another view, that is called a nested view. ES|QL allows nesting to a depth of 10. When multiple views are referenced within the same index-pattern, this leads to a branched query plan where each view will be executed independently, in parallel if possible, similar to subqueries and FORK. There is a maximum allowed branching factor of 8, for the combination of views, subqueries and FORK. So, for example, a single index pattern could reference four views and four subqueries, but adding just one more view or subquery would exceed the allowed limit and the query will fail.

Branching and nesting are allowed in combination as long as there is never more than one branch point. This means that nested branching has restrictions:

  • A view can contain subqueries, but that view cannot be used together with other views, and the subqueries can only reference nested views that contain no further branching.
  • A subquery can contain views, but those views must not introduce any additional branch points via subqueries or FORK

Views have certain limitations:

  • Commands that also generate branched query plans are usually not allowed within views, unless the branch points can be merged:
  • Cross-cluster search:
    • Remote views in CCS are not allowed (ie. FROM cluster:view will only match remote indexes with the name view. If a remote view is found, the query will fail).
    • If a remote index matches a local view name, the query will fail.
  • Serverless and Cross-project search:
    • Views are initially unavailable in serverless
  • Query parameters are not allowed in the view definition, and therefore query parameters in the main query will never impact the view results.

Views are in tech-preview and there are a number of known issues, or behavior that is likely to change in the future:

  • Query DSL filtering on the main query will currently affect the source indices in the view definition, and this will change in later releases.
    • The future design will have the query filtering impact the output of the view, not the source indices
  • METADATA directives inside and outside a view definition behave the same as they do for METADATA in subqueries. This will change for views.

The following examples show how to use views within the FROM command.

Assume we've defined three views in a similar way to the example above, each counting the number of documents that reference a particular country, but from three different source indices:

  • country_airports - reports counts of documents per country from our airports index
  • country_addresses - reports counts of documents per country from our addresses index
  • country_languages - reports counts of documents per country from our languages index

Now we can query these together with a query like:

FROM country_addresses, country_airports, country_languages
| WHERE country LIKE "United*"
| SORT country ASC, count DESC
		
count:long country:keyword
17 United Kingdom
1 United Kingdom
129 United States
1 United States
1 United States

The same country might appear in multiple views, producing multiple rows. We could combine these with a STATS command, using SUM(count) BY country.

FROM country_*
| STATS count=SUM(count) BY country
| WHERE count > 11
| SORT count DESC, country ASC
		
count:long country:keyword
131 United States
50 India
45 Mexico
41 China
38 Canada
31 Brazil
26 Russia
18 United Kingdom
17 Australia
13 Argentina
13 Germany
12 France
12 Indonesia

Note how we used SUM to combine the counts of the three previously aggregated count columns.

We can define views with complex queries, including commands like LOOKUP JOIN:

				PUT /_query/view/airports_mp_filtered
					{
    "query": """
        FROM airports
        | RENAME abbrev AS code
        | LOOKUP JOIN airports_mp ON abbrev == code
        | WHERE abbrev IS NOT NULL
        | DROP code
       """
}
		

This creates a view called airports_mp_filtered that contains all rows from the airports index that also have a matching abbrev inside the airports_mp index. This is effectively a subset of the airports index.

We could, for example, see how many airports are defined only in airports versus how many are defined in the view, by combining both a view and an index in the same FROM command:

FROM airports_mp_filtered, airports
| STATS duplications=COUNT() BY abbrev
| STATS count=COUNT() BY duplications
| SORT count DESC
		
count:long duplications:long
880 1
7 2
1 3

The METADATA directive is supported both inside and outside a view, and follows the same rules as observed for METADATA in Subqueries. Inside the view it generates columns, just like other fields, and these can be used for filtering and as output columns.

Outside the view it generates null values. Note that this is a known limitation of the current tech-preview, and is anticipated to be addressed in a future update, at which point METADATA _index will contain the name of the view.

Views, subqueries and the FORK command are related. There are many similarities and differences between them.

  • FORK allows data coming from previous commands, like an initial FROM index command, to be processed in parallel in multiple different branches, each performing different commands on the same original data.
  • Subqueries also enable parallel processing, but allow each branch to use a different source index with a different FROM command per branch. Views are reusable, named queries that act like virtual indices. Each view has its own FROM command and processing pipeline, and can be referenced like a regular index.
  • Dynamic execution. All three mechanisms will process the entire set of query definitions at query time, resulting in an up-to-date response when source indexes are changed and the query is re-run.
  • Union of columns. Columns from the results of multiple branches are merged into the main query, expanding the table of results, and inserting null values if any branch has different columns than the others.
  • Supported commands. Complex processing commands can be used inside both views and subqueries, as detailed in the description of subqueries.
  • No nested branching. Nested branching is generally not supported, but views can work around this limitation through query compaction.
  • Maximum branch count. All of these approaches to parallel processing are bound by the same maximum branch count of 8.

The FORK command never includes a FROM command, and relies entirely on an existing query to provide the incoming columns. This also means that all branches will receive identical incoming data, the same columns and the same rows. This is not true of subqueries or views, which can receive completely different columns and rows from their own FROM commands. Only one FORK command is allowed per query, so nested branches are not possible. This limitation is partially true for views and subqueries, but to a lesser extent as described below.

Views have names, and these names are unique within the index namespace. This means a view cannot have the same name as an index, and vice versa. Views can be nested within one another, as long as neither of the following two rules are broken:

  • Cyclic references are not allowed. For example, if viewA references viewB and viewB references viewC it is not allowed to have viewC reference viewA.
    • Detection of cyclic references is done at main query execution time
  • Multiple branching points do not exist

This last point highlights a difference between views and subqueries. While subqueries simply disallow the use of further subqueries or FORK within a subquery, views will allow this under limited conditions.

Consider two views, each defined as a pair of subqueries:

				PUT /_query/view/view_x
					{
    "query": """
        FROM (
            FROM app-events-* | KEEP msg, level
        ), (
            FROM auth-events-* | KEEP msg, level
        )
       """
}
		
				PUT /_query/view/view_y
					{
    "query": """
        FROM (
            FROM nginx-events-* | KEEP msg, level
        ), (
            FROM apache-events-* | KEEP msg, level
        )
       """
}
		

Used together in a single FROM clause:

FROM other-events, view_x, view_y
| STATS count(msg) BY level
		

This initially resolves to a plan with two levels of branching — three outer branches, two of which branch again inside their view definitions:

flowchart TD
    S["STATS count(msg) BY level"]
    S --> O["other-events"]
    S --> VX["view_x"]
    S --> VY["view_y"]
    VX --> AX["app-events-*"]
    VX --> AU["auth-events-*"]
    VY --> NG["nginx-events-*"]
    VY --> AP["apache-events-*"]

ES|QL allows only one branch level per query, so the equivalent subquery-only form would fail. Views, however, apply query compaction: the inner view branches are flattened into the outer branch set, producing a single-level plan with five branches:

flowchart TD
    S["STATS count(msg) BY level"]
    S --> O["other-events"]
    S --> AX["app-events-*"]
    S --> AU["auth-events-*"]
    S --> NG["nginx-events-*"]
    S --> AP["apache-events-*"]

Compaction does not apply if the view definition contains any commands after its subqueries — those commands would need to run on the combined branch output, so the branch level cannot be collapsed and the query will fail.