<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0">
  <channel>
    <title><![CDATA[Tyler Perkins - Elasticsearch Labs]]></title>
    <description><![CDATA[Articles and tutorials from the Search team at Elastic]]></description>
    <copyright><![CDATA[© 2026. Elasticsearch B.V. All Rights Reserved]]></copyright>
    <image>
      <title><![CDATA[Tyler Perkins - Elasticsearch Labs]]></title>
      <url>https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt1121c0bf0e8a6e65/6a88da6340a1841030ef456f/search-labs-thumbnail.png</url>
      <link>https://www.elastic.co/search-labs/author/tyler-perkins</link>
    </image>
    <link>https://www.elastic.co/search-labs/author/tyler-perkins</link>
    <atom:link href="https://www.elastic.co/search-labs/rss/author/tyler-perkins.xml" rel="self" type="application/rss+xml"/>
    <language><![CDATA[en]]></language>
    <lastBuildDate>Wed, 16 Sep 2026 22:13:16 GMT</lastBuildDate>
  <item>
    <title><![CDATA[Three indices walk into a FROM clause: ES|QL subqueries in Elasticsearch]]></title>
    <description><![CDATA[ES|QL subqueries give each data source its own pipeline and filters, eliminating CASE chains, restoring predicate pushdown, and making multi-index queries extensible by design.]]></description>
    <content:encoded><![CDATA[<p><a href="https://www.elastic.co/docs/reference/query-languages/esql">Elasticsearch Query Language</a> (ES|QL) now has <a href="https://www.elastic.co/docs/reference/query-languages/esql/esql-subquery">subqueries in </a><a href="https://www.elastic.co/docs/reference/query-languages/esql/esql-subquery"><code>FROM</code></a>. Three indices, different schemas, one query; each source gets its own pipeline with its own filters and transforms. No more <code>CASE</code> chains. No more client-side stitching. Add a fourth source? Add a fourth branch; zero changes to the existing three.</p><h2>The problem: Heterogeneous data, one query</h2><p>Consider a production incident investigation. Errors are spread across three microservices: an API gateway, a payments service, and an auth service, each with different field names and different conventions. Before subqueries, combining them in a single ES|QL query meant cramming everything into one <code>FROM</code> with <code>CASE</code> chains:</p><p>This is brittle and slow. The disjunctive <code>OR</code> prevents predicate pushdown; every index scans every condition. Every <code>CASE</code> chain grows with every source. Copy it into five dashboards and three alert rules, and you have eight places to update when anything changes.</p><h2>The fix: Independent pipelines</h2><p>Subqueries replace the monolithic <code>FROM</code> + <code>CASE</code> pattern. Each data source gets its own complete pipeline:</p><p>The gateway branch only scans for HTTP 500s. The payments branch only looks at transaction statuses. The auth branch only checks login failures. Because each branch has its own <code>WHERE</code>, the optimizer pushes filters independently into each index, restoring the predicate pushdown that a single <code>FROM</code> with <code>OR</code> conditions prevents. Fields that exist in one branch but not another are filled with <code>null</code>.</p><p>Adding a fourth service means adding a fourth branch. Existing branches don't change.</p><img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltad1d619d46212715/6a170d64e8fbce71fd39fcd8/f210b98824cb34c514b6cdb2d44b2bc81a51bb7b-1999x1084.png" alt="Diagram comparing two approaches. The left side is labeled “Before: EVAL + CASE: Sequential &amp; Brittle” and shows a vertical sequence with components titled “Complex Filtering (OR Conditions),” “EVAL source = CASE (dataset == …),” “Sequential Processing Chains,” “Linear Transformation,” and “Final Aggregation,” with callouts for “Choke Point” and “Latency Bottleneck.” The right side is labeled “After: Subqueries: Parallel &amp; Optimized” and shows three parallel branches for “weblogs-,” “applogs-,” and “securitylogs-*,” each with its own WHERE clause and EVAL source assignment, feeding into a “Unified Output Stream (UNION ALL Semantics).” A code block appears below the branches." /><h2>Save it as a view</h2><p>This is where subqueries and <a href="https://www.elastic.co/search-labs/blog/elasticsearch-esql-logical-views">logical views</a> combine. Wrap the subquery above in a named view, with one API call:</p>PUT _query/view/error_triage
{
  "query": "FROM (FROM svc-gateway-* | WHERE ...) , (FROM svc-payments-* | WHERE ...) , (FROM svc-auth-* | WHERE ...)"
}<p>Now consumers just write <code>FROM error_triage | STATS error_count = COUNT(*) BY service</code>. Three indices, three pipelines, one name. If you have 10 dashboards and five alert rules consuming this pattern, that's 15 copies of the same logic today; with a view, it's one definition and zero consumer-side edits when you add a fourth service. See <a href="https://www.elastic.co/search-labs/blog/elasticsearch-esql-logical-views">Elasticsearch ES|QL Views</a> for the full views deep dive.</p><h2>What you can do inside a branch</h2><p>Each branch supports the full ES|QL pipeline: <code>WHERE</code>, <code>EVAL</code>, <code>STATS</code>, <a href="https://www.elastic.co/docs/reference/query-languages/esql/commands/lookup-join"><code>LOOKUP JOIN</code></a>, <a href="https://www.elastic.co/docs/reference/query-languages/esql/commands/enrich"><code>ENRICH</code></a>, and more. See the <a href="https://www.elastic.co/docs/reference/query-languages/esql/esql-subquery">subquery documentation</a> for the complete list.</p><h2>Aggregate different metrics, and then combine</h2><p>Each branch can compute its own summary before results are merged. This is useful when different indices track the same concept under different field names:</p><p>Both branches produce <code>avg_latency</code> and <code>hour</code>, but each computes it from a different source field. The combined result is a single table you can chart or alert on, without normalizing field names at ingest time. This pattern is impossible with a single <code>FROM</code>; you can't compute different aggregations per index without subqueries.</p><h2>Subqueries vs. FORK</h2><p>ES|QL also has <a href="https://www.elastic.co/docs/reference/query-languages/esql/commands/fork"><code>FORK</code></a> (now generally available), which creates parallel execution branches from the same input. The distinction:</p><img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt6dd3fd7c3afbea5a/6a170d66b0367d56f572bd97/b5728fa2ce08df32b67ef9368c4d75d04b74ab95-1999x992.png" alt="Chart titled “Subqueries vs. FORK” with two columns comparing Subqueries and FORK across four rows: Data Sources, Processing, Mental Model, and Use Case. The Subqueries column shows different indices labeled Index A and Index B, independent FROM pipelines labeled pipeline 1 and pipeline 2, the mental model “Many inputs → one combined result,” and a use case combining web, app, and security logs with corresponding icons. The FORK column shows the same input data labeled Index A, transforms applied to the same rows labeled Transform A and Transform B, the mental model “One input → many analyses,” and a use case of full‑text search and KNN on the same index with magnifying glass and graph icons." /><p>Different indices → subqueries. Same data, different analyses → FORK.</p><h2>How this compares</h2><p>If you're coming from other query languages, here's how ES|QL subqueries stack up at the time of writing:</p><p><strong>Splunk SPL/SPL2</strong> has <code>append</code> and <code>multisearch</code> in classic SPL, and SPL2 adds a <a href="https://help.splunk.com/en/splunk-cloud-platform/search/spl2-search-reference/union-command/union-command-examples">union command</a> that merges events from multiple datasets (the closest analogue to ES|QL subqueries). Federated Search extends this across remote Splunk deployments (analogous to CCS). The differences are in how the engine handles each branch: ES|QL subqueries give each branch independent predicate pushdown, meaning filters are pushed into each index's shard-level structures separately. SPL2 <code>union</code> merges datasets but optimization across branches is limited to what the search scheduler can parallelize. Wrapping ES|QL subqueries in a <a href="https://www.elastic.co/search-labs/blog/elasticsearch-esql-logical-views">view</a> gives you engine-level encapsulation with role-based access control (RBAC); Splunk's equivalent is saved searches and macros, which are text substitution expanded at parse time.</p><p><strong>SQL databases</strong> have <code>UNION ALL</code>, which is the closest analog. The difference is that SQL <code>UNION ALL</code> typically requires matching column counts and types at parse time. ES|QL subqueries are more forgiving; columns that exist in one branch but not another get null-padded automatically, which matters when your sources have different schemas (the norm in observability data). SQL views solve the reuse problem similarly, but ES|QL views are cluster-level objects, not database-scoped; they work across <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-cross-clusters.html">cross-cluster search</a> boundaries.</p><p><strong>Grafana / Datadog / other dashboarding tools</strong> handle multisource composition at the visualization layer: Run separate queries, merge in the panel. This works for display but breaks for alerting, downstream queries, and anything that needs a single result set programmatically. ES|QL subqueries push the composition into the engine, so alerts, views, and API consumers all get the same unified result.</p><p>Capability</p><p>Splunk SPL/SPL2</p><p>SQL UNION ALL</p><p>Dashboard-layer merge</p><p>ES|QL subqueries</p><p>Independent filters per source</p><p>SPL2 `union` merges datasets; optimization is scheduler-level</p><p>Yes</p><p>N/A (separate queries)</p><p>Yes; parallel with pushdown</p><p>Schema mismatch handling</p><p>Manual field normalization</p><p>Strict column matching</p><p>Manual in panel config</p><p>Automatic null-padding</p><p>Engine-level reuse</p><p>Text macros (parse-time expansion)</p><p>Database-scoped views</p><p>Dashboard variables</p><p>Cluster-level views with RBAC</p><p>Works for alerts + API</p><p>Limited (summary indexing)</p><p>Yes</p><p>No; display only</p><p>Yes</p><p>Add a source</p><p>Edit every macro/saved search</p><p>Add a UNION branch</p><p>Add a panel query</p><p>Add a branch; existing branches unchanged</p><h2>Current constraints</h2><p>In the Tech Preview release, subqueries are non-correlated; branches run independently and can't reference the outer query. They're supported in <code>FROM</code> only (not <code>TS</code>), and <code>FORK</code> can't be used inside or after subqueries. See the <a href="https://www.elastic.co/docs/reference/query-languages/esql/esql-subquery">subquery documentation</a> for details.</p><h2>What's next for subqueries</h2><p><a href="https://github.com/elastic/roadmap/issues/60"><code>WHERE</code></a><a href="https://github.com/elastic/roadmap/issues/60"> subqueries</a> — <code>WHERE field IN (FROM other_index | ...)</code> and other correlated forms — will extend the composition model from <code>FROM</code> into filtering. This brings the familiar SQL pattern of nested filtering to ES|QL.</p><h2>Try it</h2><p>Subqueries in <code>FROM</code> are available as a Tech Preview. Try them in <a href="https://www.elastic.co/kibana">Kibana</a> Dev Tools or Discover. We'd love your feedback; file a <a href="https://github.com/elastic/elasticsearch/issues">GitHub issue</a> with the <code>ES|QL</code> label.</p><p><em>ES|QL subqueries in FROM are a Tech Preview feature. Tech Preview features are subject to change and are not covered by the support SLA of GA features. The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.</em></p>]]></content:encoded>
    <link>https://www.elastic.co/search-labs/blog/esql-subquery-from</link>
    <guid isPermaLink="true">https://www.elastic.co/search-labs/blog/esql-subquery-from</guid>
    <category><![CDATA[ES|QL]]></category>
    <dc:creator><![CDATA[Tyler Perkins]]></dc:creator>
    <enclosure url="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltddbbebe635e925fd/6a170d681949f7318ae7aaa5/2eb755dd2b2b69b8e0e8867a0da85940eb744176-1280x720.png" length="0" type="image/png"/>
    <pubDate>Wed, 06 May 2026 00:00:00 GMT</pubDate>
  </item>
  <item>
    <title><![CDATA[Elasticsearch ES|QL views: One query to rule twelve dashboards]]></title>
    <description><![CDATA[With ES|QL views, you only need one query for multiple dashboards. Define it once and let Elasticsearch keep everything in sync.]]></description>
    <content:encoded><![CDATA[<p>Elasticsearch Query Language (ES|QL) now has <a href="https://www.elastic.co/docs/reference/query-languages/esql/esql-views">logical views</a>. Define a query once, and reference it by name in <code>FROM</code>, like an index. Twelve dashboards, one definition, zero copy-paste. Update the view, and every consumer gets the change automatically.</p><p>Views don't store data; they re-execute on every read, so results always reflect the current data and the current definition. If you've used views in SQL databases, this will feel familiar. The difference: ES|QL views are engine-level virtual indices stored at the Elasticsearch cluster level, not saved query text that gets expanded client-side. They appear in <a href="https://www.elastic.co/kibana">Kibana</a> autocomplete, support <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-cross-clusters.html">cross-cluster search</a> (CCS), and are governed by dedicated role-based access control (RBAC) privileges.</p><h2>A simple view</h2><p>A view can wrap any ES|QL query. Start with a straightforward filter — HTTP 500 errors from the API gateway:</p><p>Now anyone can write <code>FROM error_triage</code> without knowing the index pattern or filter condition:</p><p>The query is defined once. Consumers reference a name.</p><img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltdab7918cab2c71e5/6a1706e8cf4f2555e2b2d0d2/68ff5a52b0f3ed3dfaa07d2af6e7f08a8c9c0f55-1999x702.png" alt="Dashboard interface showing a query editor at the top with the text “FROM error_triage | STATS error_count@error_triage | SORT error_count DESC.” A bar chart displays error counts for three services: payments, gateway, and auth. Below the chart, a results table lists the same services with error counts of 194 for payments, 37 for gateway, and 19 for auth." /><p>Views support full create, read, list, update, and delete (CRUD) via the <a href="https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-esql-put-view"><code>_query/view REST API</code></a>.</p><h2>Update propagation</h2><p>Say the team decides <code>error_triage</code> should also capture client errors, not just 500s. Update the definition in place:</p><p>Every dashboard panel, alert rule, and ad-hoc query using <code>FROM error_triage</code> immediately reflects the broader filter. No saved objects to hunt down. No stale copies. Change once, update everywhere.</p><img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltcd475eb5e0a666f8/6a1706ea6234e0e4dedb1973/aad2b22fd4d45f8db1140ae93429c6b9ca345031-1999x440.png" alt="Side‑by‑side comparison with the headings “Without Views” on the left and “With Views” on the right. Under “Without Views,” a central document connects to multiple window icons with tangled red arrows and the caption “Manual find‑and‑replace across saved objects.” Under “With Views,” a central document labeled “critical_errors” connects to similar window icons with green arrows and green check marks, along with the caption “Change once, update everywhere automatically.”" /><h2>Nested views</h2><p>Views can reference other views, enabling layered abstractions. Create views for suspicious IPs and threat intelligence, and then compose them:</p><p>Security teams query <code>FROM security_overview</code> without knowing the underlying data model. They're also shielded from any changes made to <code>suspicious_ips</code> by its owner; the abstraction boundary is real, not syntactic.</p><img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt74d422d44d683f0e/6a1706ec67045b3abd45c13e/fd8f17e0a5bd80a73737f0fe08f9e3480d70ad68-1822x642.png" alt="Diagram titled “From Text to Topography,” showing three stacked layers. The bottom layer, labeled “Bedrock – Raw Indices,” contains “svc-auth-*” and “threat-intel.” The middle layer, labeled “Base Views,” contains “suspicious_ips” and “known_threats.” The top layer, labeled “Operational View,” contains “security_overview.” Arrows point upward from each lower layer to the next. A text box on the right states that consumers only query the top layer and changes below cascade upward instantly." /><h2>Multisource views with subqueries</h2><p>A view can wrap any ES|QL query, including multisource compositions, using <a href="https://www.elastic.co/search-labs/blog/esql-subquery-from"><code>subqueries in FROM</code></a>. Each subquery branch queries one service independently (its own filters, its own field normalization), and the results combine automatically:</p><p>Consumers just write:</p><p>Two indices, two independent pipelines, one name. To add a third service later, add a third branch; existing branches don't change, and every downstream dashboard and alert reflects the update automatically. For a deep dive on subquery syntax and what you can do inside each branch, see <a href="https://www.elastic.co/search-labs/blog/esql-subquery-from">Three Indices Walk Into a FROM Clause</a>.</p><h2>How views work under the hood</h2><p>When you write <code>FROM view_name</code>, ES|QL resolves the view's stored query and executes it inline. Views are re-executed on every read, so results always reflect the current data and the current definition.</p><p>Views share a namespace with indices, aliases, and data streams. A view cannot have the same name as any of these (enforced at creation time). This keeps <code>FROM my_name</code> unambiguous regardless of whether the name resolves to a view, an index, or an alias.</p><h2>Security model</h2><p>Views are governed by four dedicated RBAC privileges: <code>create_view</code>, <code>read_view_metadata</code>, <code>delete_view</code>, and <code>manage_view</code>. Elasticsearch checks the privileges of the user running the query (invoker security), not the user who defined the view. The user querying a view needs permissions on both the view and its underlying indices.</p><h2>Kibana integration</h2><p>Views appear in Discover's ES|QL editor autocomplete alongside indices. ES|QL-based dashboard panels work with views transparently. In the initial Tech Preview release, view management is API-only. A Kibana UI for creating and managing views is planned.</p><h2>Cross-cluster search</h2><p>A view's definition can reference remote indices using <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-cross-clusters.html">CCS syntax</a>:</p><p>Consumers query <code>FROM cross_cluster_errors</code> without knowing which clusters are involved.</p><h2>Current constraints</h2><p>In the Tech Preview release, view management is API-only and SET directives can't appear inside view definitions; the caller applies them when querying. Subquery-based views can't be nested inside other multisource <code>FROM</code> expressions. See the <a href="https://www.elastic.co/docs/reference/query-languages/esql/esql-views#esql-views-limitations">views documentation</a> for the full list.</p><h2>What's next for views</h2><p>Views today are always fresh; they re-execute on read. <a href="https://github.com/elastic/roadmap/issues/49">Materialized views</a> flip that tradeoff: Pre-compute once, read instantly. Think pre-aggregated rollup views for Service Level Agreement (SLA) dashboards that load in milliseconds instead of scanning raw data on every refresh. A Kibana CRUD UI for views, including a "Save as View" workflow in Discover, is also planned.</p><h2>Try it</h2><p>Logical views are available as a Tech Preview. Try them in <a href="https://www.elastic.co/kibana">Kibana</a> Dev Tools or Discover. We'd love your feedback; file a <a href="https://github.com/elastic/elasticsearch/issues">GitHub issue</a> with the <code>ES|QL</code> label.</p><p><em>ES|QL logical views are a Tech Preview feature. Tech Preview features are subject to change and are not covered by the support SLA of GA features. The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.</em></p>]]></content:encoded>
    <link>https://www.elastic.co/search-labs/blog/elasticsearch-esql-logical-views</link>
    <guid isPermaLink="true">https://www.elastic.co/search-labs/blog/elasticsearch-esql-logical-views</guid>
    <category><![CDATA[ES|QL]]></category>
    <dc:creator><![CDATA[Tyler Perkins]]></dc:creator>
    <enclosure url="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt5417d25f64c77933/6a1706ed1949f76551e7a958/852bff427ac62b79974d88e27ce9670dc132bc46-1280x720.png" length="0" type="image/png"/>
    <pubDate>Tue, 05 May 2026 00:00:00 GMT</pubDate>
  </item>
  </channel>
</rss>