<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0">
  <channel>
    <title><![CDATA[Kevin Corcoran - Elasticsearch Labs]]></title>
    <description><![CDATA[Articles and tutorials from the Search team at Elastic]]></description>
    <copyright><![CDATA[© 2026. Elasticsearch B.V. All Rights Reserved]]></copyright>
    <image>
      <title><![CDATA[Kevin Corcoran - Elasticsearch Labs]]></title>
      <url>https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt1121c0bf0e8a6e65/6a88da6340a1841030ef456f/search-labs-thumbnail.png</url>
      <link>https://www.elastic.co/search-labs/author/kevin-corcoran</link>
    </image>
    <link>https://www.elastic.co/search-labs/author/kevin-corcoran</link>
    <atom:link href="https://www.elastic.co/search-labs/rss/author/kevin-corcoran.xml" rel="self" type="application/rss+xml"/>
    <language><![CDATA[en]]></language>
    <lastBuildDate>Tue, 15 Sep 2026 11:16:15 GMT</lastBuildDate>
  <item>
    <title><![CDATA[Elasticsearch ES|QL brings full-text search to data you never indexed]]></title>
    <description><![CDATA[MATCH and TO_TEXT bring full-text search to data you never indexed. Search computed columns, unmapped fields and federated sources in ES|QL.]]></description>
    <content:encoded><![CDATA[<p>ES|QL <code>MATCH</code> now runs full-text search on data you never indexed. Computed columns, unmapped fields, strings assembled on the fly, even federated data sitting in S3. The new <code>TO_TEXT</code> function tells ES|QL to treat any string as analyzable text, so <code>MATCH</code> can tokenize, case-fold and term-match values that exist only for the lifetime of a query. This goes beyond the <code>LIKE</code> and <code>RLIKE</code> pattern matching that most query engines offer for unindexed strings: it's real analysis. Available now in Elastic Cloud Serverless and as a technical preview in Elasticsearch 9.5.</p><h2>How MATCH and TO_TEXT enable full-text search on any ES|QL expression</h2><p>Let's start with a query that was impossible in Elasticsearch 9.4, which uses <a href="https://www.elastic.co/docs/reference/query-languages/esql/commands/eval">the <code>EVAL</code> command</a>:</p><p>In this example, <code>summary</code> has no mapping or analyzer configuration. It’s also not associated with any inverted index. It exists only for the lifetime of this query, but now you can search it anyway. Two additions make this work.</p><p>First, <a href="https://www.elastic.co/docs/reference/query-languages/esql/functions-operators/search-functions/match"><code>MATCH</code></a> now accepts any expression as its first argument, not just a mapped field. That includes columns produced by <code>EVAL</code> and function results used inline. It also includes unmapped fields loaded directly from the original document. Furthermore, all data types normally accepted by <code>MATCH</code> are supported in this new use case.</p><p>The second part of this is the new <a href="https://www.elastic.co/docs/reference/query-languages/esql/functions-operators/type-conversion-functions/to_text"><code>TO_TEXT</code></a> function, which is the first ES|QL conversion function that produces output of type <code>text</code>. Until now, <code>text</code> columns could only come from indexed mapped fields, and all strings produced by ES|QL expressions were <code>keyword</code> values rather than <code>text</code>. The distinction matters because <code>MATCH</code> treats the two differently: <code>text</code> values are analyzed, while <code>keyword</code> values are compared exactly, mirroring how a <code>MATCH</code> query on an indexed keyword field rewrites to a term query. <code>TO_TEXT(x)</code> is how you tell ES|QL: <em>treat this string as full text</em>.</p><p>This ships as a technical preview in Elasticsearch 9.5, and as such, it has some limitations:</p><ul><li><p>It’s currently filtering only. A <code>MATCH</code> on an expression doesn't contribute to the relevance score yet; only matches on indexed fields affect the score.</p></li><li><p>Querying options like <code>fuzziness</code> and others aren't yet supported when matching an expression.</p></li><li><p>Runtime text is analyzed with the standard analyzer. This isn’t configurable yet.</p></li></ul><p>Work is underway to address these limitations.</p><h2>Why use full-text search instead of LIKE or RLIKE in ES|QL?</h2><p>ES|QL already had two ways to search strings without an index: <code>LIKE</code> (wildcard patterns) and <code>RLIKE</code> (regular expressions). Both work on any string expression, so it's fair to ask what <code>MATCH</code> adds. The answer is <a href="https://www.elastic.co/docs/manage-data/data-store/text-analysis">analysis</a>, a more advanced form of search which uses techniques such as stemming and synonyms. It also uses stopword handling.</p><p><code>LIKE</code> is simple substring matching, without any understanding of the words that comprise a string. Say, for example, you're looking for log messages about a fox:</p><p>This misses <code>"Fox spotted near the henhouse"</code> due to the capitalization, while matching <code>"Outfoxed by the competition"</code>, which isn't about a fox at all. It fails in both directions, with false negatives on capitalization and false positives on substrings buried inside other words.</p><p>Regular expressions can patch the case problem, but the word-boundary problem gets ugly fast. Something like:</p><p>And even that's not right yet. It misses a fox at the end of a sentence followed by <code>!</code> or <code>?</code>, and it says nothing about tabs, quotes, or parentheses. Each fix makes the pattern longer, and the next person to read the query has to reverse-engineer what it’s actually doing.</p><p><code>MATCH</code> makes the problem go away, because it runs both the query and the value through an <a href="https://www.elastic.co/docs/reference/text-analysis/analyzer-reference">analyzer</a>, which tokenizes text into lowercase terms and then matches term against term:</p><p>This query will match values like <code>"The quick brown fox"</code> and <code>"FOX spotted near the henhouse"</code> but not <code>"Outfoxed by the competition"</code> or <code>“FOXTROT protocol enabled"</code>, regardless of any punctuation surrounding the words. Of course, this all works for multi-term queries, like <code>MATCH(TO_TEXT(message), "brown fox")</code>, too, just the way you’d expect it to.</p><p>Work is underway to enable the use of the <a href="https://www.elastic.co/docs/reference/text-analysis/analysis-lang-analyzer">36 dedicated language analyzers</a>, with support for natural languages on data that was never indexed or mapped.</p><h2>Full-text search use cases for unindexed and unmapped data</h2><p>The examples above searched values computed from <a href="https://www.elastic.co/docs/manage-data/data-store/mapping">mapped fields</a>. . The more interesting use cases for ES|QL <code>MATCH</code> on expressions involve data that was never searchable at all. Let's walk through a few.</p><h3>How to search unmapped fields in ES|QL without adding a mapping</h3><p>Sometimes you deliberately leave a field out of your mappings, such as a verbose stack trace or a raw request payload. You might even leave out a debug blob. Indexing one of these would cost disk and heap space on every document and wouldn’t be worth it for a field you might query once a quarter.</p><p>That decision has always been final, because <a href="https://www.elastic.co/search-labs/blog/esql-unmapped-fields">unmapped fields</a> were invisible to queries entirely. In Elasticsearch 9.5, you can use <code>SET unmapped_fields="load"</code> to make ES|QL load unmapped fields directly from the source document as keywords. Follow that up by wrapping it in <code>TO_TEXT</code>, and now you can run full-text search on it:</p><p>Here, <code>stack_trace</code> was never mapped. Every value is fetched from the original documents and analyzed on the fly. They’re matched row by row. That’s real work, and it will never be as fast as an <a href="https://www.elastic.co/docs/manage-data/data-store/index-basics">inverted index</a> lookup. But now, that field you didn't index is no longer unsearchable. You get to keep the mapping small for the everyday case and still answer the once-a-quarter question when it matters.</p><h3>Full-text search on a keyword field without reindexing</h3><p>Keyword fields can do a lot. They give you exact matching, fast aggregations, and sorting, which is why so many fields end up mapped that way. But mappings are decided when data arrives, and it’s easy to end up in a situation where you want to do something different with your data than you had originally intended. Maybe <code>product_name</code> was mapped as a <code>keyword</code> because the dashboards aggregate on it, and then, after receiving a year’s worth of product data, someone wants to be able to search within <code>product_name</code> values.</p><p>The old answer was to change the mapping to <code>text</code> (or add a multi-field) and reindex everything. This can be both time-consuming and costly, and in many cases, users simply won’t want to bother with it. The new answer is one function call:</p><p><code>TO_TEXT</code> converts the <code>keyword</code> values to <code>text</code> on the fly, so <code>MATCH</code> analyzes them instead of comparing them exactly. This allows you to query a <code>keyword</code> field without creating a mapping or reindexing the source document. If the search becomes an everyday query, indexing the field as <code>text</code> is still the right long-term move, but <code>TO_TEXT</code> gets you an answer today, without any extra work.</p><h3>Searching the same field across indices with different mappings</h3><p><a href="https://www.elastic.co/docs/reference/query-languages/esql/esql-multi-index">ES|QL can span many indices</a>, and the same field doesn't need to always look the same in all of them. When the same field has different types in different indices, ES|QL treats it as a union type, and a conversion function resolves the conflict. Let’s consider an example in which the <code>message</code> field has type <code>text</code> in this year's index template but was a <code>keyword</code> in last year's:</p><p>Every value is analyzed at query time, whether it came from the <code>text</code> index or the <code>keyword</code> one. The keyword values from the older indices are tokenized and lowercased like everything else, so "connection reset" finds “Connection RESET by peer”, no matter which index it lives in.</p><p>Another interesting case is when a field is mapped in only one index but also present (and unmapped) in the other:</p><p>There's a nuance worth calling out here. If <code>error_details</code> is mapped in <code>logs-2026</code> but not in <code>logs-2025</code>, Elasticsearch cannot push this query down to <a href="https://lucene.apache.org/">Lucene</a>, because the indices where the field is unmapped would silently return no matches. Instead, the planner notices that the field is potentially unmapped and evaluates the whole <code>MATCH</code> row by row, wherever the rows came from. You don't have to know which of your indices have the field mapped; the query just answers the question.</p><h2>How ES|QL analyzes text at query time without an inverted index</h2><p>When ES|QL plans a <code>MATCH</code> against an expression, it analyzes the query string once, up front, into a set of terms. How each row is then evaluated depends on the expression's type:</p><p><strong>Expression type</strong></p><p><strong>Processing</strong></p><p><strong>Matching behavior</strong></p><p><code>text</code> (via <code>TO_TEXT</code>)</p><p>Analyzer tokenizes value into lowercase terms</p><p>Token-against-token comparison; a row matches if any token equals any query term (<code>OR</code> semantics)</p><p><code>keyword</code>,<code>ip</code>, <code>date</code>, numeric</p><p>No analysis; query constant converted once to the native type</p><p>Exact comparison per row</p><p>Both paths bypass Lucene entirely and evaluate values row by row. The non-text path mirrors exactly what a match query does when pushed down to Lucene against those field types, so the semantics stay consistent regardless of whether your query hits an index.</p><p>An inverted-index lookup does its work at ingest time and never touches non-matching documents at query time. A runtime <code>MATCH</code> does that analysis at query time, for every row that reaches it. One is fast because the work already happened; the other is flexible because the data doesn't need to have been indexed at all.</p><h2>What's next for ES|QL full-text search</h2><p>Everything in this post is the first installment of a larger effort to make search in ES|QL work on anything, not just on what you indexed ahead of time. The limitations called out earlier are actively being worked on, and the roadmap goes further:</p><ul><li><p><strong>Scoring.</strong> Runtime matches will contribute to <code>_score</code>, so you can sort by relevance even when the data was never indexed.</p></li><li><p><strong><code>MATCH_PHRASE</code></strong><strong> on expressions.</strong> Already available in Elastic Cloud Serverless, and coming to the Elastic Stack in 9.6.</p></li><li><p><strong>Configurable analyzers.</strong> Analyzer support for <code>MATCH</code> and <code>MATCH_PHRASE</code> on expressions, enabling language analyzers, stemming, and synonyms at query time.</p></li><li><p><strong>Match options.</strong> Options like <code>fuzziness</code> and <code>operator</code> for runtime matches.</p></li><li><p><strong>Vector search.</strong> Generating embeddings per row and running k-nearest neighbors (kNN) on runtime <code>dense_vector</code> expressions, bringing semantic search to unindexed data, too.</p></li></ul><h2>Try ES|QL full-text search on expressions today</h2><p>You can try runtime search today. It's available now in Elastic Cloud Serverless, where new ES|QL capabilities land first, and it ships as a technical preview in Elasticsearch 9.5. Start with the <a href="https://www.elastic.co/docs/reference/query-languages/esql/functions-operators/search-functions">search functions</a> reference, and check the <a href="https://www.elastic.co/docs/reference/query-languages/esql/limitations">ES|QL limitations</a> page for the current boundaries. It's a technical preview because we want your feedback: If you search something that was never indexed and it surprises you, either positively or negatively, <a href="https://www.elastic.co/community">we'd love to hear about it</a>.</p>]]></content:encoded>
    <link>https://www.elastic.co/search-labs/blog/full-text-search-unindexed-data</link>
    <guid isPermaLink="true">https://www.elastic.co/search-labs/blog/full-text-search-unindexed-data</guid>
    <category><![CDATA[ES|QL]]></category>
    <category><![CDATA[Mappings]]></category>
    <dc:creator><![CDATA[Kevin Corcoran,Ioana Tagirta]]></dc:creator>
    <enclosure url="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt9a74e633d05585a8/6a730774b8c2e64c3ebe0fd4/image1.jpg" length="0" type="image/jpeg"/>
    <pubDate>Wed, 05 Aug 2026 00:00:00 GMT</pubDate>
  </item>
  </channel>
</rss>