Here are the highlights of what’s new and improved in Elasticsearch 7.16!
SQL: Support for cross-cluster searchedit
Elasticsearch SQL now supports cross-cluster search (CCS) using the
<remote_cluster> maps to a SQL catalog (cluster) and
to a table (index or data stream).
Search: Improved can-match phase for scalabilityedit
If a search hits a large number of shards, the search operation includes a pre-filter phase called the can-match phase. During this phase, Elasticsearch checks if an impacted shard contains data that could potentially match the search query. If not, Elasticsearch doesn’t run the query on the shard.
Previously, the search’s coordinating node sent an individual request to each shard checked during the can-match phase. However, if the search needed to check thousands of shards, the coordinating node would need to handle thousands of requests, resulting in high overhead.
With 7.16, the coordinating node instead sends a single request to each data node during the can-match phase. This request covers can-match checks for all impacted shards on the node, significantly reducing the number of requests and related overhead.
Field capabilities: Results gathered by node for scalabilityedit
Previously, field caps were gathered by sending a request per index to the appropriate nodes in the cluster. This resulted in high overhead when many indices were targeted. Now requests that target indices on a single node are grouped together, resulting in no more than a single request per node.
Frozen tier: Cached requests and queriesedit
Requests and queries against indices in the frozen tier are now cached, which improves performance for subsequent requests and queries.
Enrich processor: New
range enrich policy typeedit
With 7.16, we added the
range enrich policy type for the enrich processor.
You can use a
range policy to enrich incoming documents based on a number,
date, or IP address that matches a range in the enrich index.
For example, if incoming documents contain an IP address, you can use a
range policy to enrich the documents based on their IP range. For an in-depth
example, see Example: Enrich your data by
matching a value to a range.
Data streams: Segment sorting for faster searchesedit
For data streams, Elasticsearch now sorts a backing index’s segments by maximum
@timestamp value in descending order. This speeds up searches sorted by
@timestamp, which are common for data streams.
with runs statements for repeated eventsedit
In 7.16, we added the
with runs statement syntax to EQL sequence queries.
Sometimes you want to find a sequence that contains an event multiple times in
succession. Rather than type the same event criteria multiple times, you can use
with runs statement to declare the criteria once and run it successively.
For more details, check out the
EQL syntax documentation.