Elastic Platform 8.14 includes ES|QL GA, the use of AI for pattern recognition in logs, API key based security model for remote clusters, encryption at rest with KMS keys, retrievers, several vector optimizations, and vector quantization by default.

Elastic’s piped query language, ES|QL, is now generally available

Encrypt data and snapshots at rest with customer-managed keys

Retrievers (standard, kNN, and RRF)

1.png

Vector distance function optimized with SIMD (Neon) for int8 vectors

Int8 quantization by default for dense vector fields

General availability of Logstash on ECK

API key-based security model for remote clusters is now GA

2.png

AIOps log pattern analysis is generally available

3.png

Data stream lifecycle settings now GA

4.png

Document comparison mode in Discover and ES|QL

Links panel is GA

Region map goes GA

5.png

 New Spanish plural stemmer

Use MaxMind Enterprise and Anonymous IP files with ingest GeoIP processors

Enrich policies can target data streams

Write to an index after ILM shrink

User information in the slow log

Try it out

platform-release-blog.jpg

Elastic Platform 8.14: ES|QL GA, encryption at rest, and vector search optimizations

In Elastic 8.0, Elastic Stack security is on by default for self-managed clusters. To get started, simply spin up Elasticsearch and Kibana, connect them using the new web UI, and they work together securely, out of the box.

All the security you need with none of the hassle

screenshot-simplified-security-highlights.png

Spin up a new deployment securely in two steps

screenshot-simplified-security-elasticsearch-config.png

screenshot-simplified-security-enrollment-token.png

screenshot-simplified-security-get-started.png

screenshot-simplified-security-login.png

A quick peek under the hood

Impact on the Stack, Elastic solutions, and other products

Get started with the Elastic Stack 8.0 today

blog-thumb-simplified-stack-security.png

Securing your Elastic Stack is now simpler than ever

<p>Today we are pleased to announce the release of Elasticsearch 7.5.0, based on Lucene 8.3.0. Version 7.5 is the latest stable release of Elasticsearch, and is now available for deployment via <a href="https://www.elastic.co/elasticsearch/service">Elasticsearch Service</a> on <a href="https://www.elastic.co/cloud">Elastic Cloud</a>.</p><p>As with the <a href="https://www.elastic.co/blog/elasticsearch-7-4-0-released">previous release</a>, Elasticsearch 7.5 features a number of improvements to <a href="#core">core search and analytics capabilities</a>, <a href="#cluster">cluster management and administration</a>, <a href="#machine">machine learning</a>, and more.</p><p>If you’re interested in learning more about this release's new features, you're in the right place as we dive deeper into many of the release highlights below. Alternatively, if you’d rather skip ahead and get right to work by spinning up a cluster on <a href="https://www.elastic.co/cloud/">Elastic Cloud</a>, downloading the latest bits to your laptop, or reviewing the release notes, we have the links you need:</p><ul><li><a href="https://www.elastic.co/cloud/">Start Elasticsearch on Elastic Cloud</a></li><li><a href="https://www.elastic.co/downloads/elasticsearch">Download Elasticsearch</a></li><li><a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/release-notes-7.5.0.html">Elasticsearch 7.5.0 release notes</a></li><li><a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/breaking-changes.html">Elasticsearch 7.5.0 breaking changes</a></li></ul><h2 id="core"><strong>Core search &amp; analytics improvements</strong></h2><p><strong></strong></p><h3><strong>Simplifying ingest with the new enrich processor</strong></h3><p>We are pleased to announce the addition of the new <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/enrich-processor.html">enrich processor</a>.&nbsp; This powerful new processor enables users to more easily lookup and enrich data as it comes into Elasticsearch. With the new enrich processor in place, users seeking to query or aggregate by information that does not exist in the original documents, but that can be inferred from information in other (existing) indices, is not only possible - it’s easy.</p><p>The ingest pipeline has been a powerful tool for users to control the flow of data, and to extract and transform data in real time. Processors can perform tasks like: IP address lookups, timestamp identification, regex data extraction, and the general massaging of data (with set, rename, remove, lowercasing, and HTML stripping). The enrich processor uses the speed and scale of the ingest pipeline to bring lookup-based data enrichment to documents as they are processed.</p><p>For example, users may search by a department name, while the document that was sent to Elasticsearch only included the department ID, alternatively a user may aggregate by an electoral constituency by using the name of the constituency, while the documents that were indexed only included the geo location (longitude and latitude) of each voter.</p><p><a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/ingest-enriching-data.html#enrich-setup">Here’s how it works</a>: like most things in Elasticsearch it all starts with data. The enrich processor uses data within your existing indices to build a lookup index. This source index could be things like user data, geo location data, IP blacklists, product data, and so forth. Next comes the creation of an <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/put-enrich-policy-api.html">enrich policy</a>. The policy contains 4 items: the type, index, match field, and enrich fields. With respect to types, we have 2 types today: match and geo_match. These types are used by the enrich processor to match incoming documents with data stored in the lookup index. Match issues a term query against the lookup index while the geo_match uses geo_shapes. Once the policy is defined you can <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/execute-enrich-policy-api.html">execute</a> the policy. This creates the lookup index based on the index and fields defined in the policy. The final step is to <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/put-pipeline-api.html">create or add an ingest pipeline</a> for the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/enrich-processor.html">enrich processor</a>. Once this is done, incoming data that uses this pipeline will be enriched automatically. A user can always refresh the lookup index by re-running the policy as new data comes into the source index.</p><p>The enrich processor is being released under Elastic’s free Basic subscription tier.</p><h3><strong>Higher resolution geotile analysis with composite aggregations</strong></h3><p><a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html">Composite aggregations</a> (<a href="https://www.elastic.co/blog/composite-aggregations-elasticsearch-pizza-delivery-metrics">introduced in 6.1</a>) are used to group together all documents (even those from potentially different sources) that have the same values in certain fields. With 7.5, <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-geotilegrid-aggregation.html">geo tile grid aggregations</a> can now be a data source for a composite aggregation. This enhancement enables users to more easily aggregate / collect all documents within a given tile or set of tiles on a geographical map and provides a memory-efficient way to scroll through buckets.</p><p>This enhancement is particularly useful when needing to further process a large quantity of geo tile grid buckets, which was practically impossible prior to this enhancement. The geo tile grid aggregation answered the need for showing data to users on a map, because the number of tiles a user can comprehend is rather small, but what if an extra stage of analysis is performed by a machine, which can, and would benefit from handling, a much larger number of buckets? Enabling geotile grid aggregation to efficiently scroll through buckets facilitates more sophisticated use cases, such as running machine learning tools that utilize the initial analysis performed at scale within Elasticsearch. This also allows for the presentation of fine grained maps, which would have been challenging using geotile aggregation without a composite aggregation.</p><p>Geotile grid aggregation as a source for composite aggregation is being released under the Apache 2.0 license.</p><h3><strong>Shape support in SQL</strong></h3><p>Whether you’re a new user (likely proficient in SQL but not DSL) or seeking to integrate Elasticsearch with systems that have mechanisms for integrating with data sources using SQL, SQL support remains a top priority of the Elasticsearch team. To this end, with the release of Elasticsearch 7.5, <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/sql-functions-geo.html">all SQL functionality that worked for geo_shape</a> will now work for the <a href="https://www.elastic.co/blog/elasticsearch-7-4-0-released#core">new shape field type</a> (new as of Elasticsearch 7.4).</p><p>Shape support in SQL is being released as an experimental feature under Elastic’s free Basic subscription tier.</p><h2 id="cluster"><strong>Cluster management and administration</strong></h2><h3>Snapshot lifecycle management retention</h3><p>Elasticsearch 7.5 improves <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/getting-started-snapshot-lifecycle-management.html">snapshot lifecycle management</a> (SLM), first seen in <a href="https://www.elastic.co/blog/elasticsearch-7-4-0-released#cluster">Elasticsearch 7.4</a>, a background snapshot manager that enables an administrator to define when and how often automatic snapshots of an Elasticsearch cluster are taken.</p><p>New to SLM is the ability to manage <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/slm-retention.html">snapshot retention</a>. SLM retention gives users a configurable and automated way to manage the removal of snapshots. Retention also provides a way for users to ensure they have a minimum number of snapshots even if a snapshot has expired - protecting users from failed snapshots. Elastic also knows that users want a way to protect their storage, so we have created a way to prevent too many snapshots being created with a maximum number of snapshots to keep.</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt49b183efc41d3fb4/5dcd6516cd64b00253a37edd/slm_retention.png" data-sys-asset-uid="blt49b183efc41d3fb4" alt="slm_retention.png"/></p><p>Snapshot lifecycle management retention is being released under Elastic’s free Basic subscription tier.</p><h3>Cluster security enhancements</h3><p>In our <a href="https://www.elastic.co/blog/elasticsearch-7-4-0-released#cluster">last release of Elasticsearch</a> we introduced <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.4/security-privileges.html">new cluster privileges</a> to manage <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.4/security-api.html#security-api-keys">API keys</a>. Elasticsearch 7.5 builds upon this functionality with the introduction of an <a href="https://www.elastic.co/guide/en/kibana/7.5/api-keys.html">API keys app</a> in Kibana. This new user interface allows users to easily view and invalidate their own API keys and administrators to view and invalidate all users' API keys.</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltfaeeda36714f1dc1/5dcd6527288a950182779d5d/api_keys_ui.png" data-sys-asset-uid="bltfaeeda36714f1dc1" alt="api_keys_ui.png"/></p><p>In additional security news, Elasticsearch 7.5 includes the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/security-privileges.html#privileges-list-indices">new create_doc index privilege</a>. With the previous set of index privileges, users that were allowed to index new documents were also allowed to update existing ones.</p><p>With the new create_doc privilege, cluster administrators can create a user that is allowed to add new data only. This gives the minimum privileges needed by ingest agents, with no risk that that user can alter and corrupt existing logs. These administrators can (now) rest assured knowing that components that live directly on the machines that they monitor cannot corrupt or hide tracks that are already into the index.</p><p>Both the API keys user interface and the new create_doc index privilege are being released under Elastic’s free Basic subscription tier.</p><h3>Improvements to upgrades with Cross-Cluster Replication (CCR)</h3><p>In <a href="https://www.elastic.co/blog/elasticsearch-6-7-0-released">March 2019</a> we introduced one of the most heavily requested features for Elasticsearch — <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/ccr-getting-started.html">cross-cluster replication</a> (CCR). CCR has a variety of use cases, including <a href="https://www.elastic.co/blog/cross-datacenter-replication-with-elasticsearch-cross-cluster-replication">cross-datacenter and cross-region replication</a>, replicating data to get closer to the application server and user, and maintaining a centralized reporting cluster replicated from a large number of smaller clusters.</p><p>With the release of Elasticsearch 7.5 we are excited to introduce <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/ccr-pause-auto-follow-pattern.html">pause</a> and <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/ccr-resume-auto-follow-pattern.html">resume</a> API endpoints for CCR auto-flow patterns. CCR bi-directional replication (i.e., cross-replicating indices between multiple Elasticsearch clusters so no manual failover event is needed) is quickly becoming a popular CCR architecture. In an effort to make it easier to upgrade clusters with this architecture, the CCR pause / resume auto-follow patterns API enables a user to temporarily pause CCR auto-follow patterns during the upgrade process, essential to bi-directional replication architectures.</p><p><a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/ccr-pause-auto-follow-pattern.html">Pause</a> / <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/ccr-resume-auto-follow-pattern.html">resume</a> CCR auto-flow patterns are being released under Elastic’s Platinum subscription tier.</p><h2 id="machine"><strong>Machine Learning</strong></h2><h3>Classification API</h3><p>Elasticsearch 7.5 adds the ability to build machine learning <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-df-analytics-apis.html">data frame analytics jobs</a> via API for binary classification. Classification is a machine learning process for predicting the class or category of a given data point in a dataset. With binary classification the variable you want to predict has only two potential values.</p><p>The classification API is being released as an experimental feature under Elastic’s Platinum subscription tier</p><h2><strong>That's&nbsp;not all, folks!</strong></h2><p>While the above listed features might have taken the spotlight, there were many more features included with the release of Elasticsearch 7.5 - be sure to check out the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/release-notes-7.5.0.html">release notes</a> for additional information.</p><p>Ready to get your hands dirty? Spin up an Elasticsearch cluster on <a href="https://www.elastic.co/cloud/">Elastic Cloud</a> or <a href="https://www.elastic.co/downloads/elasticsearch">download Elasticsearch</a> today. Try it out. And be sure to let us know what you think on Twitter (<a href="https://twitter.com/elastic">@elastic</a>) or in <a href="https://discuss.elastic.co/c/elasticsearch">our forum</a>. You can report any problems on the <a href="https://github.com/elastic/elasticsearch/issues">GitHub issues page.</a></p>

blog-banner-release-elasticsearch.png

blog-thumb-release-elasticsearch.png

Elasticsearch 7.5.0 released

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Senior Product Manager

Fabio Busatto

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Communication Preferences - ONLY for /communication-preferences

<p>제출하면 귀하가 <a href="/kr/legal/elastic-cloud-account-terms" target="_self">Elastic 서비스 약관</a>을 읽고 이에 동의하며, 위에 제공된 연락처로 <a href="/kr/legal/privacy-statement#how-we-use-the-information" target="_self">선택한 커뮤니케이션</a>을 수신하는 데 동의한 것으로 간주됩니다. 언제든지 수신을 거부할 수 있으며 자세한 내용은 <a href="/kr/legal/privacy-statement/" target="_self">Elastic 개인정보 취급방침</a>을 참조하시기 바랍니다.</p>

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Serverless GDPR Text

<p>제출하면 귀하가 Elastic <a href="/kr/legal/serverless-preview-terms" target="_self">서비스 약관</a>을 읽고 이에 동의하며, 위에 제공된 세부 정보를 사용하여 Elastic이 관련 제품 및 서비스에 대해 귀하에게 <a href="/kr/legal/privacy-statement#how-we-use-the-information" target="_self">연락할 수 있음</a>을 이해한 것으로 간주됩니다. 언제든지 수신을 거부할 수 있으며 자세한 내용은 <a href="/kr/legal/privacy-statement/" target="_self">Elastic 개인정보 취급방침</a>을 참조하시기 바랍니다.</p>

<p>By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.</p>

Cloud Trial GDPR Text

<p>제출하면 <a href="/kr/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud 서비스 약관</a>에 동의하시게 됩니다. 개인 데이터는 <a href="/kr/legal/privacy-statement">Elastic의 개인정보 취급방침</a>에 따라 처리됩니다.</p>

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Newsletter GDPR Text

<p>제출하면 귀하가 Elastic <a href="/kr/legal/terms-of-use" target="_self">서비스 약관</a>을 읽고 이에 동의하며, 위에 제공된 세부 정보를 사용하여 Elastic이 관련 제품 및 서비스에 대해 귀하에게 <a href="/kr/legal/privacy-statement#how-we-use-the-information" target="_self">연락할 수 있음</a>을 이해한 것으로 간주됩니다. 언제든지 수신을 거부할 수 있으며 자세한 내용은 <a href="/kr/legal/privacy-statement/" target="_self">Elastic 개인정보 취급방침</a>을 참조하시기 바랍니다.</p>

Explore similar demos

Overview

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

이 페이지의 콘텐츠는 선택하신 언어로 제공되지 않습니다. Elastic은 다양한 언어로 콘텐츠를 제공하기 위해 최선을 다하고 있습니다.조금만 더 기다려주세요!

Author

Articles by

Learn more

Watch now (no PT)

Watch now

See all top stories

All (no PT translation)

Contact information

Press Release

Share on Reddit

Share this story

Share by Email

Thank you for your interest!

Contact Info

Follow us on Youtube

Load More

Share on LinkedIn

Follow us on LinkedIn

Search Integrations

All Solutions

Video for

Follow us on Twitter

See when this webinar starts in my time zone

내 표준 시간대로 웨비나 시작 시간 확인하기

Register to Watch

Register now

See All Posts

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

참여가 어려우신가요? 등록하시면 녹화 자료를 보내드리겠습니다. 이메일로 관련 콘텐츠도 받아보실 수 있습니다.

Small image for

On-demand webinar

Featured webinar

Register to Attend

Share on Facebook

Reset all

Upcoming webinar

View more posts

Print

Hosted by

Sign In to Attend

View next

Follow us on Facebook

Share

Highlights

Read less

You'll also receive an email with related content

이메일을 통해 관련 내용을 보내드리겠습니다.

Thank you for registering. We will send you a confirmation email soon.

등록해 주셔서 감사합니다. 곧 확인 이메일을 보내드리겠습니다.

Author

Articles by Fabio Busatto

Senior Product Manager, Elastic

Elastic Platform 8.14: ES|QL GA, encryption at rest, and vector search optimizations

Securing your Elastic Stack is now simpler than ever

Elasticsearch 7.5.0 released