A newer version is available. For the latest information, see the
current release documentation.
Elasticsearch version 7.16.0edit
Also see Breaking changes in 7.16.
Deprecationsedit
- Allocation
- Authentication
- Cluster Coordination
- Geo
- ILM+SLM
- Indices APIs
- Infra/Logging
-
-
Change default deprecation logger level to
CRITICAL
#77030 (issue: #76754) - Deprecation logs indexing is enabled by default #78991 (issues: #76292, #78850)
- Exposing the ability to log deprecated settings at non-critical level #79107 (issues: #77482, #78781)
- REST API to reset deprecation indexing cache #78392 (issue: #78134)
- Add category field to deprecation logs #77550 (issue: #67443)
-
Change default deprecation logger level to
- Infra/Scripting
- Java High Level REST Client
-
- Add deprecation headers to HLRC classes #79754
- Machine Learning
-
-
Deprecate
estimated_heap_memory_usage_bytes
and replace withmodel_size_bytes
#80545
-
Deprecate
- Monitoring
Known issuesedit
-
Painless: Any Painless script where a method call resolves a lambda argument to a
UnaryOperator
or aBinaryOperator
type results in anIllegalArgumentException
. This exception results in a compilation error when static types are used, or a runtime error for dynamic types. Use standard procedural statements as a workaround, such asif/else
,while
, andfor
statements.The following example is a
UnaryOperator
type that will result in a failure:List l = ['abc', '123']; l.replaceAll(f -> f.replace('abc', 'doremi');
The following example is a possible workaround:
List l = ['abc', '123']; for (int i = 0; i < l.size(); ++i) { l.set(i, l.get(i).replace('abc', 'doremi')); }
-
Parsing a request when the last element in an array is filtered out
(for instance using
_source_includes
) fails. This is due to a bug in Jackson parser. Fixed in Elasticsearch 8.6.1 (#91456)
New featuresedit
Enhancementsedit
- Aggregations
- Allocation
- Audit
- Authentication
-
- Remove HTTPS check for API Keys & Service Accounts #76801
- Authorization
-
-
Add privileges to
kibana_system
for upgrading endpoint hidden indices #80140 - Allow fleet-server service account to set up Fleet #78192 (issue: #78078)
-
Grant additional privileges for endpoint transform indices to
kibana_system
#79619 -
Grant privileges required by package upgrade to
kibana_system
#78049 (issue: #77294) -
Optimize FLS/DLS setup in
IndicePermission
authz #77832 - Skip loading authorized indices if requests do not need them #78321
-
Superuser fastpath for
indexAccessControl
#78498 -
Update transform destination index privilege for
kibana_system
#79076 -
Use fixed size memory allocation in
IndicesPermission
#77748 - Add proper permissions to fleet server for Endpoint response index #80231
- Migrate custom role providers to licensed feature #79127
-
Add privileges to
- CCR
- Cluster Coordination
-
- Add cluster state serialization stats #78816
- Add timing stats to publication process #76771 (issue: #76625)
- Get hot threads on lagging nodes #78879
- Improve error message in 8.x to 7.x downgrade #78644 (issue: #78638)
-
Improve logging in
LeaderChecker
#78883 - Mention "warn threshold" in master service slowlog #76815 (issue: #76625)
- Recycle pages used by outgoing publications #77407 (issue: #77317)
- Reuse previous indices lookup when possible #79004 (issues: #77888, #78980)
- Validate PING and STATE connections at join time #77741
- Composite aggregations
- Data streams
- Distributed
- EQL
- Engine
- Geo
- ILM+SLM
-
-
Improve
LifecycleExecutionState
parsing #77855 (issue: #77466) -
Reduce the number of times that
LifecycleExecutionState
is parsed when running a policy #77863 (issue: #77466) - Add built-in ILM policies for common user use cases #76791
- Allow for setting the total shards per node in the Allocate ILM action #76134
-
Improve
- Indices APIs
- Infra/Core
-
-
Add optional content checking to
ResourceWatcher
#79423 -
Add other time accounting in
HotThreads
#79392 -
Add upgrade preparation and completion callbacks to
SystemIndexPlugin
#78542 -
Cache
index.hidden
setting #78612 (issue: #77974) - Enable wait/blocked time accounting #77935 (issue: #72376)
- Handle empty /proc/self/cgroup file #78659 (issue: #77833)
- Implement framework for migrating system indices #78951
- Require System Index Descriptors to allow a specific suffix #78355
-
Support mem type in nodes
hot_threads
API #72850 (issue: #70345) -
Use enum field for
HotThreads
report type #77462
-
Add optional content checking to
- Infra/Node Lifecycle
-
- Enable exit on out of memory error #71542
- Infra/Scripting
-
- Add a direct sub classes data structure to the Painless lookup #76955
- Add ability to augment classes with fields from other classes in Painless #76628
- Add dynamic (duck) type resolution to Painless static types #78575
- Adds a lookup method to Painless for finding methods of all sub classes #77044
-
UnsignedLong
field type converter #77271 - Compile/cache eviction history metric placeholders #78257 (issue: #62899)
- Infra/Settings
- Ingest
-
- Add enrich node cache #76800 (issue: #48988)
- Add indices pipeline settings check when deleting a pipeline #77013
- Allow range types to be used for enrich matching #76110
- ECS support for Grok processor #76885 (issue: #66528)
- Improving cache lookup to reduce recomputing / searches #77259
- Make enrich policy execution cancelable #77188 (issue: #48988)
- Optimistic concurrency control for updating ingest pipelines #78551 (issue: #77031)
- Sync grok processor patterns with Logstash #76752
- Updating ingest pipeline without changes is no-op #78196 (issue: #77382)
- Java High Level REST Client
- License
- Machine Learning
-
-
Add new
normalize_above
parameter top_value
significant terms heuristic #78833 -
Add new default char filter
first_line_with_letters
for machine learning categorization #77457 -
Add new
defer_definition_decompression
parameter to put trained model API #77189 (issue: #77132) - Enable ML on macOS on ARM #78203
- Track feature usage for data frame analytics, inference, and anomaly jobs #76789
- Speed up training of regression and classification models #2024
- Improve concurrency for training regression and classification models #2031
-
Improve aspects of implementation of
skip_model_update
rule #2053 - Make sure instrumentation captures the best hyperparameters found for training classification and regression models https://github.com/elastic/ml-cpp/pull/2057{#2057}
-
Add new
- Mapping
- Monitoring
-
- Remove license check for monitoring data retention #79010
- Packaging
- Recovery
- Search
-
-
Add
_ignored
and_routing
metatada fields to fields API #78981 (issues: #75836, #78828) -
Add
_index
and_version
metatada fields to fields API #79042 (issues: #75836, #78828) -
Add ability to retrieve
_id
via fields option #78828 (issue: #75836) - Add node-level field capabilities API requests #79212 (issues: #74648, #77047, #78647)
- Add segment sorter for data streams #75195
- Add sort optimization with after from Lucene #64292
- Don’t always rewrite the Lucene query in search phases #79358
- Expand segment sorter for all timeseries indices #78639 (issue: #75195)
- Node level can match action #78765
- Search - return ignored field values from fields API #78697 (issue: #74121)
- Support request cache on frozen tier #77694 (issue: #75309)
-
Use
search_coordination
threadpool for field capabilities API requests #79378 (issue: #79212) - Create a sha-256 hash of the shard request cache key #74877 (issue: #74061)
-
Add
- Security
- Snapshot/Restore
-
- Add filtering by SLM policy to get snapshots API #77321
- Add sort by shard count and failed shard count to get snapshots API #77011
- Add descriptions to various tasks #76700
- Add maintenance service to clean up unused docs in snapshot blob cache #77686
- Add periodic maintenance task to clean up unused blob store cache docs #78438 (issue: #77686)
-
Filter Unneeded
SnapshotInfo
Instances Early inTransportGetSnapshotsAction
#78032 (issue: #74350) - Implement exclude patterns for snapshot and repository names in get snapshots API #77308
- Implement sort by repository name in get snapshots API #77049
-
Implement
from_sort_value
parameter in get snapshots API #77618
- Stats
- Transform
Bug fixesedit
- Aggregations
-
- Add extra round trip to aggregation tests #79638 (issue: #73680)
-
Fix rate aggregation with custom
_doc_count
#79346 (issue: #77734) - Fix several potential circuit breaker leaks in aggregators #79676
- Scale doubles to floats when necessary to match the field #78344 (issue: #77033)
- Support for aggregation names with dots in first element path of a pipeline aggregation #77481
- Allocation
- Authorization
- CAT APIs
-
-
Adjust
_cat/templates
to not request all metadata #78829
-
Adjust
- CCR
- CRUD
- Cluster Coordination
-
- Avoid early release of local forking requests #77641 (issues: #77407, #77634)
-
Check for global blocks after
IndexNotFoundException
inTransportMasterNodeAction
#78128 (issue: #70572) - Improve control of outgoing connection lifecycles #77295 (issue: #67873)
- Only remove active peer on connection failure #79557 (issues: #77295, #79550)
-
Reduce merging in
PersistedClusterStateService
#79793 (issue: #77466)
- CompositeAggs
- Data streams
-
- Add replicated field to get data stream API response #80988 (issue: #118899)
- Correct check for write index and increment generation on all data stream backing index operations #79916
- Fix data stream bug causing it to rollover to a non-existent Index #79759
-
Fix
IndexNotFoundException
error when handling remove alias action #80312 - Fix data stream alias validation #81040 (issue: #80972)
- Distributed
- EQL
-
- Add optional fields and limit joining keys on non-null values only #79677
- Geo
- ILM+SLM
- Indices APIs
- Infra/Core
-
- Avoid spurious deprecation warnings when calling Deprecation Info API #78151 (issue: #78098)
-
Fix overflow/underflow in
CompositeBytesReference
#78893 - Fix race condition in Feature Migration Status API #80572 (issue: #79680)
- Prevent stack overflow in rounding #80450
-
Set
LIBFFI_TMPDIR
at startup #80651 (issues: #18272, #73309, #74545, #77014, #77053, #77285, #80617) - Strip index blocks from settings for reindex targets #80887 (issue: #80654)
- Eschew leniency when parsing time zones #77267 (issues: #73955, #76415)
- Fix ingest timezone parsing #63876 (issue: #63458)
- Infra/Logging
- Infra/Settings
- Ingest
- Java High Level REST Client
- Machine Learning
-
- Address potential ML feature reset permissions bug #79179
- Adjust ML memory tracker to reduce logging impact #78482
- Audit job open failures and stop any corresponding datafeeds #80665 (issues: #48934, #80621)
- Fix acceptable model snapshot versions in ML deprecation checker #81060 (issues: #79387, #81039, #119745)
- Fix autoscaling capacity consistency #81181
- Fix bug in inference stats persister for when feature reset is called #77213 (issue: #77182)
- Fix datafeed preview with remote indices #81099 (issue: #77109)
- Fix language identification bug when multi-languages are present #80675
-
Fix model snapshot sorting when sorting by
min_version
#80596 (issue: #80561) - ML legacy index templates that are no longer needed should be deleted #80874 (issue: #80876)
- Need to tolerate .ml-config being an alias #80025
- Parent datafeed actions to the datafeed’s persistent task #81143
- Wait for .ml-state-write alias to be readable #79731 (issue: #79636)
- Improve the estimates of hyperparameter importance in data frame analytics and avoid incorrectly stopping the hyperparameter search prematurely #2073
- Fix numerical instability in hyperparameter optimization for training regression and classification models #2078
- Fix numerical stability issues in time series modeling #[2083]
- Mapping
- Packaging
-
-
Ensure
LIBFFI_TMPDIR
is exported by init script #80794
-
Ensure
- Recovery
- SQL
- Search
-
- Minimize search source of shard level search requests #80634 (issue: #80187)
-
Prevent
NullPointerException
inSourceConfirmedTextQuery
#80472 (issue: #80419) -
_terms_enum
APIindex_filter
doesn’t work with_tier
field on upgraded cluster #79553 (issue: #79200) - Undeprecate the auto complete thread pool #80204
- Wildcard field regex query fix #78839 (issue: #78391)
- Snapshot/Restore
-
- Fix Queued Snapshot Clone not Starting after Data Node Drops Out #77111 (issue: #77101)
- Fix Temporarily Leaking Shard Level Metadata Blobs in some Cases #76562
- Fix after restore Lucene.pruneUnreferencedFiles() conditional #81047 (issues: #68821, #75308)
-
Improve handling of corrupt
index.latest
blob #77339 - Submit GCS delete batch requests incrementally #80540
- Fix repository-azure for empty settings on reload #79559
- Transform
- Watcher