Migrating to 7.16edit

This section discusses the changes that you need to be aware of when migrating your application to Elasticsearch 7.16.

See also What’s new in 7.16 and Release notes.

Breaking changesedit

The following changes in Elasticsearch 7.16 might affect your applications and prevent them from operating normally. Before upgrading to 7.16, review these changes and take the described steps to mitigate the impact.

Breaking changes introduced in minor versions are normally limited to security and bug fixes. Significant changes in behavior are deprecated in a minor release and the old behavior is supported until the next major release. To find out if you are using any deprecated functionality, enable deprecation logging.

Settings changesedit

Disk threshold enabled setting is operator only

Details
Orchestrated environments such as Elasticsearch Service and Elastic Cloud Enterprise rely on the disk thresholds in Elasticsearch to operate the cluster correctly. For example the disk thresholds help determine how large an auto-scaled cluster should be. Disabling these disk thresholds prevents the orchestration system from working correctly, so starting in 7.16.0 the cluster.routing.allocation.disk.threshold_enabled setting is an operator only setting which cannot be changed by end-users.

Impact
Discontinue use of this setting in orchestrated environments such as Elasticsearch Service and Elastic Cloud Enterprise. Contact the environment administrator for help with disk space management if needed.

+ This change has no impact on users outside of orchestrated environments.

Deprecationsedit

The following functionality has been deprecated in Elasticsearch 7.16 and will be removed in 8.0. While this won’t have an immediate impact on your applications, we strongly encourage you take the described steps to update your code after upgrading to 7.16.

Significant changes in behavior are deprecated in a minor release and the old behavior is supported until the next major release. To find out if you are using any deprecated functionality, enable deprecation logging.

Security changesedit

The nameid_format SAML realm setting no longer has a default value.

Details
In SAML, Identity Providers (IdPs) can either be explicitly configured to release a NameID with a specific format, or configured to attempt to conform with the requirements of a Service Provider (SP). The SP declares its requirements in the NameIDPolicy element of a SAML Authentication Request. In Elasticsearch, the nameid_format SAML realm setting controls the NameIDPolicy value. Previously, the default value for nameid_format was urn:oasis:names:tc:SAML:2.0:nameid-format:transient. This setting created authentication requests that required the IdP to release NameID with a transient format. The default value has been removed, which means that Elasticsearch will create SAML Authentication Requests by default that don’t put this requirement on the IdP. If you want to retain the previous behavior, set nameid_format to urn:oasis:names:tc:SAML:2.0:nameid-format:transient.

Impact
If you currently don’t configure nameid_format explicitly, it’s possible that your IdP will reject authentication requests from Elasticsearch because the requests do not specify a NameID format (and your IdP is configured to expect one). This mismatch can result in a broken SAML configuration. If you’re unsure whether your IdP is explicitly configured to use a certain NameID format and you want to retain current behavior , try setting nameid_format to urn:oasis:names:tc:SAML:2.0:nameid-format:transient explicitly.

The xpack.security.transport.ssl.enabled setting will be required to configure xpack.security.transport.ssl settings.

Details
Configuring any SSL settings for xpack.security.transport.ssl without also configuring xpack.security.transport.ssl.enabled generates warnings in the deprecation log. In 8.0, this configuration will result in errors.

Impact
To avoid deprecation warnings, either:

  • Explicitly set xpack.security.transport.ssl.enabled as false
  • Discontinue use of other xpack.security.transport.ssl settings

If you want to enable SSL, follow the instructions to encrypt internode communications with TLS. As part of this configuration, explicitly set xpack.security.transport.ssl.enabled as true.

For example:

xpack.security.transport.ssl.enabled: true 
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

or false.

The xpack.security.http.ssl.enabled setting will be required to configure xpack.security.http.ssl settings.

Details
Configuring any SSL settings for xpack.security.http.ssl without also configuring xpack.security.http.ssl.enabled generates warnings in the deprecation log. In 8.0, this configuration will result in errors.

Impact
To avoid deprecation warnings, either:

  • Explicitly set xpack.security.http.ssl.enabled as false
  • Discontinue use of other xpack.security.http.ssl settings

If you want to enable SSL, follow the instructions to encrypt HTTP client communications for Elasticsearch. As part of this configuration, explicitly set xpack.security.http.ssl.enabled as true.

For example:

xpack.security.http.ssl.enabled: true 
xpack.security.http.ssl.certificate: elasticsearch.crt
xpack.security.http.ssl.key: elasticsearch.key
xpack.security.http.ssl.certificate_authorities: [ "corporate-ca.crt" ]

or false.

A xpack.security.transport.ssl certificate and key will be required to enable SSL for the transport interface.

Details
Enabling SSL for the transport interface without also configuring a certificate and key through use of the xpack.security.transport.ssl.keystore.path setting or the xpack.security.transport.ssl.certificate and xpack.security.transport.ssl.key settings generates warnings in the deprecation log. In 8.0, this configuration will result in errors.

Impact
If xpack.security.transport.ssl.enabled is set to true, provide a certificate and key using the xpack.security.transport.ssl.keystore.path setting or the xpack.security.transport.ssl.certificate and xpack.security.transport.ssl.key settings. If a certificate and key is not provided, Elasticsearch will generate warnings in the deprecation log.

A xpack.security.http.ssl certificate and key will be required to enable SSL for the HTTP layer.

Details
Enabling SSL for the HTTP layer without also configuring a certificate and key through use of the xpack.security.http.ssl.keystore.path setting or the xpack.security.http.ssl.certificate and xpack.security.http.ssl.key settings generates warnings in the deprecation log. In 8.0, this configuration will result in errors.

Impact
If xpack.security.http.ssl.enabled is set to true, provide a certificate and key using the xpack.security.http.ssl.keystore.path setting or the xpack.security.http.ssl.certificate and xpack.security.http.ssl.key settings. If a certificate and key is not provided, Elasticsearch will generate warnings in the deprecation log.

Index lifecycle management (ILM) changesedit

The ILM freeze action has been deprecated and will be a no-op in a future release.

Details
The ILM freeze action is now deprecated. This is because frozen indices provide no benefit given improvements in heap memory utilization. In 8.0 the freeze action will be a no-op and perform no action on the index, as the freeze API endpoint has been removed in 8.0.

Impact
Update your ILM policies to remove the freeze action from the cold phase.

Monitoring deprecationsedit

The use_ingest setting on Monitoring exporter configurations is deprecated.

Details
The xpack.monitoring.exporters.*.use_ingest property has been deprecated in 7.16.0 and will be removed in 8.0.0. This parameter controls the creation of pipelines for monitoring indices. These pipelines currently have no function.

Impact
Discontinue the use of the xpack.monitoring.exporters.*.use_ingest setting as it will no longer be recognized in the next major release.

The index.pipeline.master_timeout setting on Monitoring HTTP exporter configurations is deprecated.

Details
The xpack.monitoring.exporters.*.index.pipeline.master_timeout property has been deprecated in 7.16.0. This parameter sets the timeout when waiting for the remote Monitoring cluster to create pipelines. These pipelines for monitoring indices currently have no function and will be removed in 8.0.0.

Impact
Discontinue the use of the xpack.monitoring.exporters.*.index.pipeline.master_timeout setting as it will no longer be recognized in the next major release.

The index.template.create_legacy_templates setting on Monitoring HTTP exporter configurations is deprecated.

Details
The xpack.monitoring.exporters.*.index.template.create_legacy_templates property has been deprecated in 7.16.0. This parameter instructs the exporter to install the previous version of monitoring templates on the monitoring cluster. These older templates were meant to assist in transitioning to the current monitoring data format. They are currently empty and are no longer of any use.

Impact
Discontinue the use of the xpack.monitoring.exporters.*.index.template.create_legacy_templates setting as it will no longer be recognized in the next major release.

REST API deprecationsedit

The estimated_heap_memory_usage_bytes property in the create trained models API is deprecated

Details
The estimated_heap_memory_usage_bytes property in the create trained models API is deprecated in 7.16.

Impact
Use model_size_bytes instead.

Settings deprecationsedit

We no longer recommend using transient cluster settings.

Details
We no longer recommend using transient cluster settings. Use persistent cluster settings instead. If a cluster becomes unstable, transient settings can clear unexpectedly, resulting in an undesired cluster configuration.

Impact
Transient cluster settings are not yet deprecated, but we plan to deprecate them in a future release. For migration steps, see the Transient settings migration guide.

Indices deprecationsedit

Direct access to system indices is deprecated.

Details
Directly accessing system indices is deprecated. In Elasticsearch 8.0, you must add the allow_restricted_indices permission set to true on a user role to access system indices. Refer to indices privileges for information on adding this permission to an index privilege.

Impact
Accessing system indices directly results in warnings in the header of API responses and in the deprecation logs. Use Kibana or the associated feature’s Elasticsearch APIs to manage the data that you want to access. While it’s still possible to access system indices in Elasticsearch 7.16, they are reserved only for internal use by Elastic products and should not be accessed directly.

Cluster deprecationsedit

The script context cache is deprecated.

Details
Setting script.max_compilations_rate to use-context and configuring context-specific caches is deprecated.

Context-specific caches are no longer needed to prevent system scripts from triggering rate limits.

Configure the general cache to control the max compilation rate, cache size, and cache expiration for your scripts.

Impact
Remove script.max_compilations_rate: use-context and the context-specific cache settings: script.context.$CONTEXT.cache_max_size, script.context.$CONTEXT.max_compilations_rate, script.context.$CONTEXT.cache_expire.

The general cache defaults to a max size of 3000 with a rate limit of 150/5m, which allows 150 compilations per 5 minute period. By default, the entries in the cache do not expire.

To override the defaults, configure the script.cache.max_size, script.max_compilations_rate, and script.cache.expire settings.