Breaking changes in 8.0edit

This section discusses the changes that you need to be aware of when migrating your application to Elasticsearch 8.0.

See also Release highlights and Release notes.

Note

Coming in 8.0.0.

Indices created before 7.0edit

Elasticsearch 8.0 can read indices created in version 7.0 or above. An Elasticsearch 8.0 node will not start in the presence of indices created in a version of Elasticsearch before 7.0.

Important

Reindex indices from Elasticsearch 6.x or before

Indices created in Elasticsearch 6.x or before will need to be reindexed with Elasticsearch 7.x in order to be readable by Elasticsearch 8.x.

Analysis changesedit

The nGram and edgeNGram token filter names have been removededit

The nGram and edgeNGram token filter names that have been deprecated since version 6.4 have been removed. Both token filters can only be used by their alternative names ngram and edge_ngram since version 7.0.

Discovery changesedit

Removal of old discovery settingsedit

All settings under the discovery.zen namespace, which existed only for BWC reasons in 7.x, will no longer be supported. In particular, this includes:

  • discovery.zen.no_master_block
  • discovery.zen.hosts_provider
  • discovery.zen.publish_timeout
  • discovery.zen.commit_timeout
  • discovery.zen.publish_diff.enable
  • discovery.zen.ping.unicast.concurrent_connects
  • discovery.zen.ping.unicast.hosts.resolve_timeout
  • discovery.zen.ping.unicast.hosts
  • discovery.zen.unsafe_rolling_upgrades_enabled
  • discovery.zen.commit_timeout
  • discovery.zen.fd.connect_on_network_disconnect
  • discovery.zen.fd.ping_interval
  • discovery.zen.fd.ping_timeout
  • discovery.zen.fd.ping_retries
  • discovery.zen.fd.register_connection_listener
  • discovery.zen.join_retry_attempts
  • discovery.zen.join_retry_delay
  • discovery.zen.max_pings_from_another_master
  • discovery.zen.send_leave_request
  • discovery.zen.master_election.wait_for_joins_timeout
  • discovery.zen.master_election.ignore_non_master_pings
  • discovery.zen.publish.max_pending_cluster_states

Mapping changesedit

Limiting the number of completion contextsedit

The number of completion contexts within a single completion field has been limited to 10.

Defining multi-fields within multi-fieldsedit

Previously, it was possible to define a multi-field within a multi-field. Defining chained multi-fields was deprecated in 7.3 and is now no longer supported. To migrate the mappings, all instances of fields that occur within a fields block should be removed, either by flattening the chained fields blocks into a single level, or by switching to copy_to if appropriate.

Packaging changesedit

Java 11 is requirededit

Java 11 or higher is now required to run Elasticsearch and any of its command line tools.

Rollup changesedit

StartRollupJob endpoint returns success if job already startededit

Previously, attempting to start an already-started rollup job would result in a 500 InternalServerError: Cannot start task for Rollup Job [job] because state was [STARTED] exception.

Now, attempting to start a job that is already started will just return a successful 200 OK: started response.

Snapshot and Restore changesedit

Get snapshots response format is changededit

It’s possible to get snapshots from multiple repositories in one go. The response format has changed and now contains separate response for each repository.

For example, requesting one snapshot from particular repository

GET _snapshot/repo1/snap1

produces the following response

{
    "responses": [
        {
            "repository": "repo1",
            "snapshots": [
                {
                    "snapshot": "snap1",
                    "uuid": "cEzdqUKxQ5G6MyrJAcYwmA",
                    "version_id": 8000099,
                    "version": "8.0.0",
                    "indices": [],
                    "include_global_state": true,
                    "state": "SUCCESS",
                    "start_time": "2019-05-10T17:01:57.868Z",
                    "start_time_in_millis": 1557507717868,
                    "end_time": "2019-05-10T17:01:57.909Z",
                    "end_time_in_millis": 1557507717909,
                    "duration_in_millis": 41,
                    "failures": [],
                    "shards": {
                        "total": 0,
                        "failed": 0,
                        "successful": 0
                    }
                }
            ]
        }
    ]
}

See Snapshot And Restore for more information.

Deprecated node level compress setting removededit

For shared file system repositories ("type": "fs"), the node level setting repositories.fs.compress could previously be used to enable compression for all shared file system repositories where compress was not specified. The repositories.fs.compress setting has been removed.

Instead use the repository specific compress setting to enable compression. See Snapshot And Restore for information on the compress setting.

Compression of meta data files is now on by defaultedit

Previously, the default value for compress was false. The default has been changed to true.

This change will affect both newly created repositories and existing repositories where compress=false has not been explicitly specified.

For more information on the compress option, see Snapshot And Restore

Security changesedit

The accept_default_password setting has been removededit

The xpack.security.authc.accept_default_password setting has not had any affect since the 6.0 release of Elasticsearch. It has been removed and cannot be used.

The roles.index.cache.* settings have been removededit

The xpack.security.authz.store.roles.index.cache.max_size and xpack.security.authz.store.roles.index.cache.ttl settings have been removed. These settings have been redundant and deprecated since the 5.2 release of Elasticsearch.

The elasticsearch-migrate tool has been removededit

The elasticsearch-migrate tool provided a way to convert file realm users and roles into the native realm. It has been deprecated since 7.2.0. Users and roles should now be created in the native realm directly.

Index Lifecycle Management changesedit

indices.lifecycle.poll_interval must be greater than 1 secondedit

The setting indices.lifecycle.poll_interval, if set too low, can cause excessive load on a cluster. This setting must now be set to 1 second or higher.

Java API changesedit

Changes to Fuzzinessedit

To create Fuzziness instances, use the fromString and fromEdits method instead of the build method that used to accept both Strings and numeric values. Several fuzziness setters on query builders (e.g. MatchQueryBuilder#fuzziness) now accept only a `Fuzziness`instance instead of an Object. You should preferably use the available constants (e.g. Fuzziness.ONE, Fuzziness.AUTO) or build your own instance using the above mentioned factory methods.

Fuzziness used to be lenient when it comes to parsing arbitrary numeric values while silently truncating them to one of the three allowed edit distances 0, 1 or 2. This leniency is now removed and the class will throw errors when trying to construct an instance with another value (e.g. floats like 1.3 used to get accepted but truncated to 1). You should use one of the allowed values.

Changes to Repositoryedit

Repository has no dependency on IndexShard anymore. The contract of restoreShard and snapshotShard has been reduced to Store and MappingService in order to improve testability.

Network changesedit

Removal of old network settingsedit

The network.tcp.connect_timeout setting was deprecated in 7.x and has been removed in 8.0. This setting was a fallback setting for transport.connect_timeout. To change the default connection timeout for client connections transport.connect_timeout should be modified.

Node changesedit

Removal of node.max_local_storage_nodes settingedit

The node.max_local_storage_nodes setting was deprecated in 7.x and has been removed in 8.0. Nodes should be run on separate data paths to ensure that each node is consistently assigned to the same data path.

Change of data folder layoutedit

Each node’s data is now stored directly in the data directory set by the path.data setting, rather than in ${path.data}/nodes/0, because the removal of the node.max_local_storage_nodes setting means that nodes may no longer share a data path. At startup, Elasticsearch will automatically migrate the data path to the new layout. This automatic migration will not proceed if the data path contains data for more than one node. You should move to a configuration in which each node has its own data path before upgrading.

If you try to upgrade a configuration in which there is data for more than one node in a data path then the automatic migration will fail and Elasticsearch will refuse to start. To resolve this you will need to perform the migration manually. The data for the extra nodes are stored in folders named ${path.data}/nodes/1, ${path.data}/nodes/2 and so on, and you should move each of these folders to an appropriate location and then configure the corresponding node to use this location for its data path. If your nodes each have more than one data path in their path.data settings then you should move all the corresponding subfolders in parallel. Each node uses the same subfolder (e.g. nodes/2) across all its data paths.

Transport changesedit

Removal of old transport settingsedit

The following settings have been deprecated in 7.x and removed in 8.0. Each setting has a replacement setting that was introduced in 6.7.

  • transport.tcp.port replaced by transport.port
  • transport.tcp.compress replaced by transport.compress
  • transport.tcp.connect_timeout replaced by transport.connect_timeout
  • transport.tcp_no_delay replaced by transport.tcp.no_delay
  • transport.profiles.profile_name.tcp_no_delay replaced by transport.profiles.profile_name.tcp.no_delay
  • transport.profiles.profile_name.tcp_keep_alive replaced by transport.profiles.profile_name.tcp.keep_alive
  • transport.profiles.profile_name.reuse_address replaced by transport.profiles.profile_name.tcp.reuse_address
  • transport.profiles.profile_name.send_buffer_size replaced by transport.profiles.profile_name.tcp.send_buffer_size
  • transport.profiles.profile_name.receive_buffer_size replaced by transport.profiles.profile_name.tcp.receive_buffer_size

HTTP changesedit

Removal of old HTTP settingsedit

The http.tcp_no_delay setting was deprecated in 7.x and has been removed in 8.0. It has been replaced by http.tcp.no_delay.

Reindex changesedit

Reindex from remote would previously allow URL encoded index-names and not re-encode them when generating the search request for the remote host. This leniency has been removed such that all index-names are correctly encoded when reindex generates remote search requests.

Instead, please specify the index-name without any encoding.

Removal of typesedit

The /{index}/{type}/_delete_by_query and /{index}/{type}/_update_by_query REST endpoints have been removed in favour of /{index}/_delete_by_query and /{index}/_update_by_query, since indexes no longer contain types, these typed endpoints are obsolete.

Removal of size parameteredit

Previously, a _reindex request had two different size specifications in the body:

  • Outer level, determining the maximum number of documents to process
  • Inside the source element, determining the scroll/batch size.

The outer level size parameter has now been renamed to max_docs to avoid confusion and clarify its semantics.

Similarly, the size parameter has been renamed to max_docs for _delete_by_query and _update_by_query to keep the 3 interfaces consistent.

Search Changesedit

Removal of typesedit

The /{index}/{type}/_search, /{index}/{type}/_msearch, /{index}/{type}/_search/template and /{index}/{type}/_msearch/template REST endpoints have been removed in favour of /{index}/_search, /{index}/_msearch, /{index}/_search/template and /{index}/_msearch/template, since indexes no longer contain types, these typed endpoints are obsolete..

The /{index}/{type}/_termvectors, /{index}/{type}/{id}/_termvectors and /{index}/{type}/_mtermvectors REST endpoints have been removed in favour of /{index}/_termvectors, /{index}/{id}/_termvectors and /{index}/_mtermvectors, since indexes no longer contain types, these typed endpoints are obsolete..

Removal of queriesedit

The common query was deprecated in 7.x and has been removed in 8.0. The same functionality can be achieved by the match query if the total number of hits is not tracked.

Removal of query parametersedit

The cutoff_frequency parameter was deprecated in 7.x and has been removed in 8.0 from match and multi_match queries. The same functionality can be achieved without any configuration provided that the total number of hits is not tracked.

Settings changesedit

The search.remote settings have been removededit

In 6.5 these settings were deprecated in favor of cluster.remote. In 7.x we provided automatic upgrading of these settings to their cluster.remote counterparts. In 8.0.0, these settings have been removed. Elasticsearch will refuse to start if you have these settings in your configuration or cluster state.