Elasticsearch 5.0.0-alpha5 released
Today we are excited to announce the release of Elasticsearch 5.0.0-alpha5 based on Lucene 6.1.0. This is the fifth in a series of pre-5.0.0 releases designed to let you test out your application with the features and changes coming in 5.0.0, and to give us feedback about any problems that you encounter.
Open a bug report today and become an Elastic Pioneer.
IMPORTANT: This is an alpha release and is intended for testing purposes only. Indices created in this version will not be compatible with Elasticsearch 5.0.0 GA.
DO NOT DEPLOY IN PRODUCTION
Development of Elasticsearch 5.0.0 is nearing completion. This release contains more bug fixes and enhancements than it does new features (all of which you can read about in the release notes linked above), but there are a few gems worth mentioning.
- Java REST client
- Rollover indexing
- Wait for refresh
- Ingest Node
- Painless Scripting
- Instant Aggregations
- Text/Keyword fields replacing String
- Completion Suggester v2
- Settings Validation
- Safety in Production
- Resiliency Improvements
- Percolate Query
- Deleted Index Tombstones
- Dots in field names
- Cluster allocation explain API
Known networking bug in 5.0.0-alpha5
We have discovered is a major bug in the new Netty4 implementation in this release which affects any REST requests greater than 1024 bytes in size, and which will generate an exception similar to the following:
[WARN ][http.netty4] [wtOV9Vb] caught exception while handling client http traffic, closing connection [id: 0x1320b717, L:/0:0:0:0:0:0:0:1:9200 - R:/0:0:0:0:0:0:0:1:54732] java.lang.UnsupportedOperationException: unsupported message type: DefaultFullHttpResponse (expected: ByteBuf, FileRegion)
This is due to incorrect handling of the
Expect HTTP header, and it can be
worked around in one of three ways:
Use a client which does not add
Expectheaders (including the official clients).
Pass a blank
curl -H 'Expect:' ...
Use Netty3 for the HTTP layer by passing the following setting at startup:
The Elasticsearch Migration Helper is a site plugin designed to help you to prepare for your migration from Elasticsearch 2.3.x to Elasticsearch 5.0. It comes with three tools:
- Cluster Checkup
- Runs a series of checks on your cluster, nodes, and indices and alerts you to any known problems that need to be rectified before upgrading.
- Reindex Helper
- Indices created before v2.0.0 need to be reindexed before they can be used in Elasticsearch 5.x. The reindex helper upgrades old indices at the click of a button.
- Deprecation Logging
- Elasticsearch comes with a deprecation logger which will log a message whenever deprecated functionality is used. This tool enables or disables deprecation logging on your cluster.
Instruction for install the Elasticsearch migration helper.
In previous versions of Elasticsearch, when you issue a create-index request you receive a response as soon as the index is added to the cluster state, even though the shards of the index have not yet been allocated. An index with unallocated primary shards turns the cluster health red, which rings an alarm bell for sysadmins. As of this release, Elasticsearch now waits for the primary shards to be allocated before responding. This means that your index is ready to use as soon as the create-index request returns. On top of that, even dynamically created indices will no longer turn the cluster red (however briefly) unless there is a real problem which means that shards cannot be allocated. As part of this change, the
write_consistency parameter has been renamed to
wait_for_active_shards, which more clearly expresses the behaviour of this parameter.
Netty is the library that we use for networking. Changes to the design in Netty 4.0 made it difficult for us to upgrade, but those issues have been resolved in Netty 4.1. Upgrading the networking layer is a big and potentially risky change, so in this version we ship with a Netty4 and a Netty3 module. Netty4 is used by default but, in case we discover any problems, you can switch back to Netty3 by updating the
http.type settings to
histogram aggregation now supports fractional buckets and handles correct rounding of negative buckets. On top of improvements added in alpha4 to reduce the number of buckets created by the
terms aggregation where possible, this release reduces the default number of buckets created per shard to make the
terms agg lighter. Of course, multi-level aggregations on high cardinality fields can still result in combinatorial explosion with trillions of buckets, so the
request circuit breaker has been extended to protect your cluster from this abuse.
Getting started with security has historically been hard, especially when wrangling openssl to generate self-signed certificates. X-Pack now ships with a command line tool which will generate certificates (and optionally a certificate authority) for you in the simplest way possible. We have also simplified document level security by allowing metadata associated with roles (eg
access_level) to be plugged into a DLS query.
scaled_floatfield datatype allows storing a floating point number as a
longvalue, which Lucene can compress more efficiently.
- Numeric fields, indexed as block KD trees, have received further improvements to reduce disk storage, and to speed up both indexing and search performance.
Custom tokenizers, token filters, and character filters can be specified inline in the
- An unrecoverable exception (such as out-of-memory) will now cause a node to die with dignity, instead of limping on in an undefined state.
- The Java REST client now supports blocking and asynchronous requests.
- Nodes now persist their IDs across restarts, making it easier to monitor and debug problems.