Today, we are happy to announce the bugfix release of Elasticsearch 1.3.3, based on Lucene 4.9.1. You can download it and read the full changes list here: Elasticsearch 1.3.3.
Elasticsearch 1.3.3 is the latest stable release and we recommend that all users upgrade. Some of the highlights of this release include:
In version 1.2.0, we started using “paged bytes” internally to better manage memory. In other words, instead of allocating monolithic buffers which were only used once and then had to be garbage collected, we now use buffers of a fixed size, which can be reused and don’t need to be garbage collected.
However, the networking layer was not aware of this optimization and these buffers ended up being copied to another temporary buffer in the heap, and then copied to the direct memory allocated by the JVM networking layer as well. Besides the inefficiency of all this copying, large network requests could result in excessive use of direct memory as well — memory which was never reclaimed.
This release includes a change (#7811) to allow the networking layer to use the paged bytes directly. It should reduce direct memory usage, reduce garbage collection, reduce CPU usage, and improve networking performance.
When recovering a replica shard from the primary shard, Elasticsearch checks whether a file on the primary and the replica is identical by comparing their checksums and lengths. However, hash collisions can result in different files having the same checksum, which can lead to shard corruption. Instead, we haved moved the identity comparison to a per-commit / per-segment level where files are only treated as identical if all the other files in the commit / segment are the same.
Due to a bug in Lucene 4.9.0, upgrading to Elasticsearch version 1.3.2 could corrupt any index containing segments written by versions of Elasticsearch before 0.90. This bug has been fixed in Lucene 4.9.1. See #7430.