18 April 2014 Engineering

Java 1.7u55 is Safe for Use With Elasticsearch

By Michael McCandless

This post was co-authored with Robert Muir.

Apache Lucene's extensive randomized tests are so stressful that they also happily serve as strong test cases for the JVM itself.

Over time, Lucene's tests, running many times per day on multiple Jenkins installations with random JVM options, different operating systems and garbage collectors, have uncovered a number of exciting JVM bugs. There's a partial list on the Lucene Wiki.

For example, the very first Java 1.7 release had a serious bug affecting Lucene, which was fixed with the first update (1.7.0u1). Fortunately, Oracle's QA team has been working more closely with Lucene developers since then, helping to iterate on new issues, notifying us on new early-access Java releases, etc.

One recent bug, first encountered here, would cause SEGV or, worse, silent corruption in the index term vectors. Because of this bug, we've had to recommend Lucene users stick with Java 1.7u25, now almost a year old.

At Oracle, Vladimir Kozlov has worked very hard and managed to isolate and fix the problem, as of Java 1.8.0. It's so hard to imagine how he could track down this bug, when debugging tricky Lucene test failures is hard enough, so we asked him how does that, and he explained that he steps through megabytes of assembly instructions, correlates them back to the particular Java sources, thinks about which parts of the Hotspot compiler may have resulted in each part of the assembly.  This all sounds exceptionally challenging, and we are happy Vladimir is able to do it so well!

Finally, that fix has now been back-ported to Java's 1.7.x branch, and released as of 1.7u55. The Lucene developers have upgraded Lucene's continuous Jenkins builds to 1.7u55 and no tests are failing, so we are comfortable and can now recommend that 1.7u55 is safe for Elasticsearch and all other Lucene based applications.