This week in Elasticsearch and Apache Lucene: 5.1.0 is released!

Welcome to This Week in Elasticsearch and Apache Lucene! With this weekly series, we're bringing you an update on all things Elasticsearch and Apache Lucene at Elastic, including the latest on commits, releases and other learning resources.

Top News

The real deal on #elasticsearch disk capacity requirements from @acdahlqvist + @peterkimnyc: https://t.co/ol7ZaZX44o. pic.twitter.com/Tmr3xHRRTw
— elastic (@elastic) April 17, 2015

Elasticsearch Core

• Internal: Cleanup JVM info and stats (#10553, 2.0.0)
• Internal: Improve exception handling in transport local execution (#10554, 2.0.0)
• Build: Restrict read permission to project.basedir/target if security manager is used (#10572, 2.0.0)
• Core: Rename START phase into <code>VERIFY_INDEX (#10570, 2.0.0)
• Core: Move back to single EngineConfig (#10586, 2.0.0)
• Cleanup: Remove unused normsField from <code>MatchAllQuery (#10592, 2.0.0)
• Translog: Remove useless random translog directory selection (#10589, 2.0.0, 1.6.0)
• Aggregations: Fix _as_string output to only show when format specified (#10571, 2.0.0, 1.6.0, 1.5.2)
• Tests: Forbid tests from writing to CWD (#10605, 2.0.0)
• Tests: Fix shadow engine tests (#10621, 2.0.0)
• Mappings: Same code path for dynamic mappings updates and updates coming from the API (#10593, 2.0.0)
• Build: Add tests progress indicator (#10635, 2.0.0)
• Dependencies: Upgrade to Lucene-5.2-snapshot-1674183. (#10641, 2.0.0)
• Core: Fix possible NPE in InternalClusterService$NotifyTimeout, the <code>future field is set from a different thread (#10630, 2.0.0, 1.6.0, 1.5.0)
• Aggregations: Added context for significant_terms scoring (#10633, 2.0.0)
• Test: Add back really old upgrade test (#10639, 2.0.0)
• Internal: Cleanup local code transport execution (#10582, 2.0.0)
• Testing: Integrate better with lucene test framework and mockfilesystems (#10656, 2.0.0)
• Search: Cut over to IndexSearcher.count() (#10674, 2.0.0)
• Packaging: Standardization of packages structure and install (#10595, 2.0.0, 1.6.0)
• Mappings: Remove dead code after previous refactorings (#10666, 2.0.0)
• Snapshot/Restore: Improve the error message when attempting to snapshot a closed index (#10608, 2.0.0, 1.6.0)
• Internal: Ensure that explanation descriptions are not null on serialization (#10689, 2.0.0, 1.6.0)
• Core: Log sending translog operation batches to nodes (#10544, 2.0.0, 1.6.0)
• Analysis: fix ignoring tokenizer settings in SynonymTokenFilterFactory (#10489, 2.0.0, 1.6.0, 1.5.2)
• Log only a summary line of filesystem detail for all path.data on node startup (#10527, 2.0.0, 1.6.0)
• Term Vectors: terms filtering (#9561, 2.0.0)
• MLT Query: lenient default parameters (#9412, 2.0.0)
• MLT API: Fix misbehaving user parameters (#8028, 2.0.0)
• Upgrade API: Add upgrade_only_ancient_segments option to upgrade API (#10540, 2.0.0, 1.6.0)
• Startup: Export the hostname as environment variable (#9474, 2.0.0, 1.6.0)
• Mapping: Fixed an equality check in StringFieldMapper (#10359, 2.0.0, 1.6.0, 1.5.2)
• Store: Move to one data.path per shard (#10461, 2.0.0)

Apache Lucene

• 5.1.0 is released!
• FieldType's copy constructor was failing to copy numeric precision step
• Many test framework fixes: add SuppressReproduceLine</a>, make <a href="https://issues.apache.org/jira/browse/LUCENE-6431"><code>ExtraFS reproducible, add hashCode and equals to FilterPath</a>, fix <a href="https://issues.apache.org/jira/browse/LUCENE-6424">other bugs in mock filesystems</a>, check two-phased iterators in <a href="https://issues.apache.org/jira/browse/LUCENE-6419"><code>AssertingQuery, add a test annotation to suppress fsync
• SolrSynonymParser was failing to preserve the original token and set position length
• Queries and filters now check more carefully for invalid null arguments
• Move extractTerms</a> from <code>Query to Weight
• Remove the dangerous TermQuery(Term, int) constructor
• MultiPhraseQuery defers loading positions until they are needed
• IndexWriter and <code>CheckIndex enforce a maximum value for token position to prevent queries from silently not finding some results
• Improve testing of span queries
• SpanNotQuery gets two-phase iterators
• LongBitSet can now handle very large bit sets correctly
• BooleanQuery.extractTerms skips terms from non-scoring clauses
• TermsQuery.extractTerms should not throw an exception
• Upgrade to forbidden APIs v1.8
• StreamingQuadPrefixTree should improve indexing memory and disk used in certain cases
• Try to make IOUtils.spins a bit more robust
• More baking for auto-prefix

Watch This Space

Stay tuned to this blog, where we'll share more news on the whole ELK ecosystem including news, learning resources and cool use cases!