This week in Elasticsearch and Apache Lucene: Lucene 5.1.0 RC2 vote has passed
Welcome to This Week in Elasticsearch and Apache Lucene! With this weekly series, we're bringing you an update on all things Elasticsearch and Apache Lucene at Elastic, including the latest on commits, releases and other learning resources.
Found by Elastic users, we have made #Elasticsearch version 1.5.1 available for provisioning.
https://t.co/BtsPBSYQeT
— Found by Elastic (@foundsays) April 13, 2015
Elasticsearch Core
- Scripting: deprecate groovy sandbox and related settings (#10478, 1.6.0)
- Cluster state REST api: print routing_nodes out only when requested through specific flag (#10486, 2.0.0)
- Engine: sync translog before closing engine (#10484, 2.0.0, 1.6.0, 1.5.1)
- Logging: Use static logger name in
Engine.java
(#10497, 2.0.0, 1.6.0, 1.5.1) - Rivers: Deprecate rivers (#10503, 2.0.0, 1.6.0)
- Add missing
hashCode
method to <code>RecoveryState#File (#10501, 2.0.0, 1.6.0, 1.5.2) - Aggregations: Removed aggregations from
ReduceContext
(#10509, 2.0.0) - Only flush for checkindex if we have uncommitted changes (#10505, 2.0.0, 1.6.0, 1.5.2)
- Add
INFO
logging saying whether each path.data is on an SSD (#10502, 2.0.0, 1.6.0) - Don't fsync so often in tests (#10516, 2.0.0)
- Tests: Add multi data path testing to static bwc tests (#10519, 2.0.0, 1.6.0, 1.5.2)
- Build: Eclipse fixes (#10495, 2.0.0, 1.6.0, 1.5.2)
- Build: Fix to pom.xml to allow eclipse maven integration using m2e (#10524, 2.0.0, 1.6.0, 1.5.2)
- Build: Add sigar binaries when running unittests (#10529, 2.0.0, 1.6.0, 1.5.2)
- Search: deprecate the limit filter (#10532, 2.0.0)
- Geo: Correct ShapeBuilder coordinate parser to ignore values in 3rd+ dimension (#10539, 1.6.0, 1.5.2, 1.4.5)
- Dependencies: Update forbiddenapis to version 1.8 (#10555, 2.0.0, 1.6.0, 1.5.2)
- Build: Use SuppressForbidden annotation instead of class level excludes (#10560, 2.0.0)
- Allow to recover into a folder containing a corrupted shard (#10558, 2.0.0, 1.6.0)
- Build: Restrict read permission to project.basedir (#10569, 2.0.0)
- Translog: Log sending translog operation batches to nodes (#10544, 2.0.0, 1.6.0)
- Tests: Add
--all
flag to create-bwc script to regenerate all indexes (#10449, 2.0.0, 1.6.0) - Internal: Make APIs work per-segment like Lucene's Collector. (#10389, 2.0.0)
- Dependencies: Upgrade to lucene-5.1.0-snapshot-1671894. (#10468, 2.0.0)
- Core: Lucene merges should run on the target shard during recovery (#10463, 1.6.0, 1.5.1, 1.4.5)
- Transport: shortcut local execution (#10350, 2.0.0)
- Indexed scripts/templates: return response body when script is not found (#10396, 2.0.0, 1.6.0)
- Scripting: allow plugins to define custom operations that they use scripts for (#10419, 2.0.0, 1.6.0)
- Scripting: remove deprecated methods from
ScriptService
(#10476, 2.0.0) - Build: Fix maven-resources-plugin warnings (#10453, 2.0.0, 1.6.0)
- Aggregations: Fix multi-level breadth-first aggregations (#10411, 2.0.0)
- Aggregations: Speed up include/exclude in terms aggregations with regexps. (#10418, 2.0.0)
- Mappings: Bring back
numeric_resolution
(#10420, 2.0.0, 1.6.0) - Queries: Remove fuzzy-like-this support (#10391, 2.0.0)
- Geo: Fix hole intersection at tangential coordinate (#10332, 2.0.0, 1.6.0, 1.5.2, 1.4.5)
- Recovery: Integrate translog recovery into Engine / InternalEngine (#10452, 2.0.0)
- Internal: reduce the size of the search thread pool. (#9165, 2.0.0, 1.6.0)
- Validate API: provide more verbose explanation (#10147, 2.0.0, 1.6.0)
Live from #pycon, @amontalenti shows how @parsely uses #elasticsearch & #python for analytics https://t.co/NtkQxR96L0 pic.twitter.com/rfyzrbnNc3
— elastic (@elastic) April 10, 2015
Apache Lucene
- Lucene 5.1.0 RC2 vote has passed.
- Upgrade Elasticsearch to latest Lucene 5.1.x snapshot.
- Speed up spatial search with two-phased support and only post-filtering hits for those indexed spatial terms that may fall outside the query shape.
- Simplify the internal iterator Lucene uses when resolving deleted terms to docids.
- You can run the JaCoCo code coverage tool on Lucene.
- It's simpler to index an arbitrary binary token.
- Lucene's tests show how many tests are completed and the total number of tests.
TermAutomatonQuery
's advance was buggy, but it supports automata with cycles.- Remove unused
reuse
argument</a> from Lucene's <code>Terms.iteratormethod. - Simplify
QueryBuilder.<wbr>createFieldQuery
. - This new randomized test looks for bugs in Lucene's exception handling.
MemoryIndex
gets some more love, fixing a bug in its field infos, 0-length payloads and seek-by-ord.- More love for span queries, adding two-phased support for
SpanPositionCheckQuery
, <code>SpanPositionRangeQuery,SpanPayloadCheckQuery
, <code>SpanNearPayloadCheckQuery andSpanFirstQuery
. <a href="https://issues.apache.org/jira/browse/LUCENE-6394" target="_blank"><code>SpanNotQueryis also in progress. - Defensively cut over
IndexWriter
's internal change count <a href="https://github.com/apache/lucene-solr/commit/39e82d43b15d50b0113be8ecd4b765a58876d8a6" target="_blank"><code>AtomicLong. LongBitSet
fails if you try to use more the 2 billion bits.- The new auto-prefix feature is slowly baking.
- The query-time join feature continues baking.
- The near-real-time suggester feature continues baking.
Are you wondering what exactly an Apache Lucene codec is? Read all about it here: https://t.co/eXtxtHjKmL
— elastic (@elastic) April 8, 2015
Watch This Space
Stay tuned to this blog, where we'll share more news on the whole ELK ecosystem including news, learning resources and cool use cases!