This week in Elasticsearch and Apache Lucene: Deprecating Rivers feature for Elasticsearch
Welcome to This Week in Elasticsearch and Apache Lucene! With this new weekly series, we're bringing you an update on all things Elasticsearch and Apache Lucene at Elastic, including the latest on commits, releases and other learning resources.
Top News
We'll be deprecating our Rivers feature for Elasticsearch. You can read the full details, including recommendations for replacing your own Rivers plugins, in Shay Banon's blog post.
Attending
#dotnetfringe ? Come join me and @gregmarzouka to for an #elasticsearch workshop and rebuild #nuget search! http://t.co/OumsQNLMAb
— Martijn Laarman (@Mpdreamz)
March 24, 2015
Elasticsearch Core
- cat API: add node id to _cat apis (#10246, 2.0.0, 1.6.0)
- Mappings: Fix
_field_names
to be disabled on pre 1.3.0 indexes (#10268, 2.0.0, 1.6.0, 1.5.1) - Tests: add delete-by-query into translog in
OldIndexBackwardsCompatibilityTests
(#10266, 2.0.0, 1.6.0, 1.5.1) - Tests: Enable inline scripts on demand in bw comp tests (#10282, 2.0.0, 1.6.0)
- IndexService: Fail shard when index service/mappings fails to instantiate (#10283, 2.0.0, 1.6.0, 1.5.1)
- Aggregations: Fix 2 bugs in
children
agg (#10263, 2.0.0, 1.6.0, 1.5.1) - MultiPercolate: Fix wrong use of
currentFieldName
outside of a parsing loop (#10307, 2.0.0, 1.6.0, 1.5.1) - Delete-by-Query: Remove current delete-by-query implementation (#10288, 2.0.0)
- Internal: Don't try to send a mapping refresh if there is no master (#10311, 2.0.0, 1.6.0, 1.5.1)
- Recovery:
RecoveryState.File.toXContent
reports file length as recovered bytes (#10310, 2.0.0, 1.6.0, 1.5.1) - Core: Refresh if many deletes in a row use up too much version map RAM (#10312, 2.0.0, 1.6.0, 1.5.1)
- Testing: Move
FakeRestRequest
to <code>org.elasticsearch.test.rest (#10323, 2.0.0, 1.6.0) - Netty: Add getter for channel in NettyTransportChannel (#10319, 2.0.0, 1.6.0)
- Build: Add ability to launch bootstrapped ES using maven (#10287, 2.0.0, 1.6.0)
- Snapshot/Restore: Automatically add "index." prefix to the settings are changed on restore... (#10269, 2.0.0, 1.6.0, 1.5.1)
- Scripting: Remove support for
script.disable_dynamic
setting (#10286, 2.0.0) - ClusterState: Refactor state format to use incremental state IDs (#10316, 2.0.0, 1.6.0, 1.5.1)
- Function score: apply
min_score
to sub query score if no function provided (#10326, 2.0.0, 1.6.0, 1.5.1) - Aggregations: Fix bug for
inner_hits
agg where parse error is thrown if a inner filter is used in a nested filter/query (#10309, 2.0.0, 1.6.0, 1.5.1) - Geo: Fix
validate_*
merge policy for <code>GeoPointFieldMapper (#10165, 2.0.0, 1.5.1) - function_score: undo "Remove explanation of query score from functions" (#9826, 2.0.0, 1.6.0, 1.5.1)
- Delete-by-Query: Deprecate delete-by-query in client/transport/action APIs too (#10239, 1.6.0, 1.5.1)
- Mappings: Remove delete mapping API (#10231, 2.0.0)
- Scripting: add support for fine-grained settings (#10116, 2.0.0, 1.6.0)
- Mapping: Update dynamic fields in mapping on master even if parsing fails for the whole document (#9874, 2.0.0, 1.6.0)
- Core: Enable doc values by default, when appropriate (#10209, 2.0.0)
- Hunspell: Fix thread leak in Hunspell service tests (#10020, 2.0.0, 1.6.0, 1.5.1)
- Aggregations: Fix nested stored field support in
inner_hits
(#10235, 2.0.0, 1.6.0, 1.5.1) - Parent/Child: The parent option on update request should be used for upsert only. (#9612, 2.0.0, 1.6.0)
- Cleanup: Fix errors reported by
error-prone
(#9817, 2.0.0, 1.6.0) - Tests: Assert that we do not leak SearchContexts (#9595, 2.0.0)
- Search: Merge
search_type=count
and <code>size=0. (#9296, 2.0.0) - Store: shard not deleted after relocation if relocated shard is still in post recovery (#10172, 2.0.0, 1.6.0)
Elasticsearch Lyon meetup. 1pb/s of data collected by large hadron collided. One does not simply store 1pb/s.
pic.twitter.com/7yXid4jj60
— Pascal Cans ッ (@pcans)
March 24, 2015
Elasticsearch Plugin Releases
We had several plugin release in this past week. You can find more information about the updates in
- Language plugins: JavaScript plugin v 2.5.0, MVEL plugin v 1.5.0, and Python v 2.5.0.
- River plugins: CouchDB v 2.5.0, RabbitMQ v 2.5.0, Twitter v 2.5.0, Wikipedia v 2.5.0
- Transport plugins: Memcached transport v 2.5.0, Servlet transport v 2.5.0 and Thrift transport v 2.5.0.
And let's not forget the Mapper Attachment plugin, also released as version 2.5.0.
You can find full details in the release notes for each plugin as sent to the Elasticsearch User mailing list.
Apache Lucene
- 5.1.0 release may be coming soon.
- PrefixQuery now extends AutomatonQuery and PrefixTermsEnum is removed. This is in preparation for auto-prefix terms.
- Simplify TwoPhaseIterator by passing the approximation to the constructor.
- FST.save could write a corrupted FST because it didn't flush/close the BufferedOutputStream first.
- Lucene continuously tests Java early access builds, and uncovered a new Java 1.9 bug which we've worked around. Elasticsearch's sources were also affected.
- A new near-real-time, document based suggester is coming to Lucene, factored out of Elasticsearch's completion suggester.
- Span queries will soon support approximations so they are more efficient handling conjunctions of multiple span queries.
- More efficient query-time joins based on global ordinals is coming, derived from Elasticsearch's join implementations.
#elasticon sessions are out! My talk on how we use #elasticsearch at Microsoft,including some #AzureSearch details: https://t.co/4QDodRA7fs
— Pablo Castro (@pmc)
March 31, 2015
Watch This Space
Stay tuned to this blog, where we'll share more news on the whole ELK ecosystem including news, learning resources and cool use cases!