This week in Elasticsearch - July 16, 2014
Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
- Lucene: Remove deprecated
PatternTokenizer(#6171, master and 1.3)
checkVersionMapRefreshshouldn't use <code>indexWriter.getConfig() (#6794, master and 1.3)
- Mapping: Control whether
MapperServicedocMapper iterator should contain <code>DEFAULT_MAPPING (#6793, master and 1.3)
- Store: Before removing shard physically from disk verify that another node in the cluster actually holds an active shard copy (#6692, master and 1.3)
- Settings: Improve
Settings#getlookup for camel case support (#6765, master and 1.3)
- Dependencies: Upgrade to jackson core 18.104.22.168 (#6789, master and 1.3)
- Windows: Modify command window title (#6752, master and 1.3)
- REST API: Disable JSONP by default (#6795, master and 1.3)
- Netty: Improve large bytes request handling by detecting content composite buffer (#6756, master and 1.3)
- Plugins: Define default logging configuration for specific needs (#6802, master and 1.3)
- Cluster state: Add local node to cluster state (#6811, master and 1.3)
- Store: Make use of Lucene build-in checksums (#5924, master and 1.3)
DateHistogramBuilderuses wrong data type for <code>pre_offset and
post_offset(#5586, master and 1.3)
- Allocation: Improve handling of failed primary replica handling (#6825, master and 1.3)
- Replication: Do not ignore
ConnectTransportExceptionfor shard replication operations (#6813, master and 1.3)
- More Like This: ensures selection of best terms is indeed
O(n)(#6657, master and 1.3)
- Gateway: Set a default of 5m to
recover_after_timewhen any to the <code>expected*Nodes is set (#6742, master and 1.3)
- Query DSL: Function score without function throws NPE (#6292, master, 1.3 and 1.2)
- Snapshot/Restore: Add ability to restore indices without their aliases (#6457, master and 1.3)
- Recovery: Increment Store refcount on
RecoveryTarget(#6844, master and 1.3)
- Transport: Prevent possible NPE during shutdown for requests using timeouts (#6849, master and 1.3)
- Aggregations: Add infrastructure for changing easily the significance heuristic in significant terms (#6561, master and 1.3)
- Analysis: Improve Hunspell error messages (#6850, master and 1.3)
- Indexed Scripts/Templates: Add support for storing/deleting/getting scripts/template from an index (#5484, master and 1.3)
- Suggestions: Add collate option to
PhraseSuggester(#3482, master and 1.3)
- Stats: Expose IndexWriter and VersionMap RAM usage to
ShardStatsand <code>_cat endpoint (#6483, master and 1.3)
- Search: Wrap filter only once in
ApplyAcceptedDocsFilter(#6873, master and 1.3)
- Core: don't close/reopen IndexWriter when changing RAM buffer size (#6856, master and 1.3)
- Internal: make transport action name available in
TransportActionbase class (#6860, master and 1.3)
- Update API: Handle document missing exception on retry correctly (#6355, master and 1.3)
- Scripting: Add script engine for lucene expressions (#6818, master and 1.3)
- Mapping API: Add transform to document before index operation(#6566, master and 1.3)
- Internal: Make
XContentBuilder<code>Releasable (#6869, master and 1.3)
- Analysis: Share numeric data analyzer instances between mappings (#6843, master and 1.3)
- Store: Only send shard exists requests if shards exist locally on disk and are not allocated on that node according to the cluster state (#6870, master and 1.3)
- Infra: re-send failed shard messages when receiving a cluster state that still refers to them
(#6881, master and 1.3)
- Recovery: Do not start a gateway recovery if source node is not found(#6879, master and 1.3)
Igor Motov presents at last week's Elasticsearch Korea Study Session
Photo credit: JongMin Kim
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
- Elasticsearch.Net and NEST are now available as RC1. Check out the latest bits and get your feedback to us before we head into GA!
- We're streamlining maintenance of our Puppet modules. Check out the full details in this blog post.
- Itamar Syn-Hershko has released version 1.4 of his Hebrew analyzer plugin for Elasticsearch.
- Chris Allen has released Bloodhound, a Haskell client for Elasticsearch.
- Fan of Docker? Love CoreOS? Matt Wright shared an incredibly detailed how to on Running an Elasticsearch Cluster on CoreOS.
- And for a little more Docker love, check out this excellent tutorial from Graham Jenson: Vagrant with Docker: How to set up Postgres, Elasticsearch and Redis on Mac OS X.
- Florian Hopf has written the next installment in his article series, Use Cases for Elasticsearch. Latest edition: full text search.
- Matthias Ilgen shared a how to on using Elasticsearch and Excel to analyze log data (en français).
- The team at Orange, a leading provider or telecommunications services, authored an article on why they chose Elasticsearch. (en français)
- Gennaro Varriale published a detailed how to on Integrating Elasticsearch with MongoDB (in Italiano).
- JongMin Kim, organizer of the Elasticsearch Korea User Group, shared his impressions from the last Study Session. (한국어)
amazing turnout to the elasticsearch at Tokyo #elasticsearchjp pic.twitter.com/Aa88eVf5dF
— Shay Banon (@kimchy) July 14, 2014
Slides & Videos
LivingSocial shares their use case
Alexander Mols' slides from the recent Dutch PHP Conference
Learn how Yieldbot uses Kafka together with new Elasticsearch features like doc values
David Pilato at Devoxx UK 2014
- Loved David's slides, but longing for video? Look no further! Registration is required, but once you're logged in you can hear all about Advanced Search for Your Legacy Application.
Where to find Us
We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Community Manager, Leslie Hawthorn, know!
- Our brand new employee in Germany, Dmitri Marx, will be attending Java Forum Stuttgart along with Livia Froelicher. Make sure to say hello to them between sessions! The conference takes place on July 17th at the Culture & Convention Centre Liederhalle.
- Honza Kral will present at the upcoming Write the Docs unconference in Berlin on July 19th. After Honza's talk and the day's sessions, please enjoy refreshments on Elasticsearch at the post-conference meet up!
- EuroPython comes to Berlin this year, with the conference running from July 21-27th at the Berlin Congress Center. Please stop by the Elasticsearch table to learn all about how the ELK stack can make life better for Pythonistas. Make sure to catch Honza Kral, Django and Elasticsearch core developer, on lessons learned from building the Python Elasticsearch client. Honza will take the stage at 2:00 PM on July 24th. Bonus: we're hosting a meetup during the conference on Thursday, July 28th from 6-8 PM. Mark your calendar now, and keep an eye out for registration details soon!
- Following on from EuroPython, Honza Kral will be speaking at the PyData Conference on Making Sense of your big data with Elasticsearch. Honza presents on July 27th at 2:10 PM, and the PyData Conference will also be held at the Berlin Congress Center.
- The Second Elasticsearch Tel Aviv Meetup will convene on July 28th, focusing on real world use cases. The agenda is now finalized and you can hear from three different companies on how they use Elasticsearch. You can register now.
- The OpsTalk Meetup Group will host a workshop in Tel Aviv on August 4th from 6-9 PM. The organizers are looking to teach you all about the wonders of log analysis using the ELK stack. Register now to save your place.
The very first Elasticsearch Italy Meetup has been scheduled for TONIGHT July 16th! Please join us to hear from Luca Cavanna on What's New in Elasticsearch. You'll also be treated to a case study talk on how Lutech uses Elasticsearch in its Threat Management System for Breach Detection, Intelligence & Response. Doors open at 6:00 PM, and registration is now open.
首都大で7/17午後 @johtani さんと @PENGUINANA_ さんにelasticsearch入門とクックパッドにおけるサービス開発についてトークしていただきます。参加費無料、申し込み不要ですので、お気軽にどうぞ！ pic.twitter.com/LJVtmjoLFf
— Mamoru Komachi (@mamoruk) July 10, 2014
The Auckland JVM Users Group will rebooting their meetup series, with their newest offering focused on Elasticsearch. You can join them on Tuesday, July 22nd at 6:00 PM, and registration is open.
The Warsaw Java Users Group is holding a Microservices Hackathon on August 9th from 10 AM to 10 PM. You can spend those twelve hours hacking on the ELK stack and a variety of other tools. Register now to attend.
The London VoiP User Group will get together on July 22nd at 6:30 PM. These folks are in search of a venue, so if you're excited to hear about Eye-candy from CDRs with the ELK stack, register now and help these good folks find a place to meetup.
- Join the fun tonight:The TechLife Cincinnati Meetup Group will get together on July 16th at 6 PM to talk the ELK stack and other open source monitoring tools. The hosts are welcoming the Cincinnati Blacks in Technology group to present during this meetup.
- The annual O'Reilly Open Source Conference is back again in Portland, Oregon! The Elasticsearch team will be out in force, with two talks, an ELK BoF and a lovely table in the exhibit hall where we can answer all of your questions. Look out for more details in next Monday's edition of Where in the World is Elasticsearch. We hope to see many of our friends there! OSCON runs from July 20-24th at the Oregon Convention Center.
- Living in Portland but not planning to attend OSCON? No worries! We have an Elasticsearch Meetup co-located with our ELK BoF on Wednesday, July 23rd. Register now to join us for the festivities, including real-world use case story from Build Energy!
Where to Find You
Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- New York - August 12, 2014 (core Elasticsearch training)
- Sao Paulo - August 14, 2014 (core Elasticsearch training)
- Rio de Janeiro - August 18, 2014 (core Elasticsearch training)
- Amsterdam - August 20, 2014 (core Elasticsearch training)
- Boston - August 27, 2014 (core Elasticsearch training)
- San Francisco - August 28, 2014 (ELK workshop)