This Week in Elasticsearch - October 01, 2014
Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
Elasticsearch Core
- Aggregations: Significant Terms Heuristics now registered correctly (#7840, master, 1.5, 1.4 and 1.3)
- Internal: split internal free context request used after scroll and search (#7856, master, 1.5 and 1.4)
- MLT Query: Fix
percent_terms_to_match
(#7754, master and 1.5) - Core: Add
ThreadPool.terminate
to streamline shutdown (#7868, master and 1.5) - Core: Add a listener thread pool (#7837, master, 1.5 and 1.4)
- Aggs: Support for arrays of numeric values in include/exclude clauses (#7727, master and 1.5)
- Query DSL: Fix
NumberFormatException
in Simple Query String Query (#7876, master, 1.5 and 1.4) - Script scoring: Remove dependency of
DocLookup
and scorer (#7819, master, 1.5 and 1.4) - Discovery: remove any local state and use cluster service state instead (#7834, master, 1.5 and 1.4)
- NodeFaultDetection: Simplify concurrency control to fully rely on a single map
(#7889, master, 1.5 and 1.4)
- Node shutdown: make
close()
synchronized (#7885, master, 1.5, 1.4, 1.3 and 1.2) - Discovery: Close ping handler's executor service properly (#7903, master, 1.5 and 1.4)
- Internal:
MulticastChannel
should wait on receiver thread to stop during shutdown (#7835, master and 1.5) - Discovery: master fault detection fall back to cluster state thread upon error (#7908, master, 1.5 and 1.4)
- MLT Query: use minimum should match more extensive syntax (#7898, master, 1.5)
- MLT Query: Support for non-indexed documents (#7725, master and 1.5)
- Optimize API: Fix optimize behavior with
force
andflush
flags (#7904, master, 1.5 and 1.4) - REST API: Changed the root rest endpoint to use cluster service (#7933, master, 1.5 and 1.4)
- Snapshot/Restore: Allow custom metadata to specify whether or not it should be in a snapshot (#7901, master, 1.5 and 1.4)
Full house at last night's DevOps Amsterdam Meetup: DrupalCon & ELK Stack Style
Elasticsearch Ecosystem
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
- We released Elasticsearch 1.3.4 yesterday, and recommend users upgrade if they have more than 100 shards per node. You can get full details in the release blog post.
- Ever wanted to write your own input plugin for Logstash? Aaron Mildenstein shared step by step instructions in this in-depth tutorial.
- Elasticsearch Curator version 2.0 came out last Thursday, which includes updates to the Curator Python API and several new features. If you're using Curator to manage your time-based indices, this is news you can use.
- For all you .Net enthusiasts, Thomas Ardal shared a blog post exploring Date Histogram aggregations with Elasticsearch and NEST.
- Carlos Spitzer shared a very detailed how to for setting up an ELK server on RHEL 7.
- The team at Softwareschneiderei GmbH wrote up a in-depth tutorial on using the ELK stack to build your own performance and log monitoring solution.
- Vinh Quốc Nguyễn shared a how to on using Elasticsearch for full-text search on your WordPress sites.
- Jun Ohtani, our Developer Advocate in Japan, authored a post introducing some of the functionality of elasticsearch-kopf. (日本語で)
.@odolbeau and @thewhitegeek talking about #elasticsearch at @BlaBlaCar_FR for @ElasticsearchFR meetup. pic.twitter.com/Uv3RklEVSE
— David Pilato (@dadoonet) September 28, 2014
Slides & Videos
A great lessons learned story from Etsy, including how they're using Elasticsearch & Logstash in their new architecture
An incredibly in-depth use case shared at last week's Elasticsearch Netherlands Meetup. Thanks again for hosting us, Bol.com!
An end to end tutorial on the ELK stack, including code samples and Vagrant environments you can use as you watch
Elasticsearch Aggregations Webineri Tamamlandı (Video) http://t.co/70KbkLEsHU
— kodcu.com (@kodcucom) September 29, 2014
Where to Find Us
We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Director of Developer Relations, Leslie Hawthorn, know!
Austria
The Vienna Elasticsearch User Group will convene tomorrow on October 2nd at 7 PM. This next meetup will focus on Elasticsearch use cases. You can register now to attend.
Belgium
We're a proud sponsor of the 5th anniversary of DevOps Days Belgium, held in its original hometown of Ghent. Elasticsearch will have a booth at the conference, so stop by and say hello to Leslie Hawthorn between sessions. We'll be at DevOps Days October 27-28th.
China
Zeng Yong, aka Medcl, is one of our most active community members in China. He has two upcoming talks on all things Elasticsearch:
- Explore Elasticsearch Tutorial on October 9, 2014. (online course)
- Real-time data analysis at QCon Shanghai on October 17, 2014.
France
- David Pilato will be heading to BDX.io to tell you all about how to Make Sense of your Big Data. BDX.io takes place on October 17th in Bordeaux.
- David will run a hands on Elasticsearch and Kibana workshop at Forum PHP Paris 2014. You can join David on October 23rd from 9:30 AM to 1:30 PM, though separate registration is required. Forum PHP Paris runs October 23-24th at the Belfry Montrouge.
Germany
The code.talks conference is back in Hamburg this year on October 9th and 10th, though with a new name. (We had a great time there last year when this conference was known simply as Developer Conference EU.) Alexander Reelsen will be speaking once again, along with Isabel Drost-Fromm.
Here's all the Elasticsearch and ELK stack information on offer at code.talks:
- Patrick Peschlow from our partner firm codecentric AG will walk you through Elasticsearch: Lessons Learned. (Thursday, Oct. 9th at 10 AM)
- Isabel will tell you all about how to properly architect search for your website. (Friday, Oct. 10th at 10 AM)
- Alexander will give you an introduction to the ELK stack, including using it to build real-time analytics dashboards. (Friday, Oct. 10th at 2:00 PM)
The week after code.talks, Leslie Hawthorn will be heading LinuxCon Europe in Dusseldorf, where she'll speak on the panel Empowering Your Corporate Open Source Software Developers. The panel takes place on Wednesday, October 15th at 2:30 PM, and the conference runs October 13-15th at Congress Centre Dusseldorf.
And, if you happen to be in Karlsruhe on October 16th, you can catch Patrick Peschlow on Elasticsearch Performance in a Nutshell at the Search Meetup Karlsruhe. Registration is now open, and the meetup will begin at 7:15 PM.
Japan
New event: Jun Ohtani will attend the Global Hack Day with Mashape in Tokyo. Jun will present on the ELK stack and why he works for Elasticsearch. Global Hack Day takes place October 3rd-5th.
The Netherlands
Nikolay Ignatov and Welin Welchev from Propeople will present on Elasticsearch, the New School Search Engine for Drupal. Join them on Thursday at 10:45 AM to learn all about the Elasticsearch Connector module!
Norway
The next Elasticsearch Oslo Meetup is on for October 7th at 6 PM. Register now to save your seat. We'll get back to you with more details as soon as the meeting agenda is finalized.
Poland
- Honza Kral will be speaking at PyCon Poland 2014. The conference runs from October 16-19th in Orle Gniazdo. We'll be back to you with more details on Honza's presentation soon!
- Honza will speak at PolyConf 2014 on Elasticsearch's language clients, Bridging the gap: one API, 5 languages. Honza takes the stage at 5:30 PM on October 30th, and the conference runs the 30th-31st in Poznan.
Spain
The first ever Elasticsearch Meetup in Madrid has been scheduled! The meetup will take place on October 7th from 7:00 - 8:30 PM. Our very own Clinton Gormley will cover all things Elasticsearch. You can register now to save your place.
Sweden
Two of our developers, Honza Kral and Lee Hinman, will be visiting Sweden on October 7th. Starting at 7:00 PM, we'll convene our sixth Elasticsearch Stockholm Meetup, featuring a demo of our new high-level Python library elasticsearch-dsl and an overview of circuit breaker. You can register now to save your place.
Switzerland
David Pilato will return to Soft-Shake in Geneva once again this year. He will speak on Advanced Search for Your Legacy Application in the Big Data Track. Soft-Shake takes place on October 23-24th, and David will speak in Slot 5 on Thursday.
New meetup: #Elasticsearch Minneapolis on Oct. 22nd, 6:30 PM. Join us to hear @trackjs' use case. Details & reg at http://t.co/XGlqQHRzqV
— Leslie Hawthorn (@lhawthorn) October 1, 2014
United States
- In or around Bellevue, WA tomorrow, October 2nd? Cool, so are we! Please join Steve Mayzak to learn all about the ELK stack: what it is, what you can do with it and how it can provide actionable insights into your data. You can register now to attend the CloudTalk Meetup, and many thanks to these fine folks for hosting us!
- We'll have a booth at DevOps Days Chicago! Please plan to stop by and visit the Elasticsearch team in between sessions. DevOps Days Chicago runs October 7-8th at the Willis Tower.
- We'll be in the exhibits hall at Strata and Hadoop World NYC on October 13-15th. Stay tuned for news on a special Strata Meetup and more on where to find us at the conference!
- Leslie Hawthorn will be speaking at the All Things Open Conference on What Does Big Data Really Mean for Your Business? All Things Open takes place October 22-23 in Raleigh, North Carolina, and Leslie will speak at 2:15 PM on Wednesday.
Where to Find You
Our Director of Developer Relations, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.
Trainings
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- Madrid - October 7, 2014 (core Elasticsearch training)
- Amsterdam - October 14, 2014 (core Elasticsearch training)
- Amsterdam - October 16, 2014 (Getting started workshop)
- San Francisco - October 16, 2014 (Getting started workshop)
- New York - October 23, 2014 (Getting started workshop)
- London - October 28, 2014 (core Elasticsearch training)
- London - October 30, 2014 (Getting started workshop)