Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
- Discovery: UnicastZenPing don't rename configure host name (#7747, master and 1.4)
- Bulk operation can create duplicates on primary relocation (#7729, master and 1.4)
- Discovery: node join requests should be handled at lower priority than master election (#7733, master and 1.4)
- Discovery: not all master election related cluster state update task use Priority.IMMEDIATE (#7718, master and 1.4)
- Internal: ClusterHealthAPI does not respect waitForEvents when local flag is set (#7731, master and 1.4)
- Netty: Make sure channel closing never happens on i/o thread (#7726, master and 1.4)
- Discovery: UnicastZenPing - use temporary node ids if can't resolve node by it's address (#7719, master and 1.4)
- Recovery: remove unneeded waits on recovery cancellation (#7717, master and 1.4)
- Internal: make sure that original headers are used when executing search as part of put warmer (#7711, master and 1.4)
- Tools: Add script to grab ES version for BWC tests. (#7653, master and 1.4)
- Parent/child: If a p/c query is wrapped in a query filter then CustomQueryWrappingFilter must always be used and any filter wrapping the query filter must never be cached. (#7685, master, 1.4 and 1.3)
- Discovery: restore preference to latest unicast pings describing the same node (#7702, master and 1.4)
- Geo: Fixes BoundingBox across complete longitudinal range (#5128, master and 1.4)
- More Like This API: remove unused search_query_hint parameter (#7691, master and 1.4)
- Aggregations top_hits: Fixed inconsistent sorting of the hits (#7697, master, 1.4 and 1.3)
- Store: Improve exception from Store.failIfCorrupted (#7596, master and 1.4)
- Internal: refactor copy headers mechanism to not require a client factory (#7675, master and 1.4)
- Docs: Additional documentation for _score accessing (#7043, master and 1.4)
- Indices API: Added GET Index API (#4069, master and 1.4)
- Resiliency: Master election should demotes nodes which try to join the cluster for the first time (#7558, master and 1.4)
- Docs: A simple typo. (#7535, master and 1.4)
- Docs: Fix misnamed setting (#7676, master and 1.4)
- Translog: remove unused stream (#7683, master and 1.4)
- Bulk UDP: Removal. (#7595, master)
- Core: Use FixedBitSetFilterCache for delete-by-query. (#7581, master and 1.4)
- Rest: reroute API response didn't filter metadata (#7523, master and 1.4)
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
Average UK house prices 2014. elasticsearch GeoHashGrid + Percentiles agg at work. pic.twitter.com/kvTH0TnQ1W
— Mark Harwood (@elasticmark) September 14, 2014
- Looking for an ELK stack demo? Have a demo you'd like to share with the community? Look no further than our brand new Elasticsearch demos repository. We've got all the information on how to submit to the repository, plus our first shared demo - charting New York City traffic accidents - in this post.
- We've released NEST and Elasticsearch.NET 1.1. For full details on all the newly supported APIs and aggregations, check out the release announcement from Martijn Laarman.
- Love Amazon Kinesis? They've just released their connector to Elasticsearch.
- The Yelp Security Engineering team shared the story of how they use the ELK stack to make sense of Content Security Policy reports.
- Njal Karevoll authored a tutorial diving into all things scripting and Elasticsearch.
- Jacob Morgan wrote up a how to on using the RabbitMQ input to Logstash, including tuning tips for both.
- Alex Kamil authored a tutorial on connecting Apache HBase to Elasticsearch in 10 minutes or less.
- Jun Ohtani, our Developer Advocate in Japan, shared the story of the recent Elasticsearch Server book release party in Tokyo. (日本語で)
Slides & Videos
Greg Marzouka introduces the Elasticsearch .NET clients
All things Laravel and Elasticsearch
Where to find Us
We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Director of Developer Relations, Leslie Hawthorn, know!
The Vienna Elasticsearch User Group will convene on October 2nd at 7 PM. This next meetup will focus on Elasticsearch use cases. You can register now for the meetup.
Yann Cluchey, the awesome human who organizes the Elasticsearch London User Group, will take the stage at GOTO Aarhus on Tuesday, September 30th to share his company's Elasticsearch use case. Join Yann at 1:20 PM to hear all about Cogenta - Making Enterprise Data Available in Real Time with Elasticsearch. GOTO Aarhus runs September 29-30th.
- New Meetup: David Pilato and Richard Pijnenburg will join the Paris CTO meetup on September 24th to tell you all about Elasticsearch, making sense of your big data, and our Puppet modules. Doors open at 7 PM, and you can register now.
- Heading to JUG SummerCamp 2014? Great! So is our wonderful Developer Advocate, David Pilato. David will cover Advanced Search for Your Legacy Application, and JUG SummerCamp is on September 19th.
- The next Elasticsearch France Meetup will take place September 25th in Paris. We have limited space for this meetup, so registration will only be open for a short time. We still have a few spaces left, so register now to attend.
- David Pilato will be heading to BDX.io to tell you all about how to Make Sense of your Big Data. BDX.io takes place on October 17th in Bordeaux.
The code.talks conference is back in Hamburg this year on October 9th and 10th, though with a new name. (We had a great time there last year when this conference was known simply as Developer Conference EU.) Alexander Reelsen will be speaking once again, along with Isabel Drost-Fromm.
Here's all the Elasticsearch and ELK stack information on offer at code.talks:
- Patrick Peschlow from our partner firm codecentric AG will walk you through Elasticsearch: Lessons Learned. (Thursday, Oct. 9th at 10 AM)
- Isabel will tell you all about how to properly architect search for your website. (Friday, Oct. 10th at 10 AM)
- Alexander will give you an introduction to the ELK stack, including using it to build real-time analytics dashboards. (Friday, Oct. 10th at 2:00 PM)
The week after code.talks, Leslie Hawthorn will be heading LinuxCon Europe in Dusseldorf, where she'll speak on the panel Empowering Your Corporate Open Source Software Developers. The panel takes place on Wednesday, October 15th at 2:30 PM, and the conference runs October 13-15th at Congress Centre Dusseldorf.
You can join us next Monday, September 22nd for our first ever Elasticsearch Netherlands Meetup in Utrecht! The fine folks at Bol.com will share all about how they use the ELK stack and have scaled it out to easily process a daily volume of 300M log events and 185 GB of data. Register now to attend.
Can't join us in person? No problem. Thanks to our friends at Bol, the event will be livestreamed.
Save the date & watch this space: next #Elasticsearch NL meetup will be livestreamed at http://t.co/Ily9dsBFSo (22 Sept, starting @ 19 CEST)
— Leslie Hawthorn (@lhawthorn) September 1, 2014
We're also super excited that this year's DrupalCon EU will be in our hometown of Amsterdam on Sept. 29th - Oct. 3rd. Leslie Hawthorn will be out and about in the hallway track, so make sure to say hello to her! There are also some great talks on all things Elasticsearch and the ELK stack on offer from these Drupalistas:
- The DevOps Amsterdam Meetup group is hosting a special session welcoming attendees from DrupalCon. You can join us on Tuesday, September 30th at 6 PM at Old School Amsterdam, steps away from the RAI conference center. Lee Hinman will be on hand to talk about Elasticsearch, monitoring and alerting. You can register now to attend.
- Steven Merill of Phase2 Technology will teach you all about using open source logging and monitoring tools. The best part? He'll share all about how the ELK stack powers log management for Drupal.org (think billions of logs). Steve speaks on Wednesday at 2:15 PM.
- Nikolay Ignatov and Welin Welchev from Propeople will present on Elasticsearch, the New School Search Engine for Drupal. Join them on Thursday at 10:45 AM to learn all about the Elasticsearch Connector module!
The next Elasticsearch Oslo Meetup is on for October 7th at 6 PM. Register now to save your seat. We'll get back to you with more details as soon as the meeting agenda is finalized.
Patrick Peschlow from our partner firm codecentric AG will take the stage at the 2nd Annual Coding Serbia Conference on September 25th. Join him at 8:00 PM to learn how you can go from Elasticsearch Novice to Expert in just 90 minutes.
The first ever Elasticsearch Meetup in Madrid has been scheduled! The meetup will take place on October 7th from 7:00 - 8:30 PM. Our very own Clinton Gormley will cover all things Elasticsearch. You can register now to save your place.
The fifth Elasticsearch Stockholm Meetup is on for October 1st at 5:30 PM. The agenda will be posted shortly, but in the meantime register now to ensure there's a seat for you.
Please join the folks from the Enterprise Search Cambridge group for a full day Elasticsearch hackathon on September 26th! You can register now to attend.
- On tomorow night: Join us for the latest and greatest Elasticsearch Austin Meetup on Thursday, September 18th from 6-8 PM. Agenda is still being finalized, but you can register now!
- PuppetConf is around the corner! If you find yourself in the San Francisco area join us in listening to Jordan Sissel's talk on Got Logs? Get answers with Elasticsearch ELK, Wednesday 24th, 5:10 PM. The conference runs September 20-24th.
- In or around Bellevue, WA on October 2nd? Cool, so are we! Please join Steve Mayzak to learn all about the ELK stack: what it is, what you can do with it and how it can provide actionable insights into your data. You can register now to attend the CloudTalk Meetup, and many thanks to these fine folks for hosting us!
Where to Find You
Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- Paris - September 25, 2014 (Getting started with Elasticsearch & the ELK stack)
- Stockholm - October 7, 2014 (core Elasticsearch training)
- Madrid - October 7, 2014 (core Elasticsearch training)
- Amsterdam - October 14, 2014 (core Elasticsearch training)
- Amsterdam - October 16, 2014 (Getting started with Elasticsearch & the ELK stack)
- San Francisco - October 16, 2014 (Getting started with Elasticsearch & the ELK stack)
- Los Angeles - October 21, 2014 (core Elasticsearch training)
- New York - October 23, 2014 (core Elasticsearch training)
- New York - October 23, 2014 Getting started with Elasticsearch & the ELK stack)
- London - October 28, 2014 (core Elasticsearch training)
- London - October 30, 2014 Getting started with Elasticsearch & the ELK stack)