This Week in Elasticsearch - January 07, 2015

Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.

Elasticsearch core

  • Internal: PlainTransportFuture should not set <code>currentThread().interrupt() (#9001, master and 1.x)
  • FunctionScore: Fix avg score mode to correctly implement weighted mean (#9004, master, 1.x and 1.4)
  • Recovery: cleaner interrupt handling during cancellation (#9000, master and 1.x)
  • Internal: remove IndexCloseListener and <code>Store.OnCloseListener (#9009, master and 1.x)
  • Plugins: NPE when plugins dir is inaccessible (#8839, master, 1.x and 1.4)
  • Shutdown: Add support for Ctrl-Close event on Windows platforms to gracefully shutdown node (#8993, master and 1.x)
  • Mappings: Reencode transformed result with same xContent (#8974, master, 1.x and 1.4)
  • Mappings: Include currentFieldName into <code>ObjectMapper errors (#9020, master, 1.x, 1.4 and 1.3)
  • Aggregations: Adds methods to get to/from as strings for range aggs (#9003, master and 1.x)
  • Geo: Add optional left/right parameter to GeoJSON (#8978, master, 1.x and 1.4)
  • Search: parse terms filters on a single term as a term filter (#9014, master and 1.x)
  • Core: Terms filter lookup caching should cache values, not filters (#9027, master, 1.x, 1.4 and 1.3)
  • Parent/child: Fix concurrency issues of the _parent field data (#9030, master, 1.x and 1.4)
  • Core: ignore_unavailable shouldn't ignore closed indices (#9047, master, 1.x and 1.4)
  • Aggegrations: Added another more compact syntax for inner_hits (#8770, master and 1.x)
  • Geo: GIS envelope validation (#9091, master, 1.x and 1.4)
  • Scripting: Make _score in groovy scripts comparable (#9094, master, 1.x and 1.4)
  • Recovery: Release store lock before blocking on mapping updates (#9102, master and 1.x)
  • Internal: Remove OperationRouting abstraction (#9085, master)
  • Geo: Throw helpful exception for polygons with holes outside the shell (#9105, master, 1.x and 1.4)
  • Plugins: Installation failed when bin/ and plugins/ directories are on different filesystems (#9011, master and 1.x)
  • Settings: Ensure fields are overriden and not merged when using arrays (#8381, master and 1.x)
  • Search: Fix paging on strings sorted in ascending order (#9157, master, 1.x and 1.4)
  • Allocation: Weight deltas must be absolute deltas (#9023, master, 1.x, 1.4 and 1.3)
  • Core: Remove terms filter cache (#9856, master)
  • Startup: Use default stack size for threads (#9135, master and 1.x)
  • Stats: Added verbose option to segments api, with full ram tree as first additional element per segment (#9111, master)
  • Percolator: Fixed a bug that was caused by specifying routing on a multi percolate request causing an ArrayIndexOutOfBoundsException (#9161, master, 1.x, 1.4 and 1.3)
  • Query API: Ensure that named filters and queries defined in a wrapped query and filter are not lost (#9166, master, 1.x, 1.4 and 1.3)

Elasticsearch Ecosystem

Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.

  • Loren Siebert of DigitalGov gives a technical deep dive into how their analytics system works, and why they used Elasticsearch to build it, in this blog post.
  • If you're using IBM Bluemix apps, here's a great post on how to centralize your logs using the ELK stack, written up by the folks at IBM Bluemix.
  • This blog post from Mats Julian Olsen explains easily creating Elasticsearch queries in Python using addict, an open source project for creating nested dictionaries.
  • Want more Python goodness? The folks at ClusterHQ have been hard at work on a project called Eliot, a structured logging system that can help you trace actions inside your processes. Check out their blog post on using the ELK stack to analyze logs, and how they have enabled cross-process logging
  • William Durand gave a presentation on deploying a web application stack with Docker at the Clermont'ech API Hour in France last month. His follow-up blog post summarizes how he does the monitoring for that infrastructure - with the ELK stack, of course! - and includes some best practices for deploying the ELK stack with Docker.
  • Christian Uhl shares what he learned while recently deploying Elasticsearch for a unique use case, including some information on inserting versus updating your index.
  • Congratulations to our #elksinthewild holiday photo contest winners! You can see their elks in action throughout the newsletter this week.
  • Are you registered for Elastic{ON}15 yet? The first Elasticsearch user conference is happening March 9 - 11 in San Francisco. Tickets are going fast - get yours today!

Slides & Videos

Bhaskar Karambelkar of Verizon shared his tips on scaling Elasticsearch for production-scale data at the Washington, D.C. meetup on December 11.

From the Elasticsearch meetup in Madrid, Spain on December 15: On-the-fly ETL con EFK: Elasticsearch, Flume, Kibana (en español), by Gaspar Muñoz and Santiago Mola.

Where to find Us

We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Director of Developer Relations, Leslie Hawthorn, know!

Belgium

If you're headed out to FOSDEM (Europe's largest gathering of open source developers!) in Brussels at the end of January, be sure to squeeze time into your schedule for the BeLux (Belgium / Luxembourg) Elasticsearch user group meetup on Friday, January 30. Can't make it? Catch us at the Elasticsearch booth, or visit the Open Source Search devroom track at FOSDEM.

Germany

Alexander Reelsen will talk about performance in distributed systems on January 13 in Berlin. Very limited space, so please register.

Israel

The first-ever ELK meetup in Tel Aviv-Yafo will be happening on Tuesday, January 13, and they're kicking off with a great topic: an overview of the Kibana 4 Beta and real-life use cases. Get registered today!

New Zealand

Leslie Hawthorn and Mark Walkom will both be speaking at linux.conf.au (also known as LCA) in Auckland. This fantastic event runs January 12 - 16, and is definitely worth attending!

And accompanying LCA, we'll be having our first-ever meetup in Auckland on January 12. The schedule is still being finalized, but register now to save your spot.

Spain

Just a few days away - our own Pere Urbón-Bayes will be joining the Barcelona Meetup on January 8 to share his presentation, Make sense of your data with Elasticsearch. Register now to save your place - it's filling up quickly!

The Netherlands

Come to the CRI Service Kennissessie in Houten on January 15 to learn about enterprise logging with Elasticsearch, rsyslog, Docker, and Kibana. Save your place by registering for the meetup today.

United Arab Emirates

The Hadoop User Group is gathering in Dubai on January 27 for a meetup. Elasticsearch's own David Pilato will be joining via video conference to give a presentation titled "Make Sense of your (BIG) data!" Get registered today to save your spot.

United States

The Codemash conference in Sandusky, Ohio, will have Itamar Syn-Hershko speaking on making distributed search and analytics on big data easy with Elasticsearch. Register today to join this session on January 8!

If you're in Arizona, come to the Elasticsearch meetup at the GoDaddy offices in Scottsdale on Wednesday, January 14 for a great lineup of talks, including a Q & A session with Elasticsearch CTO Shay Banon. Get registered today for what is sure to be a great meetup!

The South Shore .NET user group in Plymouth, Massachusetts, is getting together on January 29 for a presentation on Elasticsearch for data mining. This meetup will be covering Elasticsearch basics and a few other areas, and have you ready to add Elasticsearch to your data analysis toolkit. Sign up to save your spot!

Finally, the Washington, D.C. Elasticsearch meetup group will have an "Unconference" meetup on January 14. Not sure what an unconference is? In short, it's a conference format where rather than having a structured schedule of talks decided in advance, the schedule is decided onsite, by the attendees themselves - which ensures that the content presented is of value and interest to the people who show up. Intrigued? Get registered today for this specially planned meetup - we're sure you'll have a great time!

Where to Find You

PSST! If you're a regular reader of This Week in Elasticsearch, a.k.a TWIES, you're thinking of skipping this section. You may even be thinking to yourself, yes of course I will drop a note on Twitter when I am giving a talk on all things ELK. That's awesome, because we'd like to showcase every meetup, conference presentation and workshop on Elasticsearch, Logstash, and Kibana happening worldwide. And now, we've made it even easier for you to get support for your meetup!

Head on over to our meetups page! (And we'll still totally send you swag if you're giving a talk on anything ELKy at a conference.)

Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note.  We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.

Trainings

If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:

  • Berlin - January 15, 2015 (Core Elasticsearch Training)
  • Chicago - January 15, 2015 (Core Elasticsearch Training)
  • Bangalore - January 19, 2015 (Getting Started Workshop)
  • Bangalore - January 20, 2015 (Core Elasticsearch Training)
  • Reston - January 21, 2015 (Core Elasticsearch Training)
  • Paris - January 21, 2015 (Getting Started Workshop)
  • Paris - January 22, 2015 (Core Elasticsearch Training)
  • London - January 29, 2015 (Core Elasticsearch Training)
  • Sydney - February 4, 2014 (Core Elasticsearch Training)
  • Sydney - February 5, 2014 (Hands-on Workshop with Elasticsearch, Logstash & Kibana)
  • San Francisco - February 4, 2014 (Core Elasticsearch Training)
  • San Francisco - February 5, 2014 (Hands-on Workshop with Elasticsearch, Logstash & Kibana)
  • Melbourne - February 9, 2014 (Core Elasticsearch Training)
  • New York - February 11, 2014 (Core Elasticsearch Training)
  • New York - February 12, 2014 (Hands-on Workshop with Elasticsearch, Logstash & Kibana)
  • Amsterdam - February 18, 2014 (Core Elasticsearch Training)
  • Amsterdam - February 19, 2014 (Hands-on Workshop with Elasticsearch, Logstash & Kibana)
  • Los Angeles - February 19, 2014 (Hands-on Workshop with Elasticsearch, Logstash & Kibana)
  • London - February 25, 2014 (Hands-on Workshop with Elasticsearch, Logstash & Kibana)