26 November 2014

This Week in Elasticsearch - November 26, 2014

By Alexander Reelsen

Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.

Elasticsearch Core

  • Logging: restrict files loaded as logging configuration based on their suffix (#7457, master, 1.x and 1.4)
  • Geo: Add LinearRing and LineString validity checks as defined by GeoJSON to ensure valid polygons are specified at parse time (#8433, master, 1.x, 1.4 and 1.3)
  • Internal: IndexService, synchronize close to prevent race condition with shard creation (#8557, master and 1.x)
  • Recovery: Allow to cancel recovery sources when shards are closed (#8555, master)
  • Core: Added query/filter wrapper that builds the actual query to be executed on the last possible moment to aid with index aliases and percolator queries using now date expression (#8534, master, 1.x, 1.4 and 1.3)
  • Aggregations: Do not take deleted documents into account in aggregations filters (#8540, master and 1.x)
  • Aggregations: Fix geohash grid aggregation on multi-valued fields (#8512, master, 1.x, 1.4 and 1.3)
  • Mapping: Fix conflict when updating root mapping with _all disabled (#8423, master, 1.x and 1.4)
  • Aggregations: Parser throws NullPointerException when Filter aggregation clause is empty (#8527, master, 1.x, 1.4 and 1.3)
  • Store: Remove memory/ram store (#8536, master)
  • Index: Add before/after indexDeleted callbacks to IndicesLifecycle (#8569, master and 1.x)
  • Bloom: Fix Bloom filter ram usage calculation (#8584, master, 1.x, 1.4 and 1.3)
  • Aggregations: Fix geohash grid doc counts computation on multi-valued fields (#8512, master, 1.x, 1.4 and 1.3)
  • Core: Ensure shards are deleted under lock on close (#8579, master and 1.x)
  • Term Vectors: More consistent naming for term vector[s] (#8484, master)
  • DateMath: Fix semantics of rounding with inclusive/exclusive ranges (#8556, master and 1.x)
  • Store: Use Lucene checksums if segment version is >= 4.9.0 (#8587, master, 1.x and 1.4)
  • Core: Fix script fields to be returned as a multivalued field when they produce a list (#8592, master)
  • Core: upgrade to current Lucene 5.0.0 snapshot (#8588, master)
  • Geo: Fix for geo_shape query with polygon from -180/90 to 180/-90 (#8521, master and 1.x)
  • Core: Fields defined in the _default_ mapping of an index template should be picked up when an index alias filter is parsed if a new index is introduced when a document is indexed into an index that doesn't exist yet (#8577, master, 1.x and 1.4)
  • Fielddata: Fix iterator over global ordinals (#8580, master, 1.x and 1.4)
  • Snapshot/Restore: restore with wait_for_completion=true should wait for successfully restored shards to get started (#8454, master, 1.x, 1.4 and 1.3)
  • Aggregations: Make size parsing consistent for strings (#6061, master, 1.x, 1.4 and 1.3)
  • Core: let Lucene kick off merges normally (#8643, master)
  • Aggregations: Added getProperty method to Aggregations (#8421, master and 1.x)
  • Translog: Cut over to Path API (#8611, master)
  • Parent/child: Fixed parent/child not being able to be used in alias filters (#8649, master, 1.x, 1.4 and 1.3)
  • Gateway: Cut over MetaDataStateFormat to Path API (#8609, master)
  • Aggregations: fix rounding issues on DST switch (#8339, master, 1.x, 1.4 and 1.3)

Elasticsearch Ecosystem

Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.

  • We've had a number of releases this week that will make your life as user of Elasticsearch in the public cloud better. Check out the updates to these plugins: Elasticsearch on AWS cloud plugin, Elasticsearch on Azure and Elasticsearch on Google Compute Engine.
  • Want to learn more about testing automation for distributed applications? Isabel Drost-Fromm's latest paper is a great place to start! Isabel will show you how we at Elasticsearch ensure quality checks are run often enough to speed up failure discovery, while still keeping the runtime of the whole test suite low enough for our developers to be able to run the test suite in their local development environment.
  • Looking to upgrade to the latest version of Elasticsearch? What a splendid (and secure!) idea. Konrad Beiske has some great tips, tricks and pointers.
  • Want a walk through of Sense's UI? Itamar Syn-Hershko's recent article has you covered.
  • Andrew Whitby shared some tips on using Elasticsearch's domain name analyzer/tokernizer.

Slides & Videos

Where to Find Us

We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Director of Developer Relations, Leslie Hawthorn, know!

Czech Republic

Our very own Honza Kral and Karel Minarik will speak at the next Elasticsearch CZ Meetup, taking place in Prague tomorrow, November 27 from 7-10 PM. Register now to save your place.

France

  • The first ever Elasticsearch France Meetup in Lyon will be on December 4. Schedule is still in progress, but you can register now to save your place.
  • The Elasticsearch France Meetup will return to Paris for Meetup #11 on December 17. Registration opens two weeks prior to the event, so stay tuned to this page to register and save your place.

Germany

On this week:

Data2day, Karlsruhe, Nov. 26-28: Patrick Peschlow on Elasticsearch successfully at scale.

On in December:
And for folks in or around Bonn on December 18, the Bonn ELK Meetup group will be holding their inaugural meetup. Register now to save your place.

India

Italy

Join David Pilato at CodeMotion Milan, November 26-29. David will teach you all about How to Make Sense of Your Big Data using Elasticsearch.

Israel

Fresh from his performance at CodeMotion Milan, the intrepid David Pilato will be reprising his staring role in "How to Make Sense of Your Big Data using Elasticsearch" at CodeMotion TelAviv on Monday, December 1.

United Kingdom

  • The Cambridge Developers User Group will convene on December 3 to talk about the ELK stack and other topics. Register now to save your place.
  • The London Elasticsearch Meetup will convene on December 10 at 6:30 PM. This one always fills up quickly, so register now to save your place.

United States

  • New meetup: The Phoenix Elasticsearch Meetup will convene on December 10 to share some tasty food and beverage and talk shop. No talks this time, just conviviality. Register now to join us and so we have an idea of how many humans will require snacks.
  • New meetup: The Washington DC Elasticsearch Meetup for December will be hosted by Comcast on the 11th and will feature Bhaskar Karambelkar, Tech Lead at Verizon, on Scaling Elasticsearch in Production. Register now to save your place.
  • The Monitoring NYC Meetup group will be talking full stack monitoring and alerting, featuring Elasticsearch, on December 2. Register now to save your place.
  • The Wilmington, NC DevOps & IT Meetup will get together on December 4 to talk the ELK Stack. Register now to save your place.

Where to Find You

PSST! If you're a regular reader of This Week in Elasticsearch, a.k.a TWIES, you're thinking of skipping this section. You may even be thinking to yourself, yes of course I will drop a note on Twitter when I am giving a talk on all things ELK. That's awesome, because we'd like to showcase every meetup, conference presentation and workshop on Elasticsearch, Logstash, and Kibana happening worldwide. And now, we've made it even easier for you to get support for your meetup!

Head on over to our meetups page! (And we'll still totally send you swag if you're giving a talk on anything ELKy at a conference.)

Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note.  We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.

Trainings

If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:

  • London - November 25, 2014 (Getting Started Workshop)
  • New York - December 3, 2014 (Getting Started Workshop)
  • Amsterdam - December 9, 2014 (Getting Started Workshop)
  • Amsterdam - December 10, 2014 (Core Elasticsearch Training)
  • Toronto - December 11, 2014 (Elasticsearch Core Training)
  • Bangalore - December 15, 2014 (Getting Started Workshop)
  • Bangalore - December 16, 2014 (Core Elasticsearch Training)
  • London - December 18, 2014 (Elasticsearch Core Training)