This Week in Elasticsearch - August 27, 2014

Par
Alexander Reelsen

Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.

Elasticsearch core

  • Geo: fixes circle radius calculation (#7301, master and 1.4)
  • Warmer API: Add support for query cache (#7326, master and 1.4)
  • Aggregations: Scriptable Metrics Aggregation (#5923, master and 1.4)
  • Internal: make sure that all shard level requests hold the original indices (#7319, master and 1.4)
  • Stats: Prevent NullPointerException in ShardStats (#7356, master and 1.4)
  • Query Cache: Add hit and miss count (7355#, master and 1.4)
  • Term vector API: return found: false for docs between index and refresh (#7121, master and 1.4)
  • Aggregations: Remove the logic to optionally sort/dedup values on the fly (#7276, master and 1.4)
  • Facets: Removal from master (#7337, master)
  • Internal: Remove CacheRecycler (#7366, master)
  • Parser: Clarify XContentParser/Builder interface for binary vs. utf8 values (#7367, master and 1.4)
  • More Like This Query: Switch to using the multi-termvectors API (#7014, master and 1.4)
  • Internal: make sure that multi_percolate request hands over its context and headers to its corresponding shard requests (#7371, master and 1.4)
  • Internal: make sure that multi_search request hands over its context and headers to its corresponding search requests (#7374, master and 1.4)
  • Aggregations: Fixes pre and post offset serialization for histogram aggs (#7312, master and 1.4)
  • Store: Allow to get metadata from arbitrary commit points (#7376, master, 1.4 and 1.3)
  • _ttl: Report conflict when trying to disable _ttl (#7316, master and 1.4)
  • Store: Improve recovery / snapshot restoring file identity handling (#7351, master and 1.4)
  • Java API: Add back string op type to IndexRequest (#7387, master and 1.4)
  • Dependencies: Upgrade to Apache HttpComponents client 4.3.5 (#7342, master and 1.4)
  • Internal: move index templates api back to indices category and make put template and create index implement IndicesRequest (#7378, master and 1.4)
  • More Like This Query: defaults to all possible fields for items (#7382, master and 1.4)
  • Engine: Force optimize was not passed to shard request (#7404, master, 1.4, 1.3 and 1.2)
  • Internal: made original indices optional for broadcast delete and delete by query shard requests (#7406, master and 1.4)
  • Context suggester: fix infinite loop in GeolocationContextMapping (#7433, master, 1.4 and 1.3)
  • Internal: discovery.id.seed is ignored (#7437, master and 1.4)
  • Stats: add segments.index_writer_max_memory to see index writer's max RAM usage before buffered documents must be written to a new segment (#7440, master and 1.4)
  • Core: use Java's built-in ConcurrentHashMap (#7296, master and 1.4)
  • Mapping API: report conflict on merge and throw exception on doc_values in the _all field (#7377, master and 1.4)
  • Engine: Wait until engine is started up when acquiring searcher (#7455, master, 1.4 and 1.3)
  • Snapshot/Restore API: Add BWC layer to .si / segments_N hashing (#7434, master and 1.4)
  • Internal: Add LZF safe encoder in LZFCompressor (#7466, master and 1.4)
  • Internal: Add all unsafe variants of LZF compress library functions to forbidden APIs (#7468, master and 1.4)
  • Parent/Child: If _parent field points to a non existing parent type, then skip the has_parent query/filter (#7362, master, 1.4 and 1.3)
  • Parent/child: Add missing support for the field data loading option to the _parent field (#7402, master, 1.4 and 1.3)

Elasticsearch Ecosystem

Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.

Slides & Videos

New to Elasticsearch? Rick Winfrey introduces you to its features, with great getting started code examples in Ruby. Thanks Rick!

Isabel Drost-Fromm treats us to walk through of the options of determining search quality & how Elasticsearch can help you in your quest

Rafael Lopes on all things Elasticsearch, AWS & High Availability at the recent Elasticsearch Brasil Meetup

The Guardian <3s ELK stack. But we can't claim credit for this one.

Where to find Us

We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Community Manager, Leslie Hawthorn, know!

Denmark

Yann Cluchey, the awesome human who organizes the Elasticsearch London User Group, will take the stage at GOTO Aarhus on Tuesday, September 30th to share his company's Elasticsearch use case. Join Yann at 1:20 PM too hear all about Cogenta - Making Enterprise Data Available in Real Time with Elasticsearch. GOTO Aarhus runs September 29-30th.

Ireland

We've got two great talks coming up on September 4th! Boaz Leskes will present at NoSQL Matters Dublin on Elasticsearch: Deep Dive into Analytics Using Aggregations. For those not attending the conference or in need of even more Elasticsearch goodness, you can catch Boaz on the full ELK stack at the DevOps Ireland Meetup that same evening. Many thanks to the DevOps Ireland folks for hosting us!

France

Heading to JUG SummerCamp 2014? Great! So is our wonderful Developer Advocate, David Pilato. David will cover Advanced Search for Your Legacy Application, and JUG SummerCamp is on September 19th.

The Netherlands

We're also super excited that this year's DrupalCon EU will be in our hometown of Amsterdam on Sept. 29th - Oct. 3rd. Leslie Hawthorn will be out and about in the hallway track, so make sure to say hello to her! There are also some great talks on all things Elasticsearch and the ELK stack on offer from these Drupalistas:

  • Steven Merill of Phase2 Technology will teach you all about using open source logging and monitoring tools. The best part? He'll share all about how the ELK stack powers log management for Drupal.org (think billions of logs). Steve speaks on Wednesday at 2:15 PM.
  • Nikolay Ignatov and Welin Welchev from Propeople will present on Elasticsearch, the New School Search Engine for Drupal. Join them on Thursday at 10:45 AM to learn all about the Elasticsearch Connector module!

Norway

Heading to JavaZone in Oslo? David Pilato will host a workshop showing you how to get Hands on with Elasticsearch and Kibana. David's workshop takes place on September 9th at 1:30 PM, and the conference runs September 9-11th.

United Kingdom

  • New Meetup: The Elasticsearch London User Group will convene on September 3rd, and registration is now open. Don't let that we're looking for speaker notice fool you - our stalwart community organizer, Yann Cluchey of Cogenta, has a full line up set. We'll announce full agenda and location details soon!
  • Please join the folks from the Enterprise Search Cambridge group for a full day Elasticsearch hackathon on September 26th! You can register now to attend.

United States

  • New Meetup: The LOPSA San Diego user group will get together tomorrow at 6 PM to talk Docker basics plus using Docker to deploy and manage a live multi-node cluster of Elasticsearch, Redis, and Logstash. Registration is still open!
  • If you're attending the Heartland Developers Conference in Omaha, make sure to check out Andrew Oschner's talk on the ELK stack. Andrew will teach you all about troubleshooting logging in production, and he'll have some cool Elasticsearch swag to hand out to session participants. Andrew speaks on September 4th, and the conference runs the 3rd-5th.
  • Attending the first ever FOSSETCON? Join our very own Jordan Sissel for a Keynote Address on Open Source is More than Code. You can also see another of our stalwart Logstash developers, Aaron Mildenstein, on an introduction to the ELK stack. FOSSETCON runs September 11-13th in Orlando.
  • Join several of our core developers for the latest Denver Meetup on September 15th. We're still finalizing the agenda, but our hosts, Sendgrid, will be discussing their Elasticsearch use case. Register now for updates on timing and location.
  • PuppetConf is around the corner! If you find yourself in the San Francisco area join us in listening to Jordan Sissel's talk on Got Logs? Get answers with Elasticsearch ELK, Wednesday 24th, 5:10 PM. The conference runs September 20-24th.

Where to Find You

Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.

Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note.  We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.

Training

If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:

  • New York - August 27, 2014 (ELK workshop)
  • London - September 2, 2014 (core Elasticsearch training)
  • London - September 4, 2014 (ELK workshop)
  • Paris - September 9, 2014 (core Elasticsearch training)
  • San Francisco - September 11, 2014 (ELK workshop)
  • Austin - September 18, 2014 (ELK workshop)
  • Bangalore - September 24, 2014 (core Elasticsearch training)
  • Paris - September 25, 2014 (ELK workshop)
  • Stockholm - October 7, 2014 (core Elasticsearch training)
  • Madrid - October 7, 2014 (core Elasticsearch training)
  • Amsterdam - October 14, 2014 (core Elasticsearch training)
  • Chicago - October 14, 2014 (core Elasticsearch training)
  • San Francisco - October 14, 2014 (core Elasticsearch training)
  • Los Angeles - October 21, 2014 (core Elasticsearch training)