This Week in Elasticsearch - November 19, 2014

Von
Alexander Reelsen

Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.

Elasticsearch core

  • Discovery: Removed unnecessary DiscoveryService reference from LocalDiscovery, which causes circular Guice dependencies errors (#8415, master, 1.x and 1.4)
  • Mappers: Better validation of mapping JSON (#7534, master)
  • Logging: Add log4j-extras dependency for rolling logfiles (#8464, master and 1.x)
  • Scripting: Provide more context variables in update scripts (#5724, master and 1.x)
  • Mapping: Fixes Merging of default analyzer (#2716, master and 1.x)
  • Core: In the bitset cache only eagerly load bitsets for parent nested object fields (#8440, master, 1.x and 1.4)
  • Configuration: Tab characters in YAML should throw an exception (#8355, master, 1.x and 1.4)
  • Aggregations: Changed child filters to not require a random access based bitset in nested agg (#8454, master, 1.x and 1.4)
  • Percolator: Remove index.percolator.allow_unmapped_fields setting (#8439, master)
  • Core: Introduce shards level locks to prevent concurrent shard modifications (#8436, master and 1.x)
  • Search: Do not force the post-filter to be loaded into a BitSet (#8488, master)
  • Bulk: Fix of Bulk update child doc, NPE error message when parent is not specified (#8365, master and 1.x)
  • Bulk API: Missing parent routing caused NullPointerException (#8365, master and 1.x)
  • Snapshot/Restore: restore of indices that are only partially available in the cluster (#8341, master, 1.x and 1.4)
  • Snapshot/Restore: keep the last legacy checksums file at the end of restore (#8358, master, 1.x, 1.4 and 1.3)
  • Store: Use DistributorDirectory only if there are more than one data directory (#8383, master)
  • Core: Ban all usage of Future#cancel(true) (#8494, master and 1.x)

Elasticsearch Ecosystem

Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.

Congratulations to the Chocolatey Team on their successful Kickstarter! We're proud to have contributed.

  • We released Elasticsearch for Apache Hadoop 2.1.Beta3 on Monday, including support for YARN and SSL.
  • It's good to be loved! Cisco recently released OpenSOC - a framework that uses big data analytics to detect threats - which is built on top of Elasticsearch, Apache Storm, and Apache Kafka.
  • If you're a fan of all things Cisco and security, you might also be interested in checking out our latest webinar on how Mozilla built their custom SIEM application atop the ELK stack and Cisco UCS.
  • More from the it's good to be loved files: the Yelp engineering team posted a deep dive on how they scaled their use of Elasticsearch to be useful for hundreds of developers. Great story of scaling up and scaling out, both in terms of software and humans.
  • And even more from the it's good to be loved files: Nickolas Sharp, a Java/JS Tech Lead at Nike, sat down with us for an in-depth interview on how Nike uses Elasticsearch to improve their customers' experience.
  • Florian Gilcher, one of the organizers of the Berlin Elasticsearch User Group, shared an overview of Britta Weber's awesome presentation at the recent GOTO Berlin Conference; read on to Make Sense of Your Logs.
  • Njal Karevoll shared an overview of document processing and Elasticsearch. If you ever wanted some pointers on data pre-processing, Njal's tutorial is an excellent place to get started.

Slides & Videos

Our very own Jordan Sissel, creator of Logstash, brings you an intro to ELK

Christine Flood on Shenandoah and how the project uses Elasticsearch

Our very own David Pilato on Elasticsearch and Drupal (en français)

Where to find Us

We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Director of Developer Relations, Leslie Hawthorn, know!

Canada

Czech Republic

Our very own Honza Kral and Karel Minarik will speak at the next Elasticsearch CZ Meetup, taking place in Prague on November 27. Register now to save your place.

France

  • The first ever Elasticsearch France Meetup in Lyon will be on December 4. Schedule is still in progress, but you can register now to save your place.
  • The Elasticsearch France Meetup will return to Paris for meetup #11 on December 17. Registration opens two weeks prior to the event, so stay tuned to this page to register and save your place.

Germany

We have a number of upcoming events in Germany:

India

The first ever Elasticsearch Delhi meetup is coming up on November 29. Register now to save your place and to hear about several real world Elasticsearch use cases. The organizers are looking for additional speakers, so don't hesitate to contact them to volunteer!

Israel

Shay Banon and Boaz Leskes will both take the stage at DevOps Days Tel Aviv, which takes place November 23-24. Shay will hold a fireside chat on Dev, Ops and Culture, and Boaz will lead a workshop on the ELK stack.

Not heading to DevOps Days? No worries! We've got a meetup planned for November 23. You can register now to attend.

Italy

Join David Pilato at CodeMotion Milan, November 26-29. David will teach you all about How to Make Sense of Your Big Data using Elasticsearch.

Spain

Heading to Strata Barcelona? Great, so are we! You can hear from Shay Banon - How Search Can Save Your Hadoop Investment - and Costin Leau - Identifying Outliers at Scale Using Real-Time Search Engines - plus visit us on the show floor. We are going to have special gifts on hand too, so make sure to stop by to say hello.

Even if you're not planning to attend Strata, we have other great talks on offer in Barcelona on November 20. You can register now for our Special Strata Meetup.

And, just because we can't have enough awesome things happening in Barcelona in one week, you can also hear from Clinton Gormley at the NoSQL Matters Conference on November 22. Clinton will deep dive into The Elasticsearch Query DSL: Not Just for Wizards!

Call Madrid home instead of Barca? Awesome, we've got you covered!

David Pilato will teach you all about How to Make Sense of Your Big Data at Codemotion Madrid on Friday, November 21 at 12:15 PM. Not attending Codemotion Madrid? There's also a meetup scheduled for November 24, where you can hear all about your fellow users' experiences with Elasticsearch. Register now to save your place.

United Kingdom

  • New meetup: The Cambridge Developers User Group will convene on December 3 to talk about the ELK stack and other topics. Register now to save your place.
  • Colin Goodheart-Smithe will deep dive into the Ins and Outs of the ELK Stack at the upcoming Umbraco Festival in London. The event takes place on Friday, November 21 and Colin takes the stage at 3:10 PM.
  • The London Elasticsearch Meetup will convene on December 10 at 6:30 PM. This one always fills up quickly, so register now to save your place.

United States

  • New meetup: The Monitoring NYC Meetup group will be talking full stack monitoring and alerting, featuring Elasticsearch, on December 2. Register now to save your place.
  • On tomorrow night: The Seattle Elasticsearch Meetup will hold its inaugural session on November 20. They're going to talking all things Elasticsearch and Azure, and you're encouraged to bring your laptop and follow along with the presenters' tips and tricks. Register now to save your place.
  • The Wilmington, NC DevOps & IT Meetup will get together on December 4 to talk the ELK Stack. Register now to save your place.
  • Enjoying the last day of Defrag? Nifty - we'll be there! Come visit us in the exhibits area, plus make sure to catch Peter Kim's talk, Getting Insights from Uncommonly Common Occurrences in your Data. See you in Broomfield November 18-19!

Where to Find You

PSST! If you're a regular reader of This Week in Elasticsearch, a.k.a TWIES, you're thinking of skipping this section. You may even be thinking to yourself, yes of course I will drop a note on Twitter when I am giving a talk on all things ELK. That's awesome, because we'd like to showcase every meetup, conference presentation and workshop on Elasticsearch, Logstash, and Kibana happening worldwide. And now, we've made it even easier for you to get support for your meetup!

Head on over to our meetups page! (And we'll still totally send you swag if you're giving a talk on anything ELKy at a conference.)

Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note.  We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.

Training

If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:

  • London - November 25, 2014 (Getting Started Workshop)
  • New York - December 3, 2014 (Getting Started Workshop)
  • Amsterdam - December 9, 2014 (Getting Started Workshop)
  • Amsterdam - December 10, 2014 (Core Elasticsearch Training)
  • Toronto - December 11, 2014 (Elasticsearch Core Training)
  • Bangalore - December 15, 2014 (Getting Started Workshop)
  • Bangalore - December 16, 2014 (Core Elasticsearch Training)
  • London - December 18, 2014 (Elasticsearch Core Training)