This Week in Elasticsearch - November 12, 2014
Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
Working on Search at #StackOverflow... Our total searches in the @elasticsearch cluster is > 14,000,000,000. Yes, that's BILLION. Holy crap.
— David Haney (@haneycodes) November 11, 2014
- Lucene: Update to Lucene 5.0 (#8347, master only)
- Core: Cut over to Path API for file deletion (#8366, master and 1.x)
- Translog: Don't leak filehandles on corruption or truncation
(#8372, master, 1.x and 1.4)
- Recovery: Refactor shard recovery from anonymous class to
- Discovery: Don't wait for
joinThreadwhen stopping (#8359, master and 1.x)
- Cluster Health: Observe cluster state via listener (#8350, master and 1.x)
- Core: Allow
-SNAPSHOTversions to be parsed by
Version.fromString(#8379, master and 1.x)
- Internal: Ban buggy IOUtils methods with forbidden API (#8375, master)
- Core: Drop pre 0.90 compression backwards compatibility (#8385, master and 1.x)
- Shard Allocation: Take percentage watermarks into account for reroute listener on disk based allocation (#8382, master and 1.x)
- Internal: Harden recovery for old segments (#8399, master, 1.x, 1.4 and 1.3)
- Store: Calculate Alder32 Checksums for legacy files in
Store#checkIntegrity(#8407, master, 1.x, 1.4 and 1.3)
- Store: Synchronize operations that modify file mappings on
DistributorDirectory(#8408, master and 1.x)
- Core: Remove NestedDocsFilter, because it isn't used and also don't eagerly load it in bitset filter cache(#8414, master, 1.x and 1.4)
- Internal: Inverse
DocIdSets' heuristic to find out fast
- Scripting: Add
AbstractSearchScript(#8416, master, 1.x and 1.4)
- Core: add
regexpquery/filter (#8357, master and 1.x)
- Internal: extend refresh-mapping logic to the
_default_type (#8413, master and 1.x)
- Query: add option for analyze wildcard/prefix also to
simple_query_stringquery (#8422, master and 1.x)
NEST 1.2.3 now available https://t.co/kjQeK3BZhK. Compatibility issues with ES 1.4 have been addressed.
— Greg Marzouka (@gregmarzouka) November 11, 2014
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
Introducing Raigad - An Elasticsearch Sidecar http://t.co/Ar0aS2CR08
— NetflixOSS (@NetflixOSS) November 10, 2014
- We released Kibana 4 Beta 2 yesterday. Check out this blog post for full details.
- Paul Stack from OpenTable published an article on using Elasticsearch with Hubot.
- Philippe Creux authored a great overview on using the ELK stack to build a data analysis dashboard. Lots of useful code sample on offer.
- Brand new to Logstash? Looking for a few tips and tricks? Look no further.
- The CyberAgent team shared details on their use case: Elasticsearch & Kibana for monitoring Cassandra. (日本語で)
Our very own @TarjeiRomtveit has written an article in NDC magazine about @elasticsearch http://t.co/KObB2gnSLL @MonokkelAS
— Aleksander Stensby (@astensby) November 11, 2014
Slides & Videos
Presented by Honza Kral at last week's Øredev 2014 Conference
Presented by Kurt Hurtado at last week's San Francisco Cloud Mafia Meetup
Presented by David Pilato at last week's JMahgreb 2014 Conference
Where to find Us
We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Director of Developer Relations, Leslie Hawthorn, know!
Headed to #LuceneSolrRev in DC? Join the #elasticsearch team for a meetup tomorrow to hear from @rcmuir & others http://t.co/JmHFihYHcj
— elasticsearch (@elasticsearch) November 11, 2014
I'll be talking about #Logstash, #Elasticsearch and #Kibana at @BSidesTO -- ELK != Plain 'Ol Syslog - More info: http://t.co/iCaXCvDQbY
— Jason Kendall (@coolacid) October 29, 2014
Simon Willnauer will keynote the DrivingIT Conference in Copenhagen. Stay tuned for more schedule details, but Simon will cover Randomized Testing during his presentation, a.k.a. "evil tests." The conference takes place on November 14, 2014.
- The 10th Elasticsearch Paris Meetup is on for November 13. RSVPs are now open, so register ASAP. We fill all of our seats very quickly for the Paris meetups.
- You can also hear from David Pilato at the Drupagora conference on all matters Elasticsearch & Drupal. Join David on November 14 at 2:00 PM for his talk; the conference runs the full day in Paris.
- The first ever Elasticsearch France Meetup in Lyon will be on December 4. Schedule is still in progress, but you can register now to save your place.
We have a number of upcoming events in Germany:
- Open Source Monitoring Conference, Nuremburg: Alexander Reelsen on Using the ELK stack in Systems Administration.
- Data2day, Karlsruhe: Patrick Peschlow on Elasticsearch successfully at scale.
- Mimacom Days, Stuttgart: Dmitri Marx on Making Sense of your data to give you actionable insights.
Isabel Drost-Fromm will be giving two talks at ApacheCon Europe:
- Patches Welcome: Contributing to Apache Projects in a Nutshell - November 17 at 2:40 PM
- Your search doesn't work: how to find out whether the search box you offer your users is helping - November 19 at 10:40 AM
ApacheCon Europe is on November 17-21, 2014 in Budapest.
Shay Banon and Boaz Leskes will both take the stage at DevOps Days Tel Aviv, which takes place November 23-24. Shay will hold a fireside chat on Dev, Ops and Culture, and Boaz will lead a workshop on the ELK stack.
Not heading to DevOps Days? No worries! We've got a meetup planned for November 23. You can register now to attend.
Join David Pilato at CodeMotion Milan, November 26-29. David will teach you all about How to Make Sense of Your Big Data using Elasticsearch.
- The Japan Java Users Group Community Conference is coming up on November 15 in Tokyo. You can hear from our Developer Advocate, Jun Ohtani, on all things Elasticsearch and Kibana. Elasticsearch is proud to sponsor this great community event!
- The next Elasticsearch Study Session has been scheduled for November 18 at 7:30 PM. Register now to save your place.
Next Elasticsearch NL meetup is scheduled. Mark the date: November 17. Talks by ING and RIPE. See you there! RSVP: http://t.co/CLtdU5ySMf
— Boaz Leskes (@bleskes) October 29, 2014
New meetup on *tonight*! Bucharest Big Data user group will talk #OpenSource Analytics using Hadoop & #Elasticsearch http://t.co/xfHKkB9BhO
— Leslie Hawthorn (@lhawthorn) November 5, 2014
Heading to Strata Barcelona? Great, so are we! You can hear from Shay Banon - How Search Can Save Your Hadoop Investment - and Costin Leau - Identifying Outliers at Scale Using Real-Time Search Engines - plus visit us on the show floor. We are going to have special gifts on hand too, so make sure to stop by to say hello.
Even if you're not planning to attend Strata, we have other great talks on offer in Barcelona on November 20. You can register now for our Special Strata Meetup.
And, just because we can't have enough awesome things happening in Barcelona in one week, you can also hear from Clinton Gormley at the NoSQL Matters Conference on November 22. Clinton will deep dive into The Elasticsearch Query DSL: Not Just for Wizards!
Call Madrid home instead of Barca? Awesome, we've got you covered!
David Pilato will teach you all about How to Make Sense of Your Big Data at Codemotion Madrid on Friday, November 21 at 12:15 PM. Not attending Codemotion Madrid? There's also a meetup scheduled for November 24, where you can hear all about your fellow users' experiences with Elasticsearch. Register now to save your place.
- New Meetup: The London Elasticsearch Meetup will convene on December 10 at 6:30 PM. This one always fills up quickly, so register now to save your place.
- The Elasticsearch team will be at the Big Data Analytics 2014 conference in London on November 13 and we are honored to have the UK's own Daily Mail present how Elasticsearch is helping to put newspapers ahead of the digital fold.
- Colin Goodheart-Smithe will deep dive into the Ins and Outs of the ELK Stack at the upcoming Umbraco Festival in London. The event takes place on Friday, November 21 and Colin takes the stage at 3:10 PM.
- New meetup: The Seattle Elasticsearch Meetup will hold its inaugural session on November 20. They're going to talking all things Elasticsearch and Azure, and you're encouraged to bring your laptop and follow along with the presenters' tips and tricks. Register now to save your place.
- New meetup: The Wilmington, NC DevOps & IT Meetup will get together on December 4 to talk the ELK Stack. Register now to save your place.
- Heading to Colorado for Defrag? Nifty - we'll be there! Come visit us in the exhibits area, plus make sure to catch Peter Kim's talk, Getting Insights from Uncommonly Common Occurrences in your Data. See you in Broomfield November 18-19!
Meetup November in Philadelphia: Building an educational content search application using #Elasticsearch @Wharton http://t.co/VVVU7P5H6h
— jamestyack (@jamestyack) October 28, 2014
PDX friends, @smayzak will demo #Kibana 4 at the November 13#Elasticsearch meetup + what's new in the ELK stack! http://t.co/3CE0TPmEYZ
— Leslie Hawthorn (@lhawthorn) November 5, 2014
Where to Find You
Our Director of Developer Relations, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- Paris - November 17, 2014 (Getting started workshop)
- Bangalore - November 17, 2014 (Getting started workshop)
- Bangalore - November 18, 2014 (Core Elasticsearch Training)
- London - November 25, 2014 (Getting Started Workshop)
- New York - December 2, 2014 (Elasticsearch Core Training)
- New York - December 3, 2014 (Getting Started Workshop)
- Amsterdam - December 9, 2014 (Getting Started Workshop)
- San Francisco - December 9, 2014 (Getting Started Workshop)
- Amsterdam - December 10, 2014 (Core Elasticsearch Training)
- Toronto - December 11, 2014 (Elasticsearch Core Training)
- Bangalore - December 15, 2014 (Getting Started Workshop)
- Bangalore - December 16, 2014 (Core Elasticsearch Training)
- London - December 16, 2014 (Elasticsearch Core Training)