Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
All new to Elasticsearch? Dive right in with Dr. Myagmar's overview.
- Parent/Child: Adding
minscore mode to parent-child queries (#7771, master and 1.5)
- cat API: Add file descriptor details to
cat/nodes(#7652, master and 1.5)
- Snapshot/Restore API: Fix snapshot of a single closed index (#8047, master, 1.5, 1.4 and 1.3)
- Search API: Fixing copy/paste mistake in
SearchRequest.extraSourceexception message (#8118, master and 1.5)
- Build: Avoid shading of
org.joda.convertpackage (#3558, master and 1.5)
- Aggregations: Buckets can now be serialized outside of an aggregation (#8113, master)
- Netty: Support to bind on multiple host/port pairs (#8098, master, 1.5 and 1.4)
- Core: Fix index throttling to notice a change to
max_merge_counton the next merge start/finish (#8132, master, 1.5, 1.4 and 1.3)
- Core: cutover to Lucene's query rescorer (#7707, master and 1.5)
- Circuit breaker: Allow setting individual breakers to
noopbreakers (#8135, master and 1.5)
- Mappings: Store
_timestampby default (#8139, master)
- Index names: Raise maximum index name length to 255 bytes (#8158, master and 1.5)
- Search API: Add
query_string(#8164, master and 1.5)
- Parent/child: Reduce memory usage in top children query (#8165, master, 1.5, 1.4 and 1.3)
- Boostrap: Log startup exception to console if needed and to file as error (#6581, master and 1.5)
- Build: Use groovy-x.y.z-indy jar for better scripting performance (#8183, master)
- Indices: Change
500(#7632, master and 1.5)
"@peterkimnyc: Interactive analysis of 14 years of #Chicago crime data using @elasticsearch, #logstash, #kibana. pic.twitter.com/Sk1pGyRFic"
— Will Button (@wfbutton) October 19, 2014
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
- Antonio Bonuccelli treated us to a deep dive yesterday on using the ELK stack to visualize security data. Check out his post on setting up a clustered instance of Elasticsearch to visualize firewall and honeypot datasources, namely IPtables & KippoSSH.
- Isabel Drost-Fromm published the last in her three part series on Testing and QA of Elasticsearch. Read on to learn all about Elasticsearch randomized testing - what it is, why we do it and how it makes our users lives better.
- Ram Viswanadha authored an in-depth tutorial on Building Scalable Search from Scratch Using Elasticsearch. Great for newbies and useful to the seasoned user, too!
- Tim Bell from CERN just completed an interview with the fine folks at opensource.com. Read on to learn about CERN's use of OpenStack, Elasticsearch and other cloud tech.
- It's good to be loved. Check out the next generation data pipeline architecture of American Express, powered by Apache Hadoop and Elasticsearch.
- Mehmet Dursun authored a detailed tutorial on using the ELK stack and OSSEC to create a cyber threat monitoring system.
- Eddie Abou-Jaoude shared a post on how Elasticsearch lays the foundations for UK open government projects.
I'll be a hosting an Oct 30th webcast featuring @elasticmark: Anomaly Detection with @elasticsearch http://t.co/f0UZQfl5Hv
— Ben Lorica (@bigdata) October 19, 2014
Slides & Videos
ElasticSearch入門 [embedded in http://t.co/nxts4erumF] https://t.co/7cP1ZJQNSZ
— ymizushi (@ymizushi) October 20, 2014
Slides for my talk at @bdxio are available at http://t.co/MfGF71Cd1a :) #elasticsearch
— David Pilato (@dadoonet) October 18, 2014
Yann Cluchey, the stalwart organizer of the Elasticsearch London Meetup, deep dives on GfK's architecture and their ELK stack use case
Need a quick review of our latest webinar? Look no further!
After fairly extensive testing, really happy that we've selected ElasticSearch over our doc-store's full-text index. Ferrari vs. Golf-cart
— Andrew Newdigate (@suprememoocow) October 15, 2014
Where to find Us
We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Director of Developer Relations, Leslie Hawthorn, know!
- We're a proud sponsor of the 5th anniversary of DevOps Days Belgium, held in its original hometown of Ghent. Elasticsearch will have a booth at the conference, so stop by and say hello to Leslie Hawthorn and Alan Hardy between sessions. We'll be at DevOps Days October 27-28th.
- David Pilato will host a Hands on Elasticsearch and Kibana workshop at Devoxx Belgium 2014. David's workshop kicks off on Monday, November 10th at 9:30 AM, and the conference runs from November 10-14th in Antwerp.
Zeng Yong, aka Medcl, is one of our most active community members in China. He is busy convening the 3rd annual Elasticsearch Users Conference in China for October 25th. You can find out more information about the event and register on meetup.com.
- Honza Kral will present Analyzing and Searching Data using Elasticsearch at the Findwise Elasticsearch Breakfast Seminar on November 5th in Copenhagen. Registration is free of charge, but you'll need to sign up to attend.
- Simon Willnauer will keynote the DrivingIT Conference in Copenhagen. Stay tuned for more schedule details, but Simon will cover Randomized Testing during his presentation, a.k.a. "evil tests." The conference takes place on November 14, 2014.
- David Pilato will run a hands on Elasticsearch and Kibana workshop at Forum PHP Paris 2014. You can join David on October 23rd from 9:30 AM to 1:30 PM, though separate registration is required. Forum PHP Paris runs October 23-24th at the Beffroi Montrouge.
- The 10th Elasticsearch Paris Meetup is on for November 13th. RSVPs won't open until two weeks before the meetup, so stay tuned to this space so you can get registered.
Utiliser ElasticSearch avec Drupal? Ca se passe à #drupagora http://t.co/zIg464wNny CC @dadoonet @gastons78 @AlterWay pic.twitter.com/kwTBlNH8zP
— Drupagora (@Drupagora) October 15, 2014
For folks in Berlin, we'll have a booth at the GOTO Berlin conference, plus two awesome talks from Elasticsearchers:
- Britta Weber: Make sense of your logs
- Leslie Hawthorn: The Human Element in Development: What Your Tools Say about Your Company Culture
Join us at the Kosmos Event Center on November 6th and 7th for GOTO Berlin 2014!
- On October 20th, community member Stefano Pampaloni will take the stage at the next IBM event in Milan to talk about Elasticsearch on IBM Power8 systems.
- Then on October 30th, we'll be working with Zimbra & Friends - the official Zimbra event in Italy (organized by Seacom) - to help show the ELK stack as a Zimbra log analyzer. You may also with to join us that evening at 5:30 PM for the second Elasticsearch Italy meetup, including talks and networking.
Our CEO, Steven Schuurman, will be speaking at the Web Summit conference. We'll have full details on his talks coming soon, but for now check out this amazing event happening in Dublin on November 4-7th.
The Japan Java Users Group Community Conference is coming up on November 15th in Tokyo. You can hear from our Developer Advocate, Jun Ohtani, on all things Elasticsearch and Kibana. Elasticsearch is proud to sponsor this great community event!
- Honza Kral will be speaking at PyCon Poland 2014. The conference runs from October 16-19th in Orle Gniazdo. Honza's keynote address takes place on Thursday night at 7 PM, and he's speaking on Polyglot Persistence.
- Honza will speak at PolyConf 2014 on Elasticsearch's language clients, Bridging the gap: one API, 5 languages. Honza takes the stage at 5:30 PM on October 30th, and the conference runs the 30th-31st in Poznan.
Honza Kral will be speaking at the Øredev conference in Malmö on Analyzing & Search Data Using Elastisearch. The conference runs Nov. 4-7th, and Honza takes the stage on Nov. 6th at 5:40 PM.
David Pilato will return to Soft-Shake in Geneva once again this year. He will speak on Advanced Search for Your Legacy Application in the Big Data Track. Soft-Shake takes place on October 23-24th, and David will speak in Slot 4 on Friday.
The Elasticsearch team will be at the Big Data Analytics 2014 conference in London on November 13th and we are honored to have the UK's own Daily Mail present how Elasticsearch is helping to put newspapers ahead of the digital fold.
- Leslie Hawthorn will be speaking at the All Things Open Conference on What Does Big Data Really Mean for Your Business? All Things Open takes place October 22-23 in Raleigh, North Carolina, and Leslie will speak at 2:15 PM on Wednesday. (You might enjoy this sneak preview of her talk, too.)
- The Elasticsearch Minneapolis Meetup will host their second meeting on Oct. 22nd. Sign up now to learn all about how TrackJS & Infinite Campus use Elasticsearch. Register now to save your place - turns out our CTO, Shay Banon, will make a special guest appearance for open Q&A on all things Elasticsearch!
- The Elasticsearch Phoenix Meetup has just kicked off, and we have a special treat for the first meeting: Meet the Kibana team! You can register now to save your place; the meetup will take place on October 22nd.
- The Elasticsearch Seattle Meetup group will be hosting their inaugural meeting on October 23rd. First up: Kibana - deployment, configuration, use and extending. You can register now to save your place.
- The Elasticsearch LAX meetup group will convene on Oct. 29th to talk Multilingual text search + Elasticsearch from the bottom up. Register now to save your place.
- The next Elasticsearch San Francisco meetup is on for October 30th. Register now to save your place.
- Kurt Hurtado will speak at the San Francisco Cloud Mafia meetup on November 5th, where he'll talk the ELK stack in a DevOps environment. This meetup is incredibly popular, so get registered now if you'd like to attend.
Where to Find You
Our Director of Developer Relations, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- London - October 30, 2014 (Getting started workshop)
- Berlin - November 11, 2014 (Core Elasticsearch Training)
- Reston - November 13, 2014 (Core Elasticsearch Training)
- Paris - November 17, 2014 (Getting started workshop)
- Bangalore - November 17, 2014 (Getting started workshop)
- Paris - November 18, 2014 (Core Elasticsearch Training)
- Bangalore - November 18, 2014 (Core Elasticsearch Training)
- London - November 26, 2014 (Core Elasticsearch Training)