Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
- Scripting: Created a parameter parse to standardised script options (#7977, master and 1.5)
- cat API: add total index memory in
_cat/indices(#7008, master and 1.5)
has_parentfilter must take parent filter into account when executing the inner query/filter (#8020, master, 1.5, 1.4 and 1.3)
- Internal: split shard search request into a local and a transport variant (#7855, master, 1.5 and 1.4)
- Core: Make fetch phase
nesteddoc aware (#7164, master and 1.5)
- Core: Fix serialization of
shortarrays (#8025, master and 1.5)
- Aggregations: Fixes scripted metrics aggregation when used as a sub aggregation (#8036, master, 1.5 and 1.4)
- Mapper: Throw exception if
null_valueis set to
null(#7273, master, 1.5, 1.4 and 1.3)
- Admin: Fix upgrade logic to work on lucene major version differences (#8013, master, 1.5 and 1.4)
- Core: Simplify discovery node initialization if version is unknown (#8055, master, 1.5 and 1.4)
- Scripting: Clear the
GroovyClassLoadercache before compiling (#8062, master, 1.5, 1.4 and 1.3)
- Internal: Fix serialization of
PendingClusterTask.timeInQueue(#8077, master, 1.5 and 1.4)
- Core: Be more explicit if
index.version.createdis not set (#8018, master, 1.5 and 1.4)
- Core: don't spin 100% CPU when disabling
refresh_interval(#8085, master, 1.5, 1.4 and 1.3)
- Core: Add cluster and index state checksums (#7586, master and 1.5)
- Internal: dangling indices import ignores aliases (#8059, master and 1.5)
I'll be a hosting an Oct 30th webcast featuring @elasticmark: Anomaly Detection with @elasticsearch http://t.co/TjCmjzjCtk
— Ben Lorica (@bigdata) October 12, 2014
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
- Kibana 4 Beta 1.1 was released last Friday, with several improvements. Keep those bug reports and feature requests coming!
- We had a huge number of plugin releases last week thanks to David Pilato and Tanguy Leroux: elasticsearch-analysis-smartcn, elasticsearch-lang-mvel, elasticsearch-mapper-attachments, elasticsearch-river-twitter, elasticsearch-transport-memcached, elasticsearch-transport-wares.
- Isabel Drost-Fromm shared the second in a three part series on how we test Elasticsearch: Testing Levels of Elasticsearch.
- Thanks to Aaron Mildenstein, we're kicking off a series of articles on the best ways to make Logstash work for you. Check out the first post in our series, on using
mutateto re-type numeric values incoming from Elasticsearch.
- Luke Marsden shared an excellent tutorial on deploying the ELK stack using Docker.
- Brad Simmons shared a how to on migrating from facets to aggregations in Elasticsearch.
- The fine folks at our partner firm, Qbox, have released an Elasticsearch connector plugin for the popular Magento e-commerce framework.
- Patrick Peschlow, with our partner firm Codecentric, wrote up a tutorial on adding transactions to Elasticsearch.
- Yan Pritzker shared the story of how his team at Reverb migrated their Elasticsearch clusters with no one noticing.
- Alex Brasetvik wrote up a deep dive on what happens when a user sends an index and a search request to Elasticsearch.
Slides & Videos
Here are the slides for my "Application Logging in the 21st century" talk I just gave at #APW2014 today: http://t.co/6Xo9mp2yFt #perl
— Tim Bunce (@timbunce) October 10, 2014
Honza Kral on the Elasticsearch Language Clients
My slides about data visualisation and D3.js from the EUROPEN 2014 conference are here: http://t.co/6RsucaAbLq
— Karel Minařík (@karmiq) October 8, 2014
Where to Find Us
We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Director of Developer Relations, Leslie Hawthorn, know!
We're a proud sponsor of the 5th anniversary of DevOps Days Belgium, held in its original hometown of Ghent. Elasticsearch will have a booth at the conference, so stop by and say hello to Leslie Hawthorn and Alan Hardy between sessions. We'll be at DevOps Days October 27-28th.
Zeng Yong, aka Medcl, is one of our most active community members in China. He has an upcoming talk on all things Elasticsearch: Real-time data analysis at QCon Shanghai on October 17, 2014.
Medcl is also busy convening the 3rd annual Elasticsearch Users Conference in China for October 25th. You can find out more information about the event and register on meetup.com.
- David Pilato will be heading to BDX.io to tell you all about how to Make Sense of your Big Data. BDX.io takes place on October 17th in Bordeaux.
- David will run a hands on Elasticsearch and Kibana workshop at Forum PHP Paris 2014. You can join David on October 23rd from 9:30 AM to 1:30 PM, though separate registration is required. Forum PHP Paris runs October 23-24th at the Beffroi Montrouge.
- The 10th Elasticsearch Paris Meetup is on for November 13th. RSVPs won't open until two weeks before the meetup, so stay tuned to this space so you can get registered.
If you happen to be in Karlsruhe on October 16th, you can catch Patrick Peschlow on Elasticsearch Performance in a Nutshell at the Search Meetup Karlsruhe. Registration is now open, and the meetup will begin at 7:15 PM.
For folks in Berlin, we'll have a booth at the GOTO Berlin conference, plus two awesome talks from Elasticsearchers:
- Britta Weber: Make sense of your logs
- Leslie Hawthorn: The Human Element in Development: What Your Tools Say about Your Company Culture
Join us at the Kosmos Event Center on November 6th and 7th for GOTO Berlin 2014!
The Hyderabad Scalability Meetup group is hosting a getting started with Elasticsearch hackathon on October 18th. Register now to save your place, looks like space is limited!
- On October 20th, community member Stefano Pampaloni will take the stage at the next IBM event in Milan to talk about Elasticsearch on IBM Power8 systems.
- Then on October 30th, we'll be working with Zimbra & Friends - the official Zimbra event in Italy (organized by Seacom) - to help show the ELK stack as a Zimbra log analyzer. And who knows, there may even be a meet up happening as well...stay tuned for more.
- Honza Kral will be speaking at PyCon Poland 2014. The conference runs from October 16-19th in Orle Gniazdo. Honza's keynote address takes place on Thursday night at 7 PM, and he's speaking on Polyglot Persistence.
- Honza will speak at PolyConf 2014 on Elasticsearch's language clients, Bridging the gap: one API, 5 languages. Honza takes the stage at 5:30 PM on October 30th, and the conference runs the 30th-31st in Poznan.
Honza Kral will be speaking at the Øredev conference in Malmö on Analyzing & Search Data Using Elastisearch. The conference runs Nov. 4-7th, and Honza takes the stage on Nov. 6th at 5:40 PM.
David Pilato will return to Soft-Shake in Geneva once again this year. He will speak on Advanced Search for Your Legacy Application in the Big Data Track. Soft-Shake takes place on October 23-24th, and David will speak in Slot 5 on Thursday.
- On Tuesday, October 21st, Skills Matter eXchange London will host an evening workshop on Real-world use cases for Elasticsearch and Lucene. Attendance is free of charge, but registration is required.
- The Elasticsearch team will be at the Big Data Analytics 2014 conference in London on November 13th and we are honored to have the UK's own Daily Mail present how Elasticsearch is helping to put newspapers ahead of the digital fold.
- New meetup: The Grand Rapids PHP Developers user group will get together on October 21st, and they'll be talking Logstash and Kibana. Register now to save your place.
- New meetup: The Elasticsearch LAX meetup group will convene on Oct. 29th to talk Multilingual text search + Elasticsearch from the bottom up. Register now to save your place.
- New meetup: The next Elasticsearch San Francisco meetup is on for October 30th. Register now to save your place.
- We'll be in the exhibits hall at Strata and Hadoop World NYC on October 13-15th. But don't just pop by our booth (P1) - we've also scheduled a special Strata NYC Meetup on Oct. 15th at Twitter's New York offices. Amongst other great talks on offer, you can hear from Costin Leau, lead developer for Elasticsearch for Apache Hadoop. He'll tell you all about our new support for SparkSQL and Apache Storm. Get registered for the meetup now, because this one always fills lightning-fast. (And many thanks to the fine folks at Twitter for hosting us.)
- Leslie Hawthorn will be speaking at the All Things Open Conference on What Does Big Data Really Mean for Your Business? All Things Open takes place October 22-23 in Raleigh, North Carolina, and Leslie will speak at 2:15 PM on Wednesday. (You might enjoy this sneak preview of her talk, too.)
- The Elasticsearch Minneapolis Meetup will host their second meeting on Oct. 22nd. Sign up now to learn all about how TrackJS & Infinite Campus use Elasticsearch. Register now to save your place - turns out our CTO, Shay Banon, will make a special guest appearance for open Q&A on all things Elasticsearch!
- The Elasticsearch Phoenix Meetup has just kicked off, and we have a special treat for the first meeting: Meet the Kibana team! You can register now to save your place; the meetup will take place on October 22nd.
- The Elasticsearch Seattle Meetup group will be hosting their inaugural meeting on October 23rd. First up: Kibana - deployment, configuration, use and extending. You can register now to save your place.
- Kurt Hurtado will speak at the San Francisco Cloud Mafia meetup on November 5th, where he'll talk the ELK stack in a DevOps environment. This meetup is incredibly popular, so get registered now if you'd like to attend.
Where to Find You
Our Director of Developer Relations, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- Palo Alto - October 29, 2014 (Core Elasticsearch training)
- London - October 30, 2014 (Getting started workshop)
- San Francisco - November 05, 2014 (Getting started workshop)
- Berlin - November 11, 2014 (Core Elasticsearch Training)
- Reston - November 13, 2014 (Core Elasticsearch Training)
- Paris - November 17, 2014 (Getting started workshop)
- Bangalore - November 17, 2014 (Getting started workshop)
- Paris - November 18, 2014 (Core Elasticsearch Training)
- Bangalore - November 18, 2014 (Core Elasticsearch Training)
- London - November 26, 2014 (Core Elasticsearch Training)