Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
- Query API: Be stricter parsing ids for
idsquery (#7945, master and 1.5)
- Mappings: Make lookup structures immutable (#7486, master and 1.5)
- Discovery: only accept unicast pings when started (#7950, master, 1.5, 1.4 and 1.3)
- Discovery: add a finalize round to multicast pinging (#7924, master, 1.5 and 1.4)
- Dependencies: Upgrade to Jackson 2.4.2 (#7932, master, 1.5 and 1.4)
- Logging: suppress long mapping logging during mapping updates (unless in
TRACE) (#7949, master and 1.5)
- Term Vectors: support for realtime (#7846, master and 1.5)
- Internal: Don't let
tookbe negative (#7968, master and 1.5)
- Admin: show open and closed indices in
_cat/indices(#7936, master, 1.5 and 1.4)
- Term Vectors/MLT Query: support for different analyzers than default at field (#7801, master and 1.5)
- Scripting: Fix NPE in ScriptService when script file with no extension is deleted (#7689, master, 1.5 and 1.4)
- Snapshot/Restore: Make sure indices cannot be renamed into restored aliases (#7918, master and 1.5)
- Search: add
formatsupport for date range filter and queries (#7821, master and 1.5)
- Discovery: remove
MasterFaultDetection.Listener.notListedOnMaster(#7995, master, 1.5 and 1.4)
- Core: Perform write consistency just before writing on the primary shard (7873#, master, 1.5 and 1.4)
- Snapshot/Restore: make it possible to delete snapshots with missing metadata file (#7980, master, 1.5, 1.4 and 1.3)
- PluginManager: Do not overwrite existing config files (#7890, master, 1.5 and 1.4)
- Added new Upgrade API to upgrade lucene indices to current version (#7884, master, 1.5 and 1.4)
So happy to be tweeting #kibana 4 screenshots. #elasticsearch #aggregations are awesome pic.twitter.com/D5eUUHz1jK
— Rashid Khan (@rashidkpc) October 6, 2014
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
- Big beta week at Elasticsearch for releases last week. We released the first beta of Elasticsearch 1.4 as well as Kibana 4 beta. We are eager to get your feedback, so please tell us what you think.
- Elasticsearch for Apache Hadoop also got two freshly baked releases, namely es-hadoop 2.1 beta2 and 2.0.2, see the blog post for more information about our brand new support for SparkSQL and Apache Storm.
- We kicked off the first in our tripartite series on Elasticsearch QA last week. Check out Isabel Drost-Fromm on our continuous integration environment. (And stay tuned to this blog tomorrow for more testing awesomeness.)
- Karel Minařík authored an in-depth tutorial on using Nginx as a proxy for Elasticsearch.
- Patrick Peschlow wrote up a nice introduction to implementing transaction like functionality using Elasticsearch
- Prasoon Kumar shared a great write up of the first ever Elasticsearch Bangalore Meetup, including some great tips on using Elasticsearch in production.
- In or around Bangalore? You may be interested to read more on our first Bangalore meetup and more upcoming events from SpringPeople, our first partner firm in India.
- Our partner firm, Exensio, shared an introduction to using Elasticsearch to gain competitive intelligence. (auf Deutsch)
- It's good to be loved! JDN named Elasticsearch and Kibana two of seven technologies you have to know before the end of 2014. Our favorite quote: "Kibana: A essayer d'urgence!" (en français)
Happy to announce that @elasticsearch for Apache Hadoop 2.1 is now certified on Cloudera 5, including support for Apache Spark!
— Cloudera Connect (@ClouderaConnect) October 2, 2014
Slides & Videos
This week, we're bringing you a whole whack of Elasticsearch love from last week's DrupalCon Amsterdam! Not a Drupalista? No problem - these videos are great for all lovers of things PHP and folks interested in learning how to scale the ELK stack for centralized logging and monitoring.
Nikolay Ignatov and Welin Welchev on Elasticsearch as the new school search engine for the Drupal Content Management System
Steven Merrill on using a fully open source stack for logging and monitoring of Drupal.org (Think billions of logs.)
Larry Garfield and Hagen Last on using Elasticsearch as a key component for building a decoupled Content Management System
Standing room only at last week's DrupalCon Amsterdam presentation by Nikolay Ignatov and Welin Welchev
Where to find Us
We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Director of Developer Relations, Leslie Hawthorn, know!
We're a proud sponsor of the 5th anniversary of DevOps Days Belgium, held in its original hometown of Ghent. Elasticsearch will have a booth at the conference, so stop by and say hello to Leslie Hawthorn and Alan Hardy between sessions. We'll be at DevOps Days October 27-28th.
Zeng Yong, aka Medcl, is one of our most active community members in China. He has two upcoming talks on all things Elasticsearch:
- Explore Elasticsearch Tutorial on October 9, 2014. (online course)
- Real-time data analysis at QCon Shanghai on October 17, 2014.
Medcl is also busy convening the 3rd annual Elasticsearch Users Conference in China for October 25th. You can find out more information about the event and register on meetup.com.
- David Pilato will be heading to BDX.io to tell you all about how to Make Sense of your Big Data. BDX.io takes place on October 17th in Bordeaux.
- David will run a hands on Elasticsearch and Kibana workshop at Forum PHP Paris 2014. You can join David on October 23rd from 9:30 AM to 1:30 PM, though separate registration is required. Forum PHP Paris runs October 23-24th at the Beffroi Montrouge.
The code.talks conference is back in Hamburg this year on October 9th and 10th, though with a new name. (We had a great time there last year when this conference was known simply as Developer Conference EU.) Alexander Reelsen will be speaking once again, along with Isabel Drost-Fromm.
Here's all the Elasticsearch and ELK stack information on offer at code.talks:
- Patrick Peschlow from our partner firm codecentric AG will walk you through Elasticsearch: Lessons Learned. (Thursday, Oct. 9th at 10 AM)
- Isabel will tell you all about how to properly architect search for your website. (Friday, Oct. 10th at 10 AM)
- Alexander will give you an introduction to the ELK stack, including using it to build real-time analytics dashboards. (Friday, Oct. 10th at 2:00 PM)
The week after code.talks, Leslie Hawthorn will be heading LinuxCon Europe in Dusseldorf, where she'll speak on the panel Empowering Your Corporate Open Source Software Developers. The panel takes place on Wednesday, October 15th at 2:30 PM, and the conference runs October 13-15th at Congress Centre Dusseldorf.
And, if you happen to be in Karlsruhe on October 16th, you can catch Patrick Peschlow on Elasticsearch Performance in a Nutshell at the Search Meetup Karlsruhe. Registration is now open, and the meetup will begin at 7:15 PM.
The Hyderabad Scalability Meetup group is hosting a getting started with Elasticsearch hackathon on October 18th. Register now to save your place, looks like space is limited!
- Honza Kral will be speaking at PyCon Poland 2014. The conference runs from October 16-19th in Orle Gniazdo. We'll be back to you with more details on Honza's presentation soon!
- Honza will speak at PolyConf 2014 on Elasticsearch's language clients, Bridging the gap: one API, 5 languages. Honza takes the stage at 5:30 PM on October 30th, and the conference runs the 30th-31st in Poznan.
Honza Kral will be speaking at the Øredev conference in Malmö on Analyzing & Search Data Using Elastisearch. The conference runs Nov. 4-7th, and Honza takes the stage on Nov. 6th at 5:40 PM.
David Pilato will return to Soft-Shake in Geneva once again this year. He will speak on Advanced Search for Your Legacy Application in the Big Data Track. Soft-Shake takes place on October 23-24th, and David will speak in Slot 5 on Thursday.
The Leeds DevOps meetup group will convene on Oct. 14th for their one year anniversary meeting! Along with the celebration, you can also enjoy a use case talk from the fine folks at LateRooms on how they use the ELK stack to grapple with millions of lines of log data. You can still save your seat!
- New meetup: The gnocode user group in New Orleans will get together tomorrow, Thursday Oct. 9th, to talk logging with the ELK stack. There are still a few seats left, so register now!
- New meetup: The Elasticsearch Minneapolis Meetup will host their second meeting on Oct. 22nd. Sign up now to learn all about how TrackJS & Infinite Campus use Elasticsearch. Register now to save your place.
- New meetup: The Elasticsearch Phoenix Meetup has just kicked off, and we have a special treat for the first meeting: Meet the Kibana team! You can register now to save your place; the meetup will take place on October 22nd.
- New meetup: The Elasticsearch Seattle Meetup group will be hosting their inaugural meeting on October 23rd. First up: Kibana - deployment, configuration, use and extending. You can register now to save your place.
- New meetup: Kurt Hurtado will speak at the San Francisco Cloud Mafia meetup on November 5th, where he'll talk the ELK stack in a DevOps environment. This meetup is incredibly popular, so get registered now if you'd like to attend.
- We'll be in the exhibits hall at Strata and Hadoop World NYC on October 13-15th. But don't just pop by our booth (P1) - we've also scheduled a special Strata NYC Meetup on Oct. 15th at Twitter's New York offices. Amongst other great talks on offer, you can hear from Costin Leau, lead developer for Elasticsearch for Apache Hadoop. He'll tell you all about our new support for SparkSQL and Apache Storm. Get registered for the meetup now, because this one always fills lightning-fast. (And many thanks to the fine folks at Twitter for hosting us)
- Leslie Hawthorn will be speaking at the All Things Open Conference on What Does Big Data Really Mean for Your Business? All Things Open takes place October 22-23 in Raleigh, North Carolina, and Leslie will speak at 2:15 PM on Wednesday. (You might enjoy this sneak preview of her talk, too.)
Where to Find You
Our Director of Developer Relations, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- Bangalore - October 8, 2014 (core Elasticsearch training)
- Bangalore - October 10, 2014 (Getting started workshop)
- Amsterdam - October 14, 2014 (core Elasticsearch training)
- New York - October 23, 2014 (Getting started workshop)
- London - October 30, 2014 (Getting started workshop)
- San Francisco - November 05, 2014 (Getting started workshop)
- San Francisco - November 06, 2014 (Core Elasticsearch Training)
- Berlin - November 11, 2014 (Core Elasticsearch Training)
- Reston - November 13, 2014 (Core Elasticsearch Training)
- Paris - November 17, 2014 (Getting started workshop)
- Bangalore - November 17, 2014 (Getting started workshop)
- Paris - November 18, 2014 (Core Elasticsearch Training)
- Bangalore - November 18, 2014 (Core Elasticsearch Training)
- London - November 26, 2014 (Core Elasticsearch Training)