This Week in Elasticsearch - December 17, 2014
Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
We are taking a break for the holidays, but we'll be back with the next edition of This Week in Elasticsearch on January 7.
Elasticsearch core
- Core: Drop support for state written pre 0.90 (#8850, master)
- Engine: Fix updates dynamic settings in
InternalEngineHolder
(#8885, master) - Geo: Fix for NPE enclosed in
SearchParseException
for a <code>geo_shape filter or query
( #8432, master and 1.4) - Core: Add
best_compression
option for Lucene 5.0 (#8863, master) - Mappings: Fix Get field mapping api with pretty flag (#8806, master and 1.x)
- Engine: Remove engine related command classes (#8900, master and 1.x)
- Scripting:
ScriptService
can deadlock entire nodes if script index is recovering (#8901, master, 1.x, 1.4 and 1.3) - HTTP: Add
http.publish_port
setting to the HTTP module (#8807, master and 1.x) - Packaging: Add java7/8 java-package paths to debian init script (#8815, master and 1.x)
- Gateway: Cleanup
LocalGatewayShardsState
(#8852, master) - Cleanup: Remove some
Internal*
abstractions (#8904, master and 1.x) - Core: don't block shard stats when phase 3 of recovery is running (#8910, master, 1.x, 1.4 and 1.3)
- Dependencies: Remove circular dependency between IndicesService and IndicesStore
( #8918, master and 1.x) - Core: Remove explicit
.cleanUp()
on cache clear (#8924, master and 1.x) - Core: use compound file by default for merged segments < 10% of index size(#8919, master and 1.x)
- Search: close active contexts on
SearchService#close()
(#8940, master and 1.x) - Cat API: Default all APIs to verbose (#8927, master)
- Discovery: concurrent node failures can cause unneeded cluster state publishing (#8933, master, 1.x and 1.4)
- Core: Remove
IndexEngine
abstraction (#8955, master) - Cleanup: Remove Gateway abstraction (#8954, master)
- Core: Wire utf-8 encoding, so unicode filenames work (#8847, master, 1.x and 1.4)
- Settings: Remove
mapping.date.round_ceil
setting for date math parsing (#8889, master) - Shutdown: Shutdown
IndicesService
last (#8940, master and 1.x)
Looks like my North American friends are all hanging out together. I wonder what mischief is afoot
#LogAllTheThings pic.twitter.com/MSN8MxrIJD
— logstash (@logstash)
December 10, 2014
Elasticsearch Ecosystem
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
- We just released Marvel 1.3.0. Get it while it's hot!
- Elasticsearch 1.4.2 and 1.3.7 have been released, featuring several bugfixes and performance improvements. Check out the release blog post!
- The first beta release of Logstash 1.5 has shipped. Check out all the new features and read more about all changes to the plugins system. (And if you're looking for something fun to do over the next few weeks, you might enjoy this handy how to on creating a Logstash plugin.)
- But wait, we're not done yet! Kibana 4 Beta3 has been released and is waiting for you to try it out. Now with support for filters and scripting!
- Check out this post about Hive, an open source project from the New York Times. Hive is a crowdsourcing framework that - you guessed it! - uses Elasticsearch as part of its underlying architecture.
- ClusterHQ's new project, Eliot, is a logging system for complex distributed systems. See how Eliot uses the ELK stack to deep-dive into their logs.
- Florian Kopf continues his blog series on use cases for Elasticsearch with a post on Use Cases for Elasticsearch: Analytics.
- Thomas explains how to centralize your logs with the ELK stack in this post.
- Hayssam Saleh explains performance tuning for Elasticsearch. (en français)
- Jorge De La Cruz brings us a post on log centralization with Logstash and Kibana. Check it out! Zimbra: Logs centralizados con Logstash y Kibana sobre Ubuntu 14.04. (en español)
Slides & Videos
Colin Goodheart-Smithe on DIY Aggregations for Elasticsearch
From the December Search Meetup Munich:
Alexander Reelsen provides an introduction to Elasticsearch and explores the gotchas of Distributed Systems
From the December London Elasticsearch Meetup: The Guardian on how they use Elasticsearch for 100s of searches per second
How Logmatic.io uses Elasticsearch in their SaaS log analysis tool (en français)
Where to find Us
We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Director of Developer Relations, Leslie Hawthorn, know!
Germany
For folks in or around Bonn on December 18, the Bonn ELK Meetup group will be holding their inaugural meetup. Register now to save your place.
Israel
The first-ever ELK meetup in Tel Aviv-Yafo will be happening on Tuesday, January 13, and they're kicking off with a great topic: an overview of the Kibana 4 Beta and real-life use cases. Get registered today!
New Zealand
Leslie Hawthorn and Mark Walkolm will both be speaking at linux.conf.au (also known as LCA) in Auckland. This fantastic event runs January 12 - 16, and is definitely worth attending!
And accompanying LCA, we'll be having our first-ever meetup in Auckland on January 12. The schedule is still being finalized, but register now to save your spot.
Spain
Our own Pere Urbón-Bayes will be joining the Barcelona Meetup on January 8 to share his presentation, Make sense of your data with Elasticsearch. Register now to save your place.
The Netherlands
Come to the CRI Service Kennissessie in Houten on January 15 to learn about enterprise logging with Elasticsearch, rsyslog, Docker, and Kibana. Save your place by registering for the meetup today.
United States
The Codemash conference in Sandusky, Ohio, will have Itamar Syn-Hershko speaking on making distributed search and analytics on big data easy with Elasticsearch. Register today to join this session on January 8!
@mola_io and @gmunozsoria just ready for @elasticsearch #meetup (flume, ES, Kibana) pic.twitter.com/dPyFFMlPla
— Stratio (@StratioBD)
December 15, 2014
Where to Find You
PSST! If you're a regular reader of This Week in Elasticsearch, a.k.a TWIES, you're thinking of skipping this section. You may even be thinking to yourself, yes of course I will drop a note on Twitter when I am giving a talk on all things ELK. That's awesome, because we'd like to showcase every meetup, conference presentation and workshop on Elasticsearch, Logstash, and Kibana happening worldwide. And now, we've made it even easier for you to get support for your meetup!
Head on over to our meetups page! (And we'll still totally send you swag if you're giving a talk on anything ELKy at a conference.)
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticsearch, Kibana, and Logstash than where you rest your head.
Trainings
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- London - December 18, 2014 (Elasticsearch Core Training)
- Cologne - January 13, 2015 (Getting Started Workshop)
- Berlin - January 15, 2015 (Core Elasticsearch Training)
- Chicago - January 15, 2015 (Core Elasticsearch Training)
- Bangalore - January 19, 2015 (Getting Started Workshop)
- Bangalore - January 20, 2015 (Core Elasticsearch Training)
- Reston - January 21, 2015 (Core Elasticsearch Training)
- Seattle - January 21, 2015 (Core Elasticsearch Training)
- Paris - January 21, 2015 (Getting Started Workshop)
- Paris - January 22, 2015 (Core Elasticsearch Training)
- London - January 29, 2015 (Core Elasticsearch Training)