This Week in Elasticsearch - June 11, 2014
@Elasticsearch: Achieving the Analytics Dream http://t.co/eBMINGGFQp @harryweller @chetanp #opensource #bigdata pic.twitter.com/8OKvfItJOs
— NEA (@NEAVC) June 5, 2014
Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
- Core: Use new
ConcurrentHashMapimplementation coming in Java 8 (#6400, master and 1.3)
- Scripting: exposed
_typefields as stored fields (
_fieldsnotation) (#6406, master, 1.3 and 1.2)
- FileSystem: Use
XNativeFSLockFactoryinstead of the buggy Lucene 4.8.1 version (#6424, master, 1.3 and 1.2)
- Compilation: Don't warn about using Sun proprietary APIs (#6423, master and 1.3)
- PUT Index Template API: unified PUT/POST behaviour in relation to create parameter (#6429, master and 1.3)
- Aggregations: Optimization for memory usage. Added changes to core Aggregator class to support a new mode of deferred collection (#6128, master and 1.3)
- Bulk API: A request which fails to create multiple indices may never return
(#6436, master and 1.3)
auto_expand_replicasexplodes with a value of
all(#5752, master and 1.3)
- Aggregations: Change the
top_hitsto be a metric aggregation instead of a bucket aggregation (which can't have an sub aggs) (#6395, master and 1.3)
EmptyOrdinals#getMaxOrd()should return 0 instead of 1, since ordinals are zero based (#5871, master and 1.3)
- Aggregations: Cardinality aggregation fails in case of empty doc values field (#6413, master, 1.3 and 1.2)
- TransportClient: Improve logging, fix minor issue (#6376, master and 1.3)
- Stats: Bugfixes and enhancements to indices stats API (#6390, master and 1.3)
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
- Our big news last week was all about how we're able to bring you the ELK stack: Elasticsearch Inc secured $70 million in our latest funding round. You'll want to check out this post from Shay Banon about how we'll use the funding and our future as an open source company. You may also be interested in learning why NEA Ventures decided to invest in our company.
- If you were impacted by the routing bug discovered in Elasticsearch 1.2 last week, check out the full details on the tool we released to correct the issue. If you never used Elasticsearch version 1.2, nothing to see here, move along.
- We released Elasticsearch Marvel 1.2.1, a bug fix release to fix a data shipping issue with the Shard Allocation dashboard.
- The fine folks at Wikimedia have released a brand new experimental highlighter plugin.
- If you're in the security game, this recent analysis of Security Information and Event Management (SIEM) using the ELK stack and other technologies is made for you!
- Ryan Buzzell shared an excellent tutorial on getting the ELK stack up and running. If you're new to the ELK stack, Ryan's article is a great getting started guide, including step-by-step instructions.
- Joachim Baran published a deep dive into using Elasticsearch with RethinkDB.
- Ricardo Martins shared a post on ingesting data into the ELK stack. (Em Português Brasileiro)
Elasticsearch raised $104M in 2.5 yrs. That's a lot in open source software. http://t.co/Rk6Iponv7P . pic.twitter.com/VcVt7Teoto
— Derrick Harris (@derrickharris) June 5, 2014
Slides & Videos
Costin Leau's slides from last week's Hadoop Summit. Video Coming Soon!
Clinton Gormley at Berlin Buzzwords 2014: Elasticsearch's Query Domain Specific Language (DSL) - Not Just for Wizards!
Talking about how we use Scala and @elasticsearch at @mysports195 #thoughworksnyc pic.twitter.com/ieDXFXBZLD
— Skot Mahr (@lazyvalue) June 4, 2014
Martijn van Groningen on Elasticsearch's Percolator at Berlin Buzzwords 2014
Robert Muir on Apache Lucene 4 from Berlin Buzzwords 2014
.@kurtado at @GeekdomSF talking about ELK in a DevOps environment for tonight's @elasticsearch San Francisco meetup. pic.twitter.com/MlYCjbiLLb
— elasticsearch (@elasticsearch) June 11, 2014
Where to find Us
We'd love to feature all the great Elasticsearch, Logstash, and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Community Manager, Leslie Hawthorn, know!
The IGT Cloud Meetup Group is hosting a workshop on Applied Cloud Computing with Google Cloud Platform at Google Tel Aviv. Topics will include BigQuery with Logstash as Application Log Analysis Platform. The workshop runs from 9:30-13:30 on July 7th.
We've got not one but two awesome conferences going on in Amsterdam the week of June 16th:
- DevOps Days Amsterdam: Join us June 19-21st for an Elasticsearch workshop, a little on DevOps for Happier Humans and all the news you can use at the Elasticsearch table. Swing by to say hello to Leslie Hawthorn and Kurt Hurtado! Lee Hinman will also join us for the workshop. Yay Lee!
- GOTO Amsterdam: Our CTO, Shay Banon, will treat the crowd to a deep dive on Elasticsearch's APIs, plus we'll have lots of knowledge and swag on offer at our booth. See you there!
Plus, we're hosting Friday drinks on June 27th at 6:00 PM in Amsterdam. You'll have the chance to meet tons of our core developers, as the whole company will be visiting our EU HQ that week for our annual all hands gathering.
- David Pilato will take the stage at DevoxxUK to discuss Advanced Search for Your Legacy Application DevoxxUK takes place June 12-13th in London, and David will speak on June 12th at 4 PM in Room 1.
- Elasticsearch will be out in force at London Technology Week. Join us June 17-19th to celebrate all that is fashionable in the UK tech scene, especially at the Networking Bar. Even cooler, on June 17th, Mark Harwood will be telling you all about how to Make sense of your (big) data and analyze in real-time like you have never done before! Mark's talk is at 4 PM. Last but not least, we'll have a London meetup on June 18th starting at 5 PM.
- Aaron Mildenstein will be speaking at Texas Linux Fest this weekend, presenting an Introduction to Logstash. TLF runs June 12-13th in Austin, and Aaron will present at 10:15 AM on Saturday.
- The Philadelphia Elasticsearch Meetup group will be getting together for their second meetup on June 26th. Details are still in the works, but sign up now so you know where to go for ELK treats on the 26th.
- The SYN/(st)ACK NYC User Group will convene on June 30th at 6:30 PM. Squarespace is hosting the festivities, and Franklin Angulo, one of their Software Engineers, will be presenting. Franklin will cover building a scalable monitoring and alerting system using Graphite, Grafana, Collectd, Nagios, Logstash, Elasticsearch, and Kibana, among others.
Where to Find You
Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticearch, Kibana, and Logstash than where you rest your head.
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- Paris - July 08, 2014 (core Elasticsearch training)
- Cologne - July 09, 2014 (core Elasticsearch training)
- Seoul - July 10, 2014 (core Elasticsearch training)
- London - July 14, 2014 (core Elasticsearch training)
- Tokyo - July 14, 2014 (core Elasticsearch training)
- Berlin - July 15, 2014 (ELK workshop)
- Milan - July 16, 2014 (core Elasticsearch training)