Closing Keynote: Cause Award Honorees and Open Q & A with Elastic Founders

At Elastic, we passionately believe technology enables us to progress toward a better future, and are very inspired by the way people are applying our software in this way.

This year we launched our inaugural Elastic Cause Awards, which will recognize three projects that are using the Elastic Stack to advance the greater good, improve the human condition, and help the planet.

Come hear the stories of this year’s honorees. We know you’ll leave feeling just as humbled and fortunate as we do to be part of such a special community.

Watch Now

Writing Logstash Plugins in the 5.X Era

Learn how to write plugins for Logstash and what goodies the 5.X version line will bring to the plugin developer. João will cover the basics, as well as how to write a Java-based plugin, how to instrument your plugin with metrics, and more!

Watch Now

Kibana Visualizations Deep Dive

Have you noticed Kibana has been looking mighty fine lately?

Attend this session to dive deeper into Kibana’s latest visualizations. You’ll get a detailed walkthrough of Tagcloud and Heatmap, new visualizations in Kibana 5.2, as well as insight into where we’re taking visualizations next. From a roadmap perspective, we’ll focus in particular on new geospatial visualizations we are working to bring out in 5.x. As a developer, you’ll also get a behind-the-scenes perspective on the evolving world of visualizations and how it may affect your custom visualization plugins. Finally, we’ll discuss dedicated UIs for time-series visualizations, from Timelion to a new visual builder for pipeline aggregations.

Watch Now

What's the Scoop on ES-Hadoop? Spark, Streaming & More

Elasticsearch is an industry-leading solution for search and real-time analytics at scale. Apache Spark has shaped into a powerhouse for processing massive data, both in batch and streaming contexts. Elasticsearch for Apache Hadoop (ES-Hadoop) is a two-way connector that provides the tools needed to marry these two together in perfect data harmony.

This talk aims to introduce the audience to the basics of ES-Hadoop’s native Spark Integration, touch upon the other features that the connector brings to the table (including native integrations with Hive, Storm, Pig, Cascading, and MapReduce), shed some light on the internals of how it works, as well as highlight what’s to come.

Watch Now

The Usual Suspects: Automatic Alerts to Monitor your Cluster

When monitoring met alerting, the average time spent to troubleshoot went down and the average sleep time went up. True story.

X-Pack, which made its first debut with the 5.0 release of the Elastic Stack, brings monitoring and alerting features together to enable built-in cluster alerts. Chris and Bohyun will go over the latest in monitoring and management in the first portion of the talk, then Antonio will talk about how to solve real-world problems using monitoring data based on customer scenarios he's helped with as part of the Elastic support team.

Watch Now

Getting Your Data Graph-Ready

Knowing what sort of data makes sense to put in Graph and how to prepare it is often a challenge for new users. This session will walk through examples of how to model your data in order to start exploring the interesting connections it contains. Learn about models for “wisdom of crowd” style applications and configurations to support “forensic” style investigations.

Watch Now

Elastic Cloud @ Fandango: How They Shifted Deployment Model to Scale & Meet Their Deadlines

Every month, more than 60 million people visit Fandango’s website to browse movie tickets as well as rent or buy TV and movie content. In order to best understand the effectiveness of their outbound marketing and offer campaigns, Fandango deployed the Elastic Stack to monitor and analyze over 5 billion web logs monthly.

In this talk, Adam will walk you through how, in one weekend, the team at FandangoNOW redesigned and re-architected their prior on-premise deployment onto Elastic Cloud in order to hit their launch date. He’ll cover their lessons learned and the journey scaling up to analyzing up to 500 million records per day.

Watch Now

Elasticsearch Search Improvements

Let's talk about search improvements coming soon to an Elasticsearch near you!

Range Fields:
Want to create a global television guide to find broadcasts airing during certain time periods? Thanks to recent advancements in Lucene this desire is now a reality.

Removing the _all field:
The _all field can be either a boon or a burden. Come hear about why the _all field is going away and what it's being replaced with!

Unified Highlighter:
Starting in 5.3, a fourth highlighter called `unified` is available in Elasticsearch.

This highlighter has landed from Lucene with a goal in mind: he wants to rule them all ! We’ll see how and why this highlighter can advantageously replace your highlighter of choice.

The Synonym Graph Filter:
Multi-term synonyms have long been buggy in Lucene and Elasticsearch, but this issue is now fixed thanks to the addition of the new synonym_graph token filter, along with support for graph token streams in query parsers.

Watch Now

Powering Uber Marketplace’s Real-Time Data Needs with Elasticsearch

Elasticsearch plays a key role in Uber’s Marketplace Dynamics core data system, aggregating business metrics to control critical marketplace behaviors like dynamic (surge) pricing, supply positioning, and assess overall marketplace diagnostics – all in real time.

In this talk, Jae and Isaac will share how Uber uses Elasticsearch to support multiple use cases at the company, handling more than 1,000 QPS at peak. They will not only address why they ultimately chose Elasticsearch, but will also delve into key technical challenges they’re solving, such as how to model Uber’s marketplace data to express aggregated metrics efficiently, and how to run multiple layers of Elasticsearch clusters depending on criticality, among others.

Watch Now

Integrating Human Genetic Data to Help Drive Drug Discovery: Elastic @ Merck

As genome sequencing’s costs have dramatically fallen, scientists have been awash in genetic data for novel research – but the existing tools and methods for analysis were not scaling well in terms of data size and harmonization, and they are also tedious, manual, and require a significant amount of expert integration.

Daniel and Bhasker will share Merck’s journey with Elasticsearch, which has enabled them to harmonize a data ingestion pipeline and create a universal coordinate system for genetic variants as a backbone to help scientists uncover new insights on human genetics across a broad spectrum of diseases (from cancers, alzheimer’s, diabetes), and to aid in the discovery and validation of new therapies.

Watch Now

Ship Your Own Data: Tailoring Beats to Your Use Case

Beats is a little bit like LEGO: You can use each Beat itself as a building block to cover your needs, but at the same time each Beat consists of different reusable and extendable elements. This makes it possible for developers and operators to combine and extend Beats in different ways.

In this talk, Nicolas and Steffen will introduce developers, operators, and Beats users to the internals of the Elastic Beats. The knowledge gained will help with making informed decisions on how to extend Beats to deal with your particular use case if it’s not fully-covered by the existing Beats features.

Watch Now

Security @ Slack

Monitoring for malicious activity and handling the resulting alerts is vital to the success of a defensive security program. Powerful, centralized logging is available to all of us, but it is only useful if we understand and take action on the data collected.

This talk will discuss tools everyone should consider using to monitor their infrastructure, including Elasticsearch, and the process by which users can create a reliable logging pipeline to handle data from thousands of hosts. Ryan and Nate will demonstrate how to scale these efforts by integrating security into a communication platform that helps users look at more data by delegating event management to the affected individuals directly.

Watch Now

Elastic at Datadog

Datadog is a SaaS-based infrastructure monitoring company that processes billions of data points every day, including metrics (CPU utilization, database keys, and queue lengths) and events (completed Chef job notifications, GitHub commits, and Docker container status). Storing this information and being able to make use of it in their Stream and Dashboards is challenging. They started with Postgres, but as their needs grew, they moved to Elasticsearch, which is now a core component of their infrastructure, indexing vast numbers of events every second.

In this talk, you will see how Datadog uses an Elasticsearch cluster to create a fast and efficient environment for thousands of customers.

Watch Now

Under the Surface of Optum's Security Big Data Lake

Optum’s Cyber Defense organization utilizes the Elastic Stack within its Security Big Data Lake (SDBL) to search and pivot between cyber threats. The Hadoop and Elastic architecture of the data lake allows correlation and enrichment of logs prior to Elastic ingestion, accelerating investigation timelines. The SBDL can replace and improve on many cyber products offered by third parties at significantly lower cost and risk.

Watch Now

Walgreens' Journey To Creating an End-to-End Search Platform

Walgreens is on a journey towards creating an end-to-end search platform for its website powered by Elasticsearch. In this talk, Syed will walk through the various steps along the way, including a look at why their old Endeca-based search platform was no longer viable, how and why Elasticsearch became their new engine, and the technical and architectural lessons and successes they learned along the way.

You'll also hear how Walgreens continues to expand its use of Elasticsearch to the Elastic Stack and X-Pack to power log management and analysis, including the use of Kibana to visualize technical and business metrics.

Watch Now

How Workday Search Built their Metrics Pipeline with the Elastic Stack

After struggling to find a traditional database that could ingest large volumes of application metrics at an acceptable rate, Workday Search noticed that each of their already existing Elastic Stack deployments were able to process over 1 billion log events a week without issues. This talk will share how Workday Search expanded their deployment by implementing a robust, easy-to-use metrics processing pipeline.

Bo and Thomas will provide details on how they architected their pipeline, the scripts and frontend tools they use to visualize their data and proactively alert them to issues in production, as well as the metrics they look at to provide insight into usage patterns and facilitate intelligent product decisions.

Watch Now

Customer Success @ Elastic: Elastic Stack + Salesforce = <3

Come and see the folks behind the Elastic curtain, and we'll show you how Elastic uses the Elastic Stack to enhance our internal technology stack.

A.J., Kristina, and Pius will discuss Elastic's Customer Success programs, showing you start to finish how they query and expose license data, how they use the Elastic Stack to enhance the Support Engineers’ experience in their support console, and how they bring all that information together in Salesforce to provide a predictive and holistic system for customer health.

Watch Now

How Warner Bros. is Using Elastic to Solve Entertainment and Media Problems at Scale

Warner Bros. processes billions of records each day globally between its web assets, digital content distribution, OTT streaming services, online and mobile games, technical operations, anti-piracy programs, social media, and retail point of sale transactions. Despite having large MPP clusters, a significant amount of dark data remained trapped in Web Logs. In this presentation, we will discuss how Warner Bros. and Decision Lab leveraged the new Elastic Stack 5.x coupled with Apache Spark, to deliver scalable insights and new capabilities to support business needs.

Watch Now

Timelion: Magic, Math, and Everything in the Middle

Timelion is a simple expression-based pluggable time series interface for everything. Whether you're brand new to Timelion, or have been using it since day 0, you'll learn something new in this session. Rashid will go over Timelion's expression syntax including data sources, chaining, and grouping, and then apply those concepts, along with a few neat tricks, to some real data.

He'll also cover multiple manners of munging data and get into the methods Timelion uses to automatically fit abnormal sources, allowing you to compare and combine sparse and incomplete datasets. Finally, we'll take a brief look at plugins and how you can extend Timelion to do so much more.

Watch Now

Monitoring Docker with Metricbeat

Over the past couple years, Docker environments have become more and more common. It is critical to monitor these environments. Metricbeat offers two different possibilities to monitor Docker containers: cgroups and Docker API.

In this talk, Andrew will discuss the differences and similarities between the two approaches with the pros and cons of each. He will also give a live demo to show how to set up Metricbeat for Docker monitoring and get insights.

Watch Now