Elastic{ON} 2018

From the technical talent behind some of the on-screen hacks on USA Network’s Mr. Robot, this talk covers how to improve incident response by combining technologies like Elasticsearch with distributed, on-endpoint analysis for comprehensive, high-speed and efficient visibility at any scale.

Data structures that enable geospatial indexing and search, field mapping strategies, geo aggregations for spatial data analysis — this talk has everything you need to know about the latest geo capabilities in Elasticsearch and Apache Lucene.

Thinking about building an end-to-end security analytics platform with the Elastic Stack? This talk explores how to do it with a homegrown solution that’s fast and scalable, so you can increase team impact by having more data faster, and gain back time for threat hunting versus responding to alerts on the fly.

Built on top of Lucene 7.0, the latest version of Elasticsearch includes faster restarts with sequence IDs, support for sparse data, and index sorting, with many new features still to come. Learn where the Elasticsearch roadmap is headed with new features and prepare for what's next.

Learn how Elastic played an integral role in the successful launch of the popular video game Overwatch. See an overview of Blizzard’s big data pipeline and dive into the details of how the Overwatch team leverages the Elastic Stack platform to ensure the game is constantly running at peak performance.

GDIT/McQueen Solutions needed to collect, store, and analyze petabytes of machine data at a large US government agency. They chose the Elastic Stack, and today the GDIT Elastic cluster entails a multi-petabyte, highly available cluster that improves search speeds from hours to seconds.

See the architecture and design principles of Citi’s application infrastructure monitoring solution and hear how they addressed cloud monitoring challenges. Plus, learn how Citi leverages advanced data search capabilities for monitoring data management, addressing both operations and business requirements.

Learn how and why Fermilab transitioned from an aging MySQL-based system to the Elastic Stack for collecting, searching, and analyzing data on the state and usage of their distributed computing system, connecting thousands of researchers with hundreds of computing resources around the world.

The scope of Lucene has expanded to drive Elasticsearch improvements, most notably with doc values to make it a competitive engine for running analytics and multi-dimensional points, and to better handle structured data. Hear about how Lucene 7 features impact Elasticsearch, plus what to expect in Lucene 8.

Kibana currently ships with CSV export improvements, a new UI for alerting, and the Kibana Query Language. Preview new features on the way, including additional accessibility improvements, even more UIs, and a streamlined approach to deploying solutions for use cases like logging and metrics.

Kyruus is using Elasticsearch features to develop solutions that help U.S. Spanish speakers search for and find medical providers. Learn the differences between Spanish and English text searches and see how Kyruus adds medical jargon search capabilities to both languages to build more effective tools.

See how the Elasticsearch team uses formal methods in the design of distributed algorithms. Get an inside look at their specification methodology, toolset, and experiences applying it to the data replication and cluster consensus algorithms using the TLA+ toolbox and the Isabelle/HOL theorem prover system.

What started with one lightweight method for shipping network data has expanded to a family of Beats for files, metrics, Windows event logs, uptime data, and audit data. The Beats ecosystem is growing — a lot. Get the grand tour of recent Beats developments and work-in-progress features.

Starting with an index of 20 million product documents and an eye toward scaling, the IEEE GlobalSpec team leveraged built-in Elasticsearch functionalities to replace their legacy product search and solve their unique challenges. Learn why they chose Elasticsearch and see the topology of their cluster.

The Credit Suisse team is leveraging cutting-edge natural language processing (NLP) techniques to surface potential malicious activity by reducing noise and false positives. See how they're using a deep learning solution in the eCommunications surveillance space to provide emotional and sentiment context.

OTTO Motors design, manufacture, and operate self-driving vehicles to help make factory and warehouse environments safer, more efficient, and agile. In this presentation, the OTTO Motors team share some of the real-world benefits of using Elastic to power their global Internet of Things (IoT) starter stack.

In 2014, the engineering team at Nativo had zero visibility into system performance and inadequate monitoring in place. No longer. Today, the Elastic Stack is a mission-critical system at Nativo. In this talk, hear how the team and their implementation of the Elastic Stack evolved over the past three years.

Credit Suisse needs to process and store more than 30 million rows of data per month, and make that data accessible to end-users. Today, they're doing all that with the Elastic Stack. Learn how the team extracts data from various sources, indexes it in Elasticsearch, and visualizes it all using Kibana.

One year ago, Vijay Samuel and the monitoring team at eBay embarked on a journey to collect logs and metrics from any application deployed in Kubernetes. And thus Collectbeat was born. Hear about the team's journey, current path, and how eBay is planning to tackle even more complex problems in the future.

Spring is a digital shopping destination that helps customers quickly find the products they love. Learn how the Spring team builds a product catalog using Elasticsearch, how they ensure product data is synced, and how the Spring team has improved their index to better understand their users’ intent.

Much has happened since 5.0. Persistent queues, pipeline viewer (x-ray vision, basically) and the ability to run multiple pipelines at the same time for different use cases, and a move to the latest version of JRuby. In this session, learn a bit more about the Logstash roadmap and what to expect next.

In 2014, the Workday team realized their Lucene-based search would not scale or provide relevance for end users. After research and prototyping, Elasticsearch was the solution. In this demo, see how Elasticsearch cut indexing time from over ten hour to minutes for their most critical use case: Recruiting.

Filebeat is a single-purpose data shipper designed to forward events from any text file containing log messages to Elasticsearch. But what if you're using an unsupported log format? In this talk, learn how to tailor existing modules to meet your logging needs and how to build a new module using Filebeat.

Opbeat joined Elastic to build an application performance monitoring solution unlike any on the market. Open source and easy to use, this Elasticsearch-based APM solution is ready to take Elastic deployments to the next level. Join the Elastic APM team to learn more about what the product has to offer.

Benchmarking is a tricky business. What is your workload and how do you model it correctly? Which hardware do you choose? How do you avoid accidentally creating bottlenecks? In this talk, we discuss common "gotchas" in benchmarking and provide tips and tricks to help you evaluate performance correctly.

The CBC shares massive amounts of radio, television, and news content every day. With the Elastic Stack, they're gaining new insights into how that content is performing. In this demo, learn how the CBC began with Elastic Cloud, and how they use the Elastic Stack for content performance analysis today.

Wondering how to justifying spending money on open source technology to upper management? In this session, we'll answer common questions about the value of open source tools, and walk you through the various ways you can convince your boss that a commercial relationship with Elastic is money well spent.

You may have used the Elastic Stack to derive insights from your log data, but did you know it's also a wonderful data analytics platform? In this session, see the data analytics capabilities of the Elastic Stack in action, and learn how the Kibana dashboard can make meaningful visualizations of your data.

Take a journey through the evolution of consensus within Elasticsearch. You'll learn about the overarching consensus strategy, why we chose to work on high throughput data replication first, and learn how (and why) the Elasticsearch team improved things iteratively, while still shipping software.

Fastenal is the largest distributor of fasteners in North America with nearly 500,000 products on their various websites. See an in-depth demo of their product search, learn how Fastenal builds their indexes nightly, and discover how their IT team uses the Elastic Stack to build a better customer experience.

Get an in-depth look at the site search capabilities of Swiftype, powered by Elasticsearch. Site search lets users quickly create and manage search experiences tailored to their specific needs. See a walkthrough of the easy-to-use UI for fine-tuning results, indexing content, adjusting relevance, and more.

Identifying, debugging, and fixing Node.js production performance issues is difficult. Learn how Node.js instrumentation and Elastic APM can help. This talks covers how to minimize instrumentation overhead, add metadata to collected metrics, and how to use custom watchers for codebase instrumentation.

How many Tweets do we send a day? Who are our top blog authors? How frequently do we publish content with specific tags over a given period of time? Hear how the mere mortals of the marketing team derive value from the Elastic Stack by creating and exploring visualizations in Kibana.

Canvas was born out of a desire to create pixel-perfect, infographic-inspired, living visualizations inside of Kibana that get viewers to do a double-take and then look closer. This demo-driven session goes from concept to creation with the Canvas expression language and UI.

Fitbit grew from 35,000 to 265,000 logs per second, increased data storage from 5 to 30 days, and upgraded from Elasticsearch 1.5.x to 5.5.x. Learn how they scaled a log aggregation pipeline to process 21 billion messages per day using the entire Elastic Stack, from log queuing to field mappings.

Deploy, configure, and gain immediate insights on your Azure infrastructure using the built-in monitoring capabilities of Azure Monitor alongside Elasticsearch, Kibana, and Logstash. It's a rich, extensible Kibana monitoring experience for analyzing overall service health and auditing infra changes.

Cox, the third-largest cable company in the US, transitioned their video delivery to internet protocol (IP), utilizing a content delivery platform that they built in-house. This is their story of how they came to aggregate, index, and analyze over 2.5 billion daily log entries — all using the Elastic Stack.

Here's what’s coming in Elastic Cloud and Elastic Cloud Enterprise, like support for mixed clusters and how to mix and match hardware for hot and warm data, machine learning, and more in a single cluster. Also, learn about our planned Kubernetes integration, security features, and more.

Stormfish uses the Elastic Stack to support the development of several MxR environments using big data, including several for the Army Research Lab. Hear how they use the Elastic Stack to ingest data from disparate sources, run real-time analytics, and export the insights to an operational MxR environment.

Argos, a leading UK digital retailer, migrated to an Elasticsearch-powered microservice-based infrastructure to improve search relevancy, performance, and customer satisfaction. Hear their lessons learned and how they made it through Black Friday 2016 with zero downtime, and how it all unfolded in 2017.

Elastic can help you collect and store operational data to simplify log collection and metrics aggregation within Kubernetes and Docker deployments. Use features in Beats, Elasticsearch, and Kibana to make managing containerized, scheduled, and otherwise automated environments a great experience.

From recent developments with TLS everywhere, to what's on the horizon with encrypted settings and authentication protocols such as SAML and oAuth, the security world of the Elastic Stack is robust. Learn about existing Elastic Stack security features and plan ahead for future developments.

This is the story of how we built an Elastic upgrade experience that spans products, versions, and users. Hear about the surprises we experienced along the journey that impacted release dates, the empathy and relationships needed for our success, and see what to expect in the 7.0 upgrade.

See how the Elastic logging service drives SAP Concur's logging, alerting, monitoring, and analytics strategy. Then, learn how SAP centrally collects all application and system logs, and how their site reliability teams use machine learning and alerting features to ensure peak performance, 24/7.

Explore how the Elastic Stack can be used for a comprehensive operational analytics deployment, including infrastructure and application logging and metrics, as well as deep APM transaction analysis. See examples from users and customers, as well as Elastic's own deployments successfully doing this.

With nearly 55 million requests per day to their website, Rightmove is the UK’s most visited property portal. Hear how they use Elasticsearch geo capabilities to improve relevancy, how percolation allows them to alert users to properties of interest, and how they monitor 17 TB of logs on an ongoing basis.

Machine learning for the Elastic Stack helps you understand the behavior of your data. Learn about single- and multi-metric anomaly detection jobs, automated anomaly notifications that simplify tasks like pre-configuring NGINX log analysis, time series forecasting, and automatic log data categorization.

Discover what drove Grubhub to convert to Elasticsearch and how they tailored their stack to improve delivery — for both engineers and diners. Learn about the importance of ephemeral clusters in deployment, integrating with Eureka, using Impression Engine to improve search, and other best practices.

A distributed system, built by a distributed team, in a company committed to distributed work. This session covers Elastic the company: who we are, where we are, and how we work. What do we care about and what are the aspirational statements we make to chart our course into the future?

There are many things to consider and many tools and techniques at your disposal when you begin running and managing the Elastic Stack in production. This talk highlights management features in Kibana and covers the most important elements for running and managing the Elastic Stack in production.

We're adding machine learning functionality to our products in the form of shrink-wrapped applications. What started as time series anomaly detection has grown to forecasting what the time series might do. Learn some of the modeling techniques we've used and the key ways we've addressed these requirements.

Learn about the architecture of OnCommand, a sophisticated IT infrastructure management tool built on Elasticsearch. Topics include the decisions behind replacing Cassandra with Elasticsearch, optimizing for an embedded use-case, alerting on dynamic thresholds using X-Pack machine learning, and more.

In Elastic Cloud, we've migrated from a polyglot logging solution to one based entirely on the Elastic Stack. Hear members of the Cloud SRE team talk about making the switch, architectural and implementation concerns, care, and feeding, as well as lessons learned.

JPL systems engineers are responsible for the system design across the lifecycle of a flight project, and capturing the complexity of this decision-making process is a difficult task. See the software approach that allows system engineers to document, query and perform analysis on highly structured data.

Elasticsearch gives you so many different options and configurations to support a variety of use cases. In this talk, members of the Elastic support team discuss the top three common customer mistakes and misconfigurations along with best practices and recommendations.

New Elastic Cloud capabilities allow you to match your provisioned hardware to your use cases, making it easier to efficiently run Elasticsearch for multiple workloads such as hot/warm architectures, dedicated master nodes, and machine learning and enable Elastic Stack features like APM, Logstash, and more.

Java has always had a special place in our hearts, but we've never had a great Java client for Elasticsearch. It is time to finish the High-Level REST client and remove its dependency on Elasticsearch. Learn how we’re ushering in a new era with the Elasticsearch Java client and what to expect next.

GoDaddy uses the Elastic Stack and other technologies to perform real-time anomaly detection, log analysis, and auto-remediation on 10+ TB of new data daily. See how they transitioned from a farm of data silos to a centralized platform that ingests, analyzes, and visualizes data throughout the enterprise.

Elasticsearch SQL made its debut at last year's conference. From converting SQL statements into Elasticsearch queries to a console experience for exploring data using "SELECT x FROM y WHERE z > 1000" type queries, there’s a lot on the horizon. Get the full rundown on where we are one year later.

For seven months, Lyft was Amazon's biggest hosted Elasticsearch customer and most frequent support headache. After two weeks of migration, Lyft became self-hosted and both companies breathed a sigh of relief. Hear their story, plus what they've learned from almost a year of operating their own cluster.

See how the RBC leverages a powerful event-driven microservices architecture in order to evolve from a systems-of-record way of working to systems-of-engagement. Learn how they utilized Elastic Cloud Enterprise to accomplish mainframe offload through data analytics and drive tremendous cost savings.

In 2017 the Elastic community zoomed past 120K members across GitHub, Meetups, forums and more. In this talk, members of the Elastic Developer Relations team share current metrics around the Elastic community, resources we provide to support your endeavors in open source, and our future plans and goals.

The Refugee Datathon started in 2015, when thousands of refugees were welcomed in Munich. Hear how Suny Kim and Maxi Richt used the Elastic Stack and the power of open data to help activists get reliable, recent data on refugee movement, and make an impact on the pro-refugee movement.

Dimagi uses Elasticsearch to help power key workflows in fighting India's tuberculosis epidemic. Hear how they generalized this solution in their platform to deal with migration and drug adherence of HIV/AIDS patients in South Africa, Lesotho, Swaziland, Uganda, and Cameroon.

To aid those in need, Libraries Without Borders (LWB) developed the Ideas Box, a portable multimedia toolkit that's easy to transport, has minimal energy needs, and supplies its own power. Learn how LWB uses the Elastic Stack to improve digital content, and to develop search for offline digital libraries.

Thorn's Innovation Lab builds technology to defend children from sexual abuse. Focusing on both domestic child sex trafficking and child sexual abuse material, they are a powerful force that works with tech companies, law enforcement, and governments. Learn why Thorn has used Elasticsearch from the start.