Elastic{ON} 2017

IBM at the Elastic{ON} 2017 Opening Keynote

IBM discusses the Bluemix platform and how it enables logging with the Elastic Stack. Take your interconnected, cloud-hosted micro services and bring them together quickly and simply so you spend less time instrumenting your app for logging applications and more time enhancing it.

What's Evolving in Elasticsearch

Elasticsearch team and tech leads give an overview of the changes already released in 5.x series, and a taste of the new features coming in 6.0.

MZ Games: The Evolution of Log Insight

Machine Zone is the company behind massively-multiplayer online games Game of War, Fire Age, and Mobile Strike. Real-time operations is essential for a great user experience. The MZ engineering team shares how they use the Elastic Stack to track player activity to support customer service and more.

Walmart: Near Real Time Analytics

Did you know that Walmart sells about 156 bananas per second? Thanks to the Elastic Stack, they do. What started as an engineering hack to prove the banana myth is now a fully-operational self-service analytics platform for Walmart’s data scientists.

NVIDIA's User Experience Streaming Analytics: Data Intelligence with the Elastic Stack at Scale

NVIDIA collects metrics for every frame streamed, every second spent on GeForce NOW, its streaming video game platform, using the Elastic Stack – enabling business, engineering, operations, and quality assurance teams to assess quality and debug issues in real time.

Ryft: How to Leverage Heterogeneous Compute to Extend and Accelerate Elasticsearch

Search-based analytics are a critical function for any organization. Ryft leverages FPGA/x86 heterogeneous compute technology to eliminate indexing and transformation, in addition to providing enhanced Elasticsearch functionality that accelerates workflows, increases the speed of search and analysis, and enhances wildcard searches.

What's Cookin' in Kibana

Kibana team and tech leads cover the latest news in Kibana — heatmaps, log context, pipeline aggregations, CSV export, and more. They look back at what's happened since 5.0 and forward to what's coming in future releases.

Cisco's Journey to Cloud Native

Cisco's Commerce Platform powers product configuration, pricing, quoting, export compliance, credit checks, and order booking across all product lines. Learn how they've implemented a 180+ node Elasticsearch deployment to improve customer experience and business agility.

General Mills: Using Elasticsearch for Application Search

General Mills replaced their Endeca-based search and service-oriented architecture with Elasticsearch on core website properties such as Pillsbury.com, BettyCrocker.com, and more. Their use cases continue to expand as they move away from a relational database model to a flexible, document-based model.

What's Brewing in Beats?

Beats are a family of lightweight shippers that send data from edge machines to Elasticsearch. Beats creators Monica and Tudor walk through the latest, including Filebeat modules, which simplify the collection and parsing of common log files down to a single command, and Heartbeat, for uptime monitoring.

NERSC Data Collect: Using Elasticsearch as a Long-Term Data Storage Engine

The NERSC data collect system provides access to 30 TB of logs and time-series data generated by the supercomputers at Berkeley Lab. Explore how NERSC uses Elasticsearch as a large, long-term data storage engine, including index allocation tagging, use of index aliases, and more.

On Distributed Systems and Distributed Teams

The distributed systems movement, and open source more broadly, is fueled by solving a series of complex problems: consensus, leader election, failure semantics, among others. How does a distributed team building distributed systems, at Elastic, function?

Streamlining Healthcare & Research at UCLA with Elasticsearch

As healthcare institutions generate more data, they need a way to search through electronic health records (EHR) and find meaningful insights. UCLA Health has chosen Elasticsearch as its tool of choice to index, search, and produce more thorough, actionable results for clinicians and researchers.

XPO Logistics: How to Become a Superhero with Elasticsearch

Learn how a team of two with a vision for the future grew into a group of more than 30 engineers operating as the de-facto Elasticsearch Center of Excellence at XPO Logistics.

Second Annual Elastic{ON} Women's Breakfast

We assembled a panel of executives and engineers to talk about their experiences in the software and technology space. They discussed approaches to growing their careers, navigated key challenges and difficult decisions, and how they have promoted diversity and equality in tech.

What's the Latest in Logstash?

Logstash team and tech leads address questions like: Can we go faster? What is the persistent queue? How do I monitor Logstash? What is the future of Logstash configuration? How does the team keep systadmins and DevOps in mind when working on the product? And what's in store for the Logstash UI?

Get the Lay of the Lucene Land

In spite of being close to 20 years old, the Lucene project keeps innovating. Hear stories of the latest features in Lucene 6, how they impacted Elasticsearch, and what to expect in Lucene 7.

Microservices, Continuous Delivery, and Elasticsearch at Capital One

The Continuous Delivery and Microservices principles in Capital One's cybersecurity data platform ingests 6 TB of data every day, and Elasticsearch is a core component. Learn how their microservices foundation helps them build a reusable, shareable, and portable platform supporting Elasticsearch.

The Nature Conservancy, the Elastic Stack, and Security Logs

IBM: Localizing Kibana for the Global Language Landscape

IBM and their customers identified multiple languages localization support as a key capability needed in all Kibana visualizations and dashboards. Learn about the key aspects of Kibana globalization from design to capability delivered, as well as some of the design journey they took to get there.

What's X-citing in X-Pack?

Adding security, alerting, monitoring, and graph exploration capabilities to the Elastic Stack with X-Pack has never been easier. Over the last year, X-Pack usability has improved, including a better getting started experience and several UI features. The engineering team tells all.

The Dell.com Search Story: Products, Support, Commerce, and Relevancy

Dell.com shares how they transformed their search platform from a high-touch, targeted experience into a single Elasticsearch-powered responsive and multi-lingual experience. They also share how Elasticsearch collects and analyzes user events, such as click tracking and overall user interaction, and more.

Tinder: Using the Elastic Stack to Make Connections Around the World

Tinder relies on the Elastic Stack to analyze, visualize, and predict not only which people a user will swipe right on, or which people will swipe right on that user, but also when there's a mutual swipe match. Hear how the service is growing into a global platform for social discovery in many facets of life.

How Blackboard Curbs Cheating with Elasticsearch

Blackboard uses Elasticsearch to power SafeAssign, its plagiarism detection software. Learn about their internal web search engine, how they transitioned from a Solr deployment to a multi-terabyte, full-text search engine powered by Elasticsearch, and their experiences running their infrastructure on AWS.

Machine Learning in the Elastic Stack

Learn how to apply machine learning features to the Elastic Stack and what business problems they will help you solve. These new capabilities let you answer new questions like "are users exfiltrating data unusually?" and "is my website response time atypical?"

Terradue: Advancing Earth Science with Elasticsearch

Terradue develops and operates large cloud solutions for processing Earth observation satellite data, and uses Elasticsearch to empower users to expound it. Learn how they use the Elasticsearch .NET client to tackle challenges such as geohazards for rapid response monitoring of volcanoes or earthquakes...

Blizzard: Building a Near Real-Time Data Pipeline

Learn how Blizzard Entertainment — makers of Overwatch and World of Warcraft — leverages Elasticsearch, Kibana, Logstash, Kafka, tribes, and Node.js to generate actionable value from gamer and server events.

Strengthen Your SIEM: Using Logstash to Connect ArcSight to the Elastic Stack

As of Logstash 5.1.1, you can easily connect any device that supports the CEF data format as a codec to the Elastic Stack via files, Kafka, or syslog, allowing you to extend and complement your existing ArcSight deployment with the Elastic Stack.

Browse Raw Logs in One Place: Open Source Plug-in for Kibana

The ability to effectively browse, scroll, and sift through raw log files is critical for IT teams and developers to identify errors, perform root cause analysis, and troubleshoot. Learn about a new open source Kibana plugin from Search Technologies that enables this.

What's Next for Elastic Cloud

Exciting developments are coming for those who want to deploy and manage the Elastic Stack with the click of a button on AWS — or choose to run it the way they want, in the environment they want.

Barclays: Using Elasticsearch, Kibana, and Logstash for Cyber Security

Barclays utilizes Elasticsearch in key data analytics initiatives to enable cyber security and cyber defense.

eBay: Elasticsearch as a Service

With countless business-critical text search and analytics use cases that utilize Elasticsearch, eBay has created a custom 'Elasticsearch as Service' platform. Learn about sizing, provisioning, configuring, maintaining, auto-scaling, and decommissioning states for every Elasticsearch cluster.

Correlating Metrics and Logs

Metrics and logs are meant to be together. Why do we insist on keeping them apart? Learn about our mission to reunite them, in the process deriving powerful operational insights using brand-new Kibana visualizations and machine learning techniques.

Elastic Cloud Inside Out

The Elastic Cloud team runs thousands of clusters and is growing rapidly, while maintaining solid SLAs and allowing users to scale, upgrade, and reliably monitor their clusters. Ever wonder how it works?

Elasticsearch SQL

SQL for Elasticsearch is coming. Learn how this feature converts a SQL statement into an Elasticsearch query, executes a SQL query and return the results in tabular form, provides a console to explore data, and more.

Consensus and Replication in Elasticsearch

Distributed systems rely on consensus algorithms. Choosing among Paxos and its variants determines the underlying system's performance and fault-tolerance. Learn about the mechanics of quorum-based consensus algorithms and tradeoffs compared to the primary-backup approach.

Small, Medium, or Large: Evolve Your Elastic Stack to Fit

The few thousands operations per second indexing requirement is now few millions; a single use case has grown to multiple; one user group has become tens of user groups. Learn about Elastic Stack topologies to meet evolving use case(s), and deployment patterns for projects of various sizes.

IBM Bluemix Platform Logging with the Elastic Stack

IBM Bluemix platform logs to the Elastic Stack. Learn about the flexibility of their logging solutions in microservice environments. See Docker and Cloud Foundry logging — no user configuration necessary. Runtime logs are easily added. Additionally, see an example logging from a Watson service.

IT as the Transmission of the Sprint Business Engine

The IT department at Sprint ingests nearly 3 billion records a day, including data from logs, databases, emails, syslogs, test messages, and internal vendor application APIs into the Elastic Stack. This insight empowers teams across marketing, retail operations, wholesale sales, and more.

Machine Learning and Statistical Methods for Time Series Analysis

Dive into the new machine learning technologies available in the Elastic Stack and how to apply them. Explore the unsupervised machine learning techniques Elastic uses, and the challenges and constraints that exist in order to provide operationally useful insight when applying them to time series data.

Elasticsearch, Kibana, and IoT: Capturing a ballerina's movement in real time.

How do you take lessons from ballet and performance into distributed systems, and how do you take lessons from distributed systems into biometric and kinetic sensing wearables?

Why Contain Yourself? Official Elastic Stack for Docker

The Elastic Docker team discusses running the Elastic Stack in container environments using the official images they maintain and publish.

Monitoring Docker with Metricbeat

The ability to monitor Docker environments is critical. Metricbeat offers two different possibilities to do so with cgroups and the Docker API. Learn about the differences and similarities between the two approaches.

Timelion: Magic, Math, and Everything in the Middle

Timelion is a simple expression-based pluggable time series interface for everything. Learn about Timelion's expression syntax including data sources, chaining, and grouping, and then apply those concepts, along with a few neat tricks, to some real data.

Workday: Building a Metrics Pipeline with the Elasticsearch, Kibana, and Logstash

Learn how Workday Search expanded their Elastic Stack deployment by implementing a robust, easy-to-use metrics processing pipeline for over 1 billion logs.

Walgreens: Using Elasticsearch, Kibana, Logstash, and X-Pack for Log and Business Analytics

Walgreens is creating a search platform for its website powered by Elasticsearch and is expanding to use X-Pack for log management and analysis. Learn about their migration from Endeca, and their technical and architectural lessons and successes.

Elastic at Datadog

Storing metrics and events in Postgres wasn’t enough for Datadog, a SaaS-based infrastructure monitoring company. Learn how and why they moved to Elasticsearch to create a fast and efficient environment for thousands of customers.

Slack: Using Elasticsearch for Security Analytics

Monitoring malicious activity and handling the resulting alerts is vital to the success of a defensive security program. Slack talks tools everyone should consider to monitor their infrastructure, including Elasticsearch, and how to create a reliable logging pipeline to handle data from thousands of hosts.

Ship Your Own Data: Tailoring Beats to Your Use Case

Get introduced to extending Beats, the platform for lightweight data shippers to Elasticsearch and Logstash, and learn how to extend it to deal with your particular use case.

Customer Success @ Elastic: Elastic Stack + Salesforce = <3

The folks behind the Elastic curtain shows how they use the Elastic Stack to enhance their internal technology stack. They show you from start to finish how they query and expose license data, enhance the support engineers’ experiences in their console, and how to bring it all together in Salesforce.

Merck: Using Elasticsearch to Leverage Human Genetic Data for Drug Discovery

Elasticsearch enabled Merck to harmonize a data ingestion pipeline and create a universal coordination system for genetic variants as a backbone to help scientists uncover new insights on human genetics across a spectrum of diseases and aid in the discovery and validation of new therapies.

Elasticsearch Search Improvements

Improvements are coming to Elasticsearch including range fields, removing the _all field, a unified highlighter, and the synonym graph filter.

Powering Uber Marketplace’s Real-Time Data Needs with Elasticsearch

Elasticsearch plays a key role in Uber's Marketplace Dynamics core data system, aggregating business metrics to control critical marketplace behaviors like dynamic (surge) pricing, supply positioning, and assess overall marketplace diagnostics — all in real time. Learn how they do it.

Elastic Cloud @ Fandango: How They Shifted Deployment Model to Scale & Meet Their Deadlines

In order to understand their outbound marketing and campaigns, Fandango deployed the Elastic Stack to monitor and analyze over 5 billion web logs monthly. In one weekend, the FandangoNOW team redesigned and re-architected their on-premise deployment onto Elastic Cloud to analyze 500 million records per day.

Getting Your Elasticsearch Data Graph-Ready

Knowing what sort of data makes sense to put in Graph and how to prepare it is often a challenge for new users. This session walks through examples of how to model your Elasticsearch data in order to start exploring the interesting connections it contains using this X‑Pack feature.

Elasticsearch for Hadoop, Spark, Streaming, and More

Get introduced to the basics of ES-Hadoop's native Spark Integration, touch upon the other features that the connector brings to the table (including native integrations with Hive, Storm, Pig, Cascading, and MapReduce), have a look at the internals to see how it works, and see what's to come.

Automatic Alerts to Monitor Your Elasticsearch Cluster

When monitoring met alerting, the average troubleshooting time went down and the average sleep time went up. True story. X-Pack brings both features together to enable built-in cluster alerts. Learn about the latest in monitoring and management and how to solve real-world problems using monitoring data.

Kibana Visualizations Deep Dive

Get a detailed walkthrough of Tagcloud and Heatmap (new visualizations in Kibana 5.2), see what's coming with future geospatial visualizations, and learn about dedicated UIs for time series visualizations in Timelion and a new visual builder for pipeline aggregations.

Writing Logstash Plugins in the 5.X Era

Learn how to write plugins for Logstash and what goodies the 5.X version line will bring to the plugin developer. We cover the basics, as well as how to write a Java-based plugin, how to instrument your plugin with metrics, and more!

Elastic{ON} 2017 Closing Keynote: Cause Award Honorees and Q&A with Elastic Founders

Elastic believes technology enables us to progress toward a better future. Inspired by the people applying Elastic software this way, Elastic recognized three projects using the Elastic Stack to advance the greater good, improve the human condition, and help the planet.