Powering Uber Marketplace’s Real-Time Data Needs with Elasticsearch
Elasticsearch plays a key role in Uber’s Marketplace Dynamics core data system, aggregating business metrics to control critical marketplace behaviors like dynamic (surge) pricing, supply positioning, and assess overall marketplace diagnostics – all in real time.
In this talk, Jae and Isaac will share how Uber uses Elasticsearch to support multiple use cases at the company, handling more than 1,000 QPS at peak. They will not only address why they ultimately chose Elasticsearch, but will also delve into key technical challenges they’re solving, such as how to model Uber’s marketplace data to express aggregated metrics efficiently, and how to run multiple layers of Elasticsearch clusters depending on criticality, among others.
Jae Hyeon Bae
Jae Hyeon Bae is highly experienced on overall data engineering areas, especially real-time data processing. He is the technical lead on data systems serving for Uber Marketplace dynamics. He has in-depth knowledge on data pipeline and analytics engine such as Elasticsearch or Druid. Prior to Uber, he built Netflix real-time data pipeline and initiated successful analytics projects.
As a software engineer on Uber's Marketplace Data stack, Isaac builds systems that provide temporal-spatial data at Uber. Isaac focuses on the query and storage layers, contributing to the hexagonal spatial grid system used throughout Uber's Marketplace.