The Math Behind Elastic Machine Learning
Machine learning functionality is being added to our products in the form of shrink-wrapped applications. We began by focusing on time series anomaly detection, which required the ability to predict what the time series might do next. It wasn't such a large step to go from this to forecasting what the time series might do over the next day, week, or month. This capability was added in 6.2.
To automate this process, an ML system needs expressive modeling, which can adapt to different data characteristics, relearn its parameters (or select a new model as things change), seamlessly deal with abnormal periods to minimize their impact on the modeling, and much more. Dig into the details of some of the modeling techniques we’ve used for these features and some of the key ways we have addressed these requirements.
Hendrik Muhs is a software engineer at Elastic. He is a member of the machine learning team. Before joining Elastic he spent more than a decade as backend engineer in the search space hacking on e.g., NLP, ranking, scale, and autocompletion.
Tom Veasey joined Elastic in September 2016. He is a member of the machine learning team. He started out as a data scientist working on satellite, radar, and drug discovery projects, and had detours into EDA and FX derivatives pricing. He has a Masters in Physics from the University of Cambridge.