How Elasticsearch Made Us Faster – Literally
Oliver Eilhard is a programming addict working at Meplato GmbH on e-procurement solutions. At a recent Munich Search Meetup, he presented on Elastic, an Elasticsearch client for the Go programming language. He also presented a rather unique example of using Elasticsearch: for auto racing.
At Meplato, our work focuses on e-procurement. In a nutshell, e-procurement is online shopping for employees on behalf of their company, plus integrating this purchasing process with various business systems. Of course, we're using Elasticsearch extensively in this setup, e.g. when searching for products, but also for analytics.
But some of us at Meplato have another passion: racing. For many years, we've toured race tracks and attended various track days and trainings. So, finally, some of us decided to get our official licenses and take part in a professional race series, the VLN season 2014.
The VLN is a series of 10 races over 4-6 hours, all taking place at the Nürburgring-Nordschleife, a 21 km race track in Germany. At the VLN, you typically see about 200 cars on the track, ranging from Porsche 911s to Ferrari 458s and Opel OPCs, all racing at the same time and at (very) different speeds. Obviously, overtaking is not the exception, but the rule here.
So what does this all have to do with Elasticsearch?
Let me explain. In our first races, we found that we were slower than our competitors in similar cars. So what do good engineers do? First, they gather data. Second, they analyze it. And that's exactly what we did, using Elasticsearch.
What we obviously needed was data about location and speed. While you can get a ton of data from a race car (worth a separate article), we found that getting data from only our car didn't solve the problem. After all, we only have data from one car, and we need data from the competitors as well. So we came up with a different solution.
Virtually every team in the VLN uses an iOS application called Racing App to track the location of their car; remember, it's a 21 km track. I've even heard that the Race Control Center uses it. (And kudos to the team at GPSoverIP for such an awesome app!) The app tracks the exact location of every car on the track and streams it over the internet. While GPS-over-IP is not as exact as GPS data from the cars themselves, it's available for every car and, as it turns out, good enough for us to solve our problem.
So, we wrote a proxy to record data on the wire and stored it in Elasticsearch. Looking at the data, we found that not only is the latitude and longitude of every car provided, but also its speed, steering wheel position, and some other metrics. Nice.
Next, we hacked up a program to analyze the recorded data. We used GeoJSON to prepare a list of sectors for the track.
Then, we used the geo-queries and geo-filters of Elasticsearch to find entry and exit times for each car in every sector. Given enough laps, that gave us a good estimate of where we lost time. We even tracked time loss down to individual corners of the track. Then, we generated a nice PDF of the results.
One evening after the race, we figured that with all this data at hand, we could just replay the whole race. So, some of us sat down and, with the help of D3.js, came up with this:
While we extracted the data for this replay, we could have easily done this in real-time with Elasticsearch.
All in all, we had a lot of fun at the Nürburgring. Not only did we learn a lot about racing and cars but also about how technology, and Elasticsearch in particular, can help us to solve problems. And we finally made it to the podium two times.
If you liked this post, you may enjoy Oliver’s slide deck from his Search Meetup Munich presentation.