Elastic Fantastic: Excelian's review of Elastic{ON} Tour London

Elastic{ON}Tour was a sell-out Elastic user conference held in London on November 3, 2015. It represents the largest gathering of Elasticsearch, Logstash and Kibana expertise all in one room. This year I was delighted to be asked to present at the show and, in particular, the results of a recent successful grid project which uses Elasticsearch functionality. 

In this particular project we were tasked with creating a Grid reporting framework for one of the large investment banks. The project used Elasticsearch, Logstash, and Kibana to store and provide analytics on grid performance, capacity, logs, and so forth. It was a very successful project for Excelian and had fantastic feedback from the client. I believe one of the reasons for the success of this project was the close relationships that developed between Excelian and some of the Elastic engineers. They were able to optimise the setup but were also on hand to troubleshoot any issues that arose. When I conveyed the feedback to Elastic they were delighted and that is when they invited me to present at  Elastic{ON} Tour.

Elastic{ON}Tour was a fantastic and genuinely inspiring event. From a personal perspective, I managed to meet up with various users within the Elasticsearch community and discovered many interesting use cases for Elasticsearch. I already knew that many large organisations such as Netflix, Facebook, Microsoft, and Cisco are using Elasticsearch for building out their search capabilities within the organisation. However, it was also really interesting to learn about some of the more unusual use cases. I wasn’t previously aware that NASA uses Elasticsearch to store and analyse data sent back from the Mars Curiosity Rover, or how the Victoria and Albert Museum plans to use Elasticsearch to analyse visitor data to determine how they could optimise the placement of their artifacts within the museum.

The event was very well structured. The morning of the conference was focused on the Elastic roadmap and introducing the new features of Elasticsearch 2.0, Kibana 4.2, Beats, Watcher, and Found. A nice little insight into the creation of Elasticsearch was revealed when Shay Banon (the creator of ElasticSearch) began his keynote speech by explaining how he created Elasticsearch in order to create a tool for his wife to search for cooking recipes!

The later part of the day was more focused on real life user stories and how they use Elasticsearch within their organisation. Graham Tackley from The Guardian gave us a walkthrough of the tools they have built with the Elasticsearch stack to do real-time media analytics that can process up to 40 million documents a day. There were also two presentations from Goldman Sachs on how they utilise Elasticsearch to build a Goldman Sachs search engine and also a firm-wide task list.

My particular presentation was focused on how we decided on the use of Elasticsearch, some of the reporting requirements from the user and details on the architecture we built out. There were only a handful of people in the audience who were from the financial services industry but I did manage to get some very interesting questions during the talk and even during the drinks reception. It was really good to see how the technology was able to be transferred across multiple industries, and how we can all learn from each other.

After the presentations, it was time for the more informal part of the conference and everyone adjourned for beer and food. It was during this time that I got the chance to talk to Shay Banon and discussed some interesting trends within financial services which are perfect candidates for Elasticsearch, e.g. fraud detection, market sentiment analysis, and suchlike. I also had an opportunity to talk to Elastic's co-founder Uri Boness on the challenges of deploying Elasticsearch in the financial services industry and some upcoming features in the roadmap that would help with this, i.e. data centre replication.

Elastic{ON} Tour London 2015 was an enlightening experience for me. I discovered so many novel use cases for Elasticsearch and I had a chance to talk to the people who built those solutions. I was also very excited to be able to share my experience in building out an Enterprise Grid reporting solution based on Elasticsearch.

This definitely will not be my last Elastic conference.

You can watch my talk here but if you have any further questions please do contact me directly at  jay.chin@excelian.com. For more information on the upcoming Elastic{ON} in San Francisco, please click here.

Jay is currently a Principal Consultant for Excelian where he specialises in helping companies make the most out of their Grid and HPC systems. Prior to Excelian, Jay was Vice President at Bank of America Merrill Lynch where he managed various EMEA infrastructure teams for Grid and distributed systems. While not busy hacking away at Python code, Jay likes to spend his time fly-fishing and reading science fiction.

The original blog post has been published a few days ago here.