Elastic Fantastic: Excelian's review of Elastic{ON} Tour London
Elastic{ON}Tour was a sell-out Elastic user conference held in London on November 3, 2015. It represents the largest gathering of Elasticsearch, Logstash and Kibana expertise all in one room. This year I was delighted to be asked to present at the show and, in particular, the results of a recent successful grid project which uses Elasticsearch functionality.
In this particular project we were tasked
with creating a
Grid reporting framework for one of the large investment banks.
The project used Elasticsearch, Logstash, and Kibana to store and provide analytics on grid performance, capacity, logs, and so
forth. It was a very successful project for Excelian and had fantastic feedback
from the client. I believe one of the reasons for the success of this project
was the close relationships that developed between Excelian and some of the
Elastic engineers. They were able to optimise the setup but were also on hand
to troubleshoot any issues that arose. When I conveyed the feedback to Elastic
they were delighted and that is when they invited me to present at
Elastic{ON} Tour.
Elastic{ON}Tour was a fantastic and genuinely inspiring event.
From a personal perspective, I managed to meet up with various users within the
Elasticsearch community and discovered many interesting use cases for
Elasticsearch. I already knew that many large organisations such as Netflix,
Facebook, Microsoft, and Cisco are using Elasticsearch for building out their
search capabilities within the organisation. However, it was also really
interesting to learn about some of the more unusual use cases. I wasn’t
previously aware that NASA uses Elasticsearch to store and analyse data sent
back from the Mars Curiosity Rover, or how the Victoria and Albert Museum plans
to use Elasticsearch to analyse visitor data to determine how they could
optimise the placement of their artifacts within the museum.
The event was very well structured. The morning of the
conference was focused on the Elastic roadmap and introducing the new features
of Elasticsearch 2.0, Kibana 4.2, Beats, Watcher, and Found. A nice little
insight into the creation of Elasticsearch was revealed when Shay Banon (the
creator of ElasticSearch) began his keynote speech by explaining how he created
Elasticsearch in order to create a tool for his wife to search for cooking
recipes!
The later part of the day was more focused on real life user
stories and how they use Elasticsearch within their organisation. Graham
Tackley from The Guardian gave us a walkthrough of the tools they have built
with the Elasticsearch stack to do real-time media analytics that can process
up to 40 million documents a day. There were also two presentations from
Goldman Sachs on how they utilise Elasticsearch to build a Goldman Sachs search
engine and also a firm-wide task list.
My particular presentation was focused on how we decided on the use of Elasticsearch, some of the reporting requirements from the user and details on the architecture we built out. There were only a handful of people in the audience who were from the financial services industry but I did manage to get some very interesting questions during the talk and even during the drinks reception. It was really good to see how the technology was able to be transferred across multiple industries, and how we can all learn from each other.
After the presentations, it was time for the more informal part
of the conference and everyone adjourned for beer and food. It was during this
time that I got the chance to talk to Shay Banon and discussed some interesting
trends within financial services which are perfect candidates for
Elasticsearch, e.g. fraud detection, market sentiment analysis, and suchlike. I
also had an opportunity to talk to Elastic's co-founder Uri Boness on the
challenges of deploying Elasticsearch in the financial services industry and
some upcoming features in the roadmap that would help with this, i.e. data
centre replication.
Elastic{ON} Tour London 2015 was an enlightening experience for me. I
discovered so many novel use cases for Elasticsearch and I had a chance to talk
to the people who built those solutions. I was also very excited to be able to
share my experience in building out an Enterprise Grid reporting solution based
on Elasticsearch.
This definitely will not be my last Elastic conference.
You can watch my talk here but if you have any further questions please do contact me directly at jay.chin@excelian.com. For more information on the upcoming Elastic{ON} in San Francisco, please click here.
Jay is currently a Principal Consultant for Excelian where he specialises in helping companies make the most out of their Grid and HPC systems. Prior to Excelian, Jay was Vice President at Bank of America Merrill Lynch where he managed various EMEA infrastructure teams for Grid and distributed systems. While not busy hacking away at Python code, Jay likes to spend his time fly-fishing and reading science fiction.
The original blog post has been published a few days ago here.