21 September 2017 User Stories

Using Elastic Machine Learning to Automate Complex Data Analysis at Sunhotels

By Juan Cidade

An Interview with Juan Cidade, Head of IT Ops, Sunhotels

How has the exponential growth of incoming data affected the way you do business?

As an online business providing travel services to European businesses, Sunhotels has always been data-driven at its core. Consequently, we have always embraced open source technologies - particularly Elasticsearch - to help us gain insight from our customer data and improve our services as a result. Now, as part of Australian travel group Webjet, we’re focusing heavily on the use of next generation data technologies to drive ongoing innovation and growth.

Recently, the volume of data we handle has grown exponentially. We process search and booking requests for thousands of travel agencies and B2B suppliers across Europe. This includes enquiries from tens of thousands of travel agencies, and a huge swathe from third party aggregators. In two years, the average number of requests per second has grown from 600 to over 4000. That’s 250 million requests and thousands of bookings per day, originating from a host of different sources. We needed a platform that could process this kind of volume of data at speed and scale but, importantly, one that would reveal meaningful insight.

How have Elastic technologies helped you manage the data revolution?

Elasticsearch has been a part of the business for many years. Originally, we wanted to have some idea about search response times, and to find a way to differentiate between kinds of searches and their outcomes. Soon, we wanted to capture different strands of metadata from each search; helping us to understand more about each interaction, and be more responsive to issues. We started logging around 15 different fields across each search and booking; response time, destination, customer type, product contract (third party or direct etc).

When putting all the raw data into Elasticsearch, we not only see response times across the board at a granular level, but we can analyse and understand all the different factors that might affect availability in one hotel: low availability, seasonality, price sensitivity. Using this information, we work with individual hotels, destinations, providers or clients to suggest solutions; change pricing, improve contracting/mapping/availability, connecting/disconnecting providers.

Working with SQL, it was hard to track what was being searched versus what was being booked. It was particularly hard to find blind spots in sales. Now, the wider business uses the same dashboards that operations use, taking the insight to inform long-term strategies.

Being able to track ‘look to book’ ratios and analyse how many searches result in a booking, per destination, per client, allows Sunhotels to tune sales and contracting strategies; useful when onboarding new providers and clients. We can make educated, profile-based assumptions and deliver a service based on historical insight. This enables us to optimize traffic. More bookings with less traffic; win-win.

How does this help Sunhotels with its ongoing strategy of innovation and growth?

There are two main ways in which our data management infrastructure benefits both us and our partners. Firstly we want to detect changes in behaviour more quickly so we can respond proactively. When we see trends are appearing over time (e.g. January is a busy time for beach holiday booking in Scandinavia versus the Spanish habit of booking last minute), we’re able to predict behaviours and align the service accordingly.

Predicting behaviour from historical insight means we can be proactive in offering our sales/contracting department the information they need to act swiftly, too. This might mean boosting the number of properties available in a certain area or at a certain price point, if we expect demand to be high. This could also mean increasing activities in quieter regions; improving bookings and warning partners about anticipated lulls or spikes.

Secondly, by being able to crunch terabytes of data within Elasticsearch, we are a very lean organisation. The travel industry - particularly B2B - is very traditional, characterised by large teams doing much of the heavy analytical lifting. With Elasticsearch taking the strain out of processing volume data, our team remains focused on using the analysis to improve the service. We’re experiencing dramatic efficiencies in terms of processing time and man hours, as a result.

What part will machine learning play in Sunhotel’s strategy?

As the business looks to us to provide deeper insight, Elastic’s X-Pack, and its machine learning functionality, are enabling us to automate a huge layer of analysis, currently being handled by 12 bespoke-built robots, managed by a very small team of engineers. We can spend a lot of time on monitoring and tweaking parameters, simply because we have ten times more data, external connections, and incoming requests than we did a year ago. We had to find a way to automate complex analysis, at scale.

If one of our customers does 150 bookings a day, our robots can look at some of the behaviours and anomalies associated with those transactions but, to make meaningful assumptions about the relationship between points, a complex coding process has to take place. Without machine learning, it’s very hard to cross analyse; select metrics from multiple indices.

We also want the infrastructure to take seasonality into consideration automatically. We usually have to alter this manually across all robots, which is very time-consuming and technical. Machine learning can help us analyse against consistent rules, implemented automatically.

With multiple clusters and multiple nodes we knew we weren’t talking about trivial money and so, after a period of investigation with a number of machine learning solutions, we were pleased to see machine learning included as part of X-Pack. Machine learning can easily be deployed within the existing Elastic ecosystem; no integration or the need for separate logins.

sunhotels_machine learning.png

Figure 1: Jobs related to bookings (number and value of cancellations, bookings per provider, booking errors etc.)

sunhotels_machine learning_external bookings.png

Figure 2: Response time from external providers