• Application Search
  • Retail

Shopback: Shopping and searching smarter with help from the Elastic Stack

At a glance
  • 13
    million products indexed
  • 2
    million categories
  • 27
    GB worth of logs and metrics per day
The Challenge

How do you offer a smarter shopping experience for 2 million online shoppers?

The Solution

By switching from AWS ES to the Elastic Stack and X-Pack to scale and expand the business into new markets.

Shopback’s Journey with Elastic

ShopBack partners with more than 1,300 merchants in six different countries to bring happiness to online shoppers in the form of rewards and new store discoveries. With the help of Elastic, it’s enhancing users’ online experiences and making it faster and easier for them to find the best deals.

ShopBack initially used Amazon Web Services (AWS) CloudSearch to power user search but performance was lagging as product searches took an average of two seconds while bringing up a store listing could take up to 30. The business migrated from AWS Cloudsearch and the Amazon Elasticsearch Service to working with Elastic to set up its own Elastic Stack cluster.

Through AWS, we were using an older version of Elasticsearch and missing out on the full features and benefits of the Elastic Stack. Also, the service we had was not set up to handle multi-lingual search. So to scale the business we knew we had to make a switch.

Alberto Resco Perez
Engineering Manager at ShopBack

ShopBack made the decision to implement its own cluster, subscribing to both the Elastic Stack and X-Pack. The premium support that came with the subscription helped ShopBack to design its Elasticsearch schemas while the features offered by X-Pack helped to ensure the security of its cluster.

With the help of the local Elastic team, ShopBack spun up its first cluster in just a few hours. This cluster supports the indexing and search of ShopBack’s product catalogue and has evolved over time to support the indexing of 13 million products and 15 thousand categories.

Performance has improved significantly with product searches taking as little as one millisecond and store listings coming up in just four seconds. Also, auto-suggestion is now live on the site, helping ShopBack’s two million users find what they want even faster.

“We’ve migrated from Elasticsearch 2.4 to 5.0 and not only has performance improved but we now have a schema for multi-country, multi-lingual search. This has aided our expansion into Indonesia and Taiwan where we support local languages,” said Perez.

New Insights Improve Site Performance and Availability

More recently ShopBack has implemented a new Elastic Stack cluster to simplify logging and metric collection. In the past, ShopBack had used Papertrail, AWS Cloud Watch and New Relic to collect and analyse various metrics. However, the combination of all three solutions made it expensive and complex to scale. ShopBack also wanted to collect more metrics and get more insight into the performance of its services.

It now uses Logstash and Metricbeat to ingest logs and collect metrics from a variety of sources. It uses Kibana to visualise and analyse the data. The Elastic-based solution offers the ShopBack team greater insights at a reduced cost. It’s also more secure with X-Pack security features used to protect both of ShopBack’s clusters.

“In the past, we had limited visibility into our environment. The Elastic Stack and additional monitoring and alerting capabilities of the X-Pack have resolved that and helps us to detect and resolve issues so much faster,” said Alberto. “It also helps us to analyse user behaviour so we can scale our systems and meet demand.”

As an example, ShopBack now receives updates on CPU memory and disk demand every 15 seconds as opposed to every five minutes. This gives the business more time to manoeuvre to scale capacity and reduce the risk of downtime.

ShopBack has plans to extract even more benefit from the Elastic Stack in the future. This includes using Filebeat to collect all its service metrics and leveraging machine learning for fraud detection.

Fraud is a big issue for e-commerce and one that we hope to tackle with the help of machine learning from Elastic.

Alberto Resco Perez
Engineering Manager at ShopBack

Currently Elasticsearch is used to index the content of four of their major sites — energyvoice.com, pressandjournal.co.uk, eveningexpress.co.uk and eveningtelegraph.co.uk — and to provide related articles, which increases page views and advert revenue. The ultimate goal is to migrate most of the news sites to Elasticsearch and off of the 3rd party CMS they run on now (Polopoly) and put them on the Wordpress cluster. The plugin for Wordpress works with Elasticsearch, so with that migration they will automatically be running on Elasticsearch.

It took me 10 lines of configuration and I had my main cluster setup in less than 20 minutes.

Alistair Colville
Lead Digital Developer, DC Thomson

Ever since the change from Solr to Elasticsearch, DC Thomson has seen a great increase in pageviews by up to 40% and that is mainly a result of the added ‘related article’ function, which helps to serve up readers with related and relevant content. By adding class names to the related article links, DC Thomson is able to track the clicks. Even after 2 weeks of having Elasticsearch in place, they have been seeing an increase in page views, which helped hugely to justify that they’ve chosen the right solution.

Furthermore the bounce rate decreased, the number of pages per session has gone up and the general user experience is much faster and smoother than before. All of this benefited DC Thomson hugely as it showed that they can monetize Elastic given that they have a paywall on the Press and Journal and Energy Voice sites. It is important for their business to move people around their sites and feed them with the information they are looking for and Elastic plays a crucial part in this.

Content is critical to our monetization strategy, and Elastic helps us deliver personalized, real- time content so that our subscribers get the news they want anytime, and on any device.

Martin Little
Online Editor, Press & Journal

Another use case that DC Thomson has on their road map is to create an editorial backend for journalists that would allow them to investigate previous articles and render the homepages to see how popular certain articles are. With the ELK stack the digital director can show immediate results, which is what the board wants to see and hear.

Being able to monitor page views will allow journalists to curate content and tell the user what the best stories are by implementing certain trending and most read, tech widgets on the site. Ultimately the goal for DC Thomson is to dynamically show the most popular articles (most reads, social shares etc.) on the homepage. Finally, getting insights into the reader’s data (location, behaviour on the site etc.) will further allow DC Thomson to see who they really want to target with a personalised experience when visiting the news page (“These are the articles you should be interested in”).

Elasticsearch is a powerful tool and we wish to use it to its full extent!

Alistair Colville
Lead Digital Developer, DC Thomson
Products Used