Shopback: Shopping and searching smarter with help from the Elastic Stack


  • 13
    million products indexed
  • 2
    million categories
  • 27
    GB worth of logs and metrics per day

The Challenge

How do you offer a smarter shopping experience for 2 million online shoppers?

The Solution

By switching from AWS ES to the Elastic Stack and X-Pack to scale and expand the business into new markets.

Shopback’s Journey with Elastic

ShopBack partners with more than 1,300 merchants in six different countries to bring happiness to online shoppers in the form of rewards and new store discoveries. With the help of Elastic, it’s enhancing users’ online experiences and making it faster and easier for them to find the best deals.

ShopBack initially used Amazon Web Services (AWS) CloudSearch to power user search but performance was lagging as product searches took an average of two seconds while bringing up a store listing could take up to 30. The business migrated from AWS Cloudsearch and the Amazon Elasticsearch Service to working with Elastic to set up its own Elastic Stack cluster.

Through AWS, we were using an older version of Elasticsearch and missing out on the full features and benefits of the Elastic Stack. Also, the service we had was not set up to handle multi-lingual search. So to scale the business we knew we had to make a switch.

– Alberto Resco Perez, Engineering Manager at ShopBack

ShopBack made the decision to implement its own cluster, subscribing to both the Elastic Stack and X-Pack. The premium support that came with the subscription helped ShopBack to design its Elasticsearch schemas while the features offered by X-Pack helped to ensure the security of its cluster.

With the help of the local Elastic team, ShopBack spun up its first cluster in just a few hours. This cluster supports the indexing and search of ShopBack’s product catalogue and has evolved over time to support the indexing of 13 million products and 15 thousand categories.

Performance has improved significantly with product searches taking as little as one millisecond and store listings coming up in just four seconds. Also, auto-suggestion is now live on the site, helping ShopBack’s two million users find what they want even faster.

“We’ve migrated from Elasticsearch 2.4 to 5.0 and not only has performance improved but we now have a schema for multi-country, multi-lingual search. This has aided our expansion into Indonesia and Taiwan where we support local languages,” said Perez.

New Insights Improve Site Performance and Availability

More recently ShopBack has implemented a new Elastic Stack cluster to simplify logging and metric collection. In the past, ShopBack had used Papertrail, AWS Cloud Watch and New Relic to collect and analyse various metrics. However, the combination of all three solutions made it expensive and complex to scale. ShopBack also wanted to collect more metrics and get more insight into the performance of its services.

It now uses Logstash and Metricbeat to ingest logs and collect metrics from a variety of sources. It uses Kibana to visualise and analyse the data. The Elastic-based solution offers the ShopBack team greater insights at a reduced cost. It’s also more secure with X-Pack security features used to protect both of ShopBack’s clusters.

“In the past, we had limited visibility into our environment. The Elastic Stack and additional monitoring and alerting capabilities of the X-Pack have resolved that and helps us to detect and resolve issues so much faster,” said Alberto. “It also helps us to analyse user behaviour so we can scale our systems and meet demand.”

As an example, ShopBack now receives updates on CPU memory and disk demand every 15 seconds as opposed to every five minutes. This gives the business more time to manoeuvre to scale capacity and reduce the risk of downtime.

ShopBack has plans to extract even more benefit from the Elastic Stack in the future. This includes using Filebeat to collect all its service metrics and leveraging machine learning for fraud detection.

Fraud is a big issue for e-commerce and one that we hope to tackle with the help of machine learning from Elastic.

– Alberto Resco Perez, Engineering Manager at ShopBack

Products Used