How BigSpark levels up the price comparison for mobile tech services with Elasticsearch
BigSpark is the publisher of some of the biggest tech websites in the Netherlands, among which AndroidPlanet.nl, iPhoned.nl and OnlySim. Every month, more than two million consumers visit these websites on which we inform about and inspire with up-to-date news and in-depth articles on subjects like mobile tech, apps, games and devices. Furthermore, we advise them on specific purchases using our product and price comparison service.
Our mission is to inspire consumers to get the most out of technology and to help them to make better choices. “Elasticsearch helps us by accomplishing this mission”, says Peter Geurts - Founder of BigSpark.
Dominant player in the Netherlands
In just three years, BigSpark managed to become a dominant player on the Dutch market for product and price comparison services for mobile tech devices. Every month, the telecom experts of BigSpark advise customers on 70,000 purchases of smartphones, tablets, wearables and plans. Our price comparison service is powered by Elasticsearch, which makes it possible for BigSpark to claim a unique position in the market and continuously compare three million product prices of among others:
- Android devices and iOS devices, new and refurbished;
- Smartphones with a plan (renewal) deals, i.e. the iPhone 6S with plan;
- SIM only deals.
Thanks to Elasticsearch, consumers can easily search the complete product offerings of Dutch telecom providers and shops. They can use facets to filter on product features such as color, storage, data allowances, provider, as well as conditions such as time of delivery and user reviews of shops.
Where we come from
Originally, the setup of our price comparison service was based on Mongo DB and Node.js, but they did not keep up with our requirements anymore. Our databases were getting too big, aggregations became too slow and the performance of the service became unpredictable.
A new setup in PostgreSQL, connected with Elasticsearch, proved to be the right solution. Suddenly, it became easy to quickly search the database, taking different groups and facet preferences into account. “When you ask the right question to Elasticsearch, you’ll get an answer in no time”, says one of BigSpark’s Development engineers.
Under the hood
Elasticsearch has been installed on a cluster of two instances with 8GB RAM each. The index with offers is around 3GB and has a very good performance in combination with a low load. With Elasticsearch, the average query takes around 50ms which is significantly faster than previous queries. We are very impressed by the way Elasticsearch allows us to still deliver big performance with relatively limited resources.
The future with Elasticsearch
Elasticsearch has several interesting out-of-the-box features with which we can improve the power of our price comparison service. These are for example function scores, which enables us to add a higher value to certain factors (like conversion, changes in search volume or the profile of the consumer) in order to influence the search results.
Furthermore, we are going to give our users the possibility to intelligent suggestions on free text queries such as "did you mean "Galaxy S7"?" when someone accidentally typed "Galaxie S7”.
The success of BigSpark is partly thanks to the possibilities and accessibility of Elasticsearch: “With relatively little resources, acceptable hosting rates and a good Elasticsearch setup, superpowers are within reach for everyone nowadays”, concludes Peter Geurts.