Growth Intelligence

Searching through data on millions of companies

Growth Intelligence offers a B2B marketing tool providing up-to-date data on 3 million private companies in the UK. The company's customers – major UK-based banks, telecoms and technology companies – use this information to identify highly-qualified sales leads.

Growth Intelligence customers access all the data via a web portal containing two lead generation tools: Campaign Builder and Prospect Recommender. Search is a key functionality for this Software-as-a-Service (SaaS) tool, enabling the users to find targeted leads.

"Before Elasticsearch, we used a combination of more traditional relational database technology, like MySQL and Sphinx," recalls Prash Majmudar, CTO of Growth Intelligence. "It was far too slow. As we started to scale up our data, we had real speed issues, in terms of query times. We also had deployment and scalability issues. It became unfeasible to move that amount of data around from our test system to our live system."

Built for speed on Elasticsearch

To solve performance and scalability issues, Growth Intelligence deployed Elasticsearch as one of the core elements of the API that drives the web portal.

"Our web portal is built for speed on Elasticsearch," says Majmudar. "Elasticsearch enables us to rapidly provide prioritized leads, and quickly filter down the leads that are most relevant to our customers today."

"Before Elasticsearch, our query types tended to be from seconds to tens of seconds," he continues. "Some of our slowest queries with complex filters could take up to 30 seconds. With Elasticsearch, we are at 100 milliseconds and even faster for our more complex searches."

In addition to query speed, another speed requirement for Growth Intelligence is data indexing. The data must be available to customers the same day it is collected, to support timely sales and marketing decision-making.

"Elasticsearch gives us the advantage of being able to index our data quickly, and get it into our system quickly," says Majmudar.

"It comes down to trust, because the user searches our system for contracts awarded or product launches for potential leads," elaborates Alex Mitchell, Data Team Lead at Growth Intelligence. "If they don't see that data from the day before, they lose faith in the data and our tool pretty quickly."

Quality without sacrificing performance

"We live or die by the quality of the leads we provide to our customers," Mitchell explains. "If they are not satisfied with the top 10 leads we return, they are not going to trust the tool. It is really important that we display the most relevant results in a timely fashion to our users, so they can gain confidence in our system. With Elasticsearch, we know we are giving our customers the best leads. Elasticsearch allows us to do that, because of the richness of the scoring and ranking."

"The flexibility and richness of the queries are crucial for us," he continues. "We can build sophisticated queries quickly and have confidence that is going to give our users leads they can believe in."

"We have seen a 14 times increase in conversion rates for our customers," Mitchell adds. "The reason  is that we have been able to offer them highly qualified, highly relevant leads – and that is because we use technologies like Elasticsearch."

Index 200K new documents every day

Growth Intelligence indexes hundreds of millions of documents via Elasticsearch. Every day they add hundreds of thousands of new pieces of content about UK businesses, that they call events. In addition, they are continually adding new sources of information and new types of events, driving further data growth.

For Growth Intelligence, scalability is about being able to rapidly increase the number of businesses they can cover. This is essential because the company plans to expand from the UK into Europe, and eventually the US.

"It is very easy for us to scale out with Elasticsearch, without fear of performance hits," says Majmudar.

"Elasticsearch allows us to be more agile with our customers," he continues. "Sometimes they request new types if data. In Elasticsearch, we can just update the mappings and index a whole new data set that immediately adds value to our customers."

Easy and economical development

"One of the main advantages of Elasticsearch is the low cost of development," Majmudar says. "We can rapidly build new features into our tools, to add more value for our customers."

"Elasticsearch allows us to do that very easily for a few reasons," he continues. "First, Elasticsearch uses a very standard JSON REST interface, which significantly lowers the cost of development time for us. Second, Elasticsearch is pain-free to use. It is easy to create a new index, to set up mappings, and to start adding data to our system."

The ease and low cost of development also allow Growth Intelligence to try out new features with minimal risk. For example, they can experiment with new ways to filter or facet data.

"One of our new developers was able to get up and running with Elasticsearch in a couple of hours and fix a bug," says Mitchell. "With our old system that would be a week of downloading libraries, plug-ins, etc."

"Our development velocity has increased as much as 20% by using Elasticsearch," Mitchell concludes.