CreatorIQ: Powering the search for social media influencers with Elasticsearch

This post is a recap of a community talk given at a recent Elastic{ON} Tour event. Interested in seeing more talks like this? Check out the conference archive or find out when the Elastic{ON} Tour is coming to a city near you.

CreatorIQ’s SaaS marketing platform enables users to sift through millions of social media influencers to find a handful of endorsers that can best drive their brand forward. Say a company wants to broaden their social media presence on a particular marketing campaign. Maybe they’re looking for a Midwestern mom turned blogger and baby product endorser. Or a twenty-something musician with an affinity for adult beverages (good luck narrowing that down). This type of brand advocacy has seen huge growth, but how does CreatorIQ help companies pluck out these influencers? Ones that align with desired target demos and that can promote products and services via social media in a way that traditional ads just can’t? Easy — with highly relevant search.

Built on top of the Elasticsearch Service on Elastic Cloud, the CreatorIQ Discovery app allows users to pore over millions of popular influencer social media accounts — athletes, actors, comedians, gamers, models, vloggers, and so on — representing over 25 million different social media accounts and billions of data points (posts, audience, demographics, and many others) to find the perfect influencer accounts for the user’s needs. “The [influencer marketing] industry was exploding… and the user story starts with search — finding these influencers,” as CreatorIQ Product Manager Henry Pray put it.

CreatorIQ not only needed fast, relevant search, but also a robustness to it that went well beyond basic searches of vanity metrics such as followers and likes. But like a lot of lean tech startups, CreatorIQ’s resources were tight. The early versions of Discovery were built on top of a SQL database, which was easy to deploy and manage, but full-text search across a lot of data sources was too slow. They knew they needed faster search, and they needed that system to stay fast as they grew. “We [didn’t want to] reinvent the wheel on search — we’re not search experts. We needed a bigger and better tool to take care of our search functionality, and naturally Elasticsearch was an immediate go-to for us,” said Pray.

But with just a few engineers on staff, a very limited budget, and an old version of Elasticsearch (2.x), CreatorIQ was spending time they didn’t have managing and optimizing their deployment. They knew they needed a better solution, one that enabled them to concentrate on what they did best. “We wanted to move away from managing [search] ourselves, and [instead] focus on what we do well: analyzing social media and building a workflow management tool,” said Pray. Because CreatorIQ was already running on AWS, their initial hunch was to switch over to AWS Elasticsearch Service. But they realized rather quickly that the Elasticsearch Service on Elastic Cloud was the superior solution for their needs.

“These are the guys that built Elasticsearch. This is their bread and butter. This is what they do,” said Pray. “They offer all this support… plus machine learning and security — all these things that we didn’t want to have to deal with developing and Amazon doesn’t offer.” Henry Pray, Product Manager | CreatorIQ

The support that comes from Elastic ended up being the game changer. Dealing with the ever-changing APIs of the social media giants — and the accompanying reindexing challenges — was magnifying CreatorIQ’s technical difficulties and pulling valuable time and resources away from product development. On top of that, CreatorIQ didn’t have a proper staging environment, which left them vulnerable to downtime as they implemented changes. When a cluster failed the day before a big client demo, they immediately called Elastic support for help. Even though CreatorIQ only had access to Elastic Cloud’s standard support offerings, the Elastic Support team jumped in right away to help solve their problem (and later helped them set up a staging environment for appropriate testing). CreatorIQ upgraded to a Gold subscription that day, which gave them guaranteed resources and response times from Elastic, and helped ensure that CreatorIQ could meet service level agreements with their customers going forward. “We should have [gotten a subscription] in the first place, before launch… it’s been really huge for us,” said Pray.

Watch the full CreatorIQ Elastic{ON} Anaheim presentation to find out more about how CreatorIQ’s Discovery app has evolved, and their plans to try out the Elastic APM and App Search products.  

Watch the full CreatorIQ presentation from Elastic{ON} Tour Anaheim

Are you considering hosted Elasticsearch Service on AWS? Not all offerings are created equal. Find out more about why the fully loaded one from us, Elastic, is right for you. See AWS Elasticsearch comparison: Amazon and Elastic for the complete tale of the tape.