The Challenge
The Solution
Case Study Highlights
Create an amazing user experience
- Use search as navigation
- Update your searches immediately with user contributed content
- Roll out new features quickly and often automatically
Develop and scale easily
- Use high velocity when working on schema-affecting bugs and features, so a complete reindex should take on the order of an hour
- Scale to the next order of magnitude and beyond with minimum effort
- Maintain access through APIs that you design and control
Powerful and Intelligent Search is Key to Enhanced Customer Experience
Providing the best experience for customers to view, search and explore relevant content in a timely and meaningful manner is paramount for SoundCloud. As the world's leading social sound platform, SoundCloud lets anyone create, record, promote and share their sounds on the web in a simple, accessible and feature-rich way. SoundCloud enables sound creators to instantly record or upload original audio content, embed sound across websites and blogs, share publicly and privately, receive detailed analytics, plus get feedback from the community directly onto the waveform. Recently, SoundCloud set out to redesign their search infrastructure to meet growing customer demands and to keep pace with the massive growth in their data.
The new infrastructure has been an unqualified success. Our velocity is through the roof, we have a clear path for orders-magnitude-growth, and we have a stable platform for a long time to come.
Redesigning the Search Infrastructure to Optimize Customer Experience
Front and center of the new, redesigned SoundCloud is intelligent search capabilities which are key to providing a world-class customer experience. SoundCloud has made the search box one of the first things a user sees and augmented it with suggestions that enable a user to jump directly to people, sounds, groups, and sets of interests. The company has also added an Explore section that guides users through the massive landscape of sounds on SoundCloud.
"Essential to our redesign was an overhaul of our search infrastructure," explains Alexander Gross, VP of Engineering, SoundCloud. "We needed a new infrastructure to help us deliver more relevant results, scale with ease, and experiment quickly with new features and models."
SoundCloud’s mobile interface
SoundCloud Moves from Apache Solr to Elasticsearch
When SoundCloud was first founded, the company relied upon Apache Solr for their search requirements. Over time, however, SoundCloud’s data requirements exploded, their social sound platform became more complex, and their business rules multiplied. Solr began to run into problems in terms of speed, scalability, flexibility and overall search performance.
"Being real-time is critical in the world of sound," said Alexander. "It is essential that our users’ sounds are searchable immediately after download and it became clear that as we redesigned our site we needed a more performant and reliable search engine to replace Solr."
After an extensive survey of the state-of-the art in search technologies, SoundCloud decided to abandon Solr in favor of Elasticsearch.
"The real-time aspect of Elasticsearch is very important. Previously, with Solr, it would take days for something to become searchable," says Gross. "We wanted a universal search as navigation capability. With Elasticsearch we get much more valuable insight into our data, and it is much easier to configure and maintain."
