By using Elasticsearch as the foundation of your risk management system
Case Study Highlights
- Process 500 search requests per second
- Increase processing speed by 5x
- Search 200GB index in milliseconds
- Reduce code and complexity per second
- Utilize easy-to-use APIs
- Free up developers for more productive tasks
Protecting More than $6 Billion in Transactions
Xoom (xoom.com) is a worldwide leader in the digital money transfer industry. Based in San Francisco, the company enables consumers to use their mobile phone, tablet, or computer to send money to more than 30 countries in a secure, fast, and cost-effective way. From July 2013 to June 2014 alone, Xoom's more than 1.1 million active customers sent more than $6.1 billion to family and friends.
Xoom's goal is to allow users to send money as quickly and easily as possible. While this type of service may be good for customers, it makes Xoom a target for fraud and illegal activity. To combat these threats, Xoom has become very successful at detecting that type of activity. As a result, the company's loss rates are very low.
"We couldn't do our job without search," explained Josh Holtzman, Senior Engineering Manager for Infrastructure at Xoom. "Our risk engineering team has used search technologies for a long time to do this type of analysis. Before Elasticsearch, we used an older version of the Lucene library but our indices were becoming unmanageable. We really needed to shard them in some way."
We desperately wanted to get rid of as much legacy code as possible. We needed better performance, and a search engine that was easier to manage. We needed Elasticsearch.
They also had trouble keeping their indices current and synchronized. Consequently, running one transaction through two different boxes produced two different results instead of two identical results. This was particularly problematic for their business and Holtzman's team didn't have a clear enough understanding of the cause to be able to resolve the mismatch.
"Managing the legacy search system required a lot of code to keep track of how up-to-date each of these individual indices were, and what needed to happen," Holtzman added. "It was a code nightmare; it was far too complicated for what it was doing. In addition, our search system was slow, and we couldn't process transactions as fast as we wanted."
"We desperately wanted to get rid of as much legacy code as possible," Holtzman recalled. "We needed better performance, and a search engine that was easier to manage. We needed Elasticsearch."
Boosting Performance by 5x
Every year, Xoom has more customer transactions than the year before, and more data with each transaction to search through. As a result, the ability to seamlessly scale the search system is critical for the business.
"We see between one hundred and five hundred search requests per second," said Holtzman. "The ability to search a 200GB index in milliseconds is a huge advantage of Elasticsearch. With our legacy search system, we were unable to get that kind of performance when the indices grew larger."
When Holtzman's team implemented Elasticsearch, they saw processing time drop from 20 seconds to 4 seconds, boosting their performance fivefold.
"We needed to improve our search," Holtzman added. "It was a matter of keeping the business running—and Elasticsearch came through for us."
We see between one hundred and five hundred search requests per second. The ability to search a 200GB index in milliseconds is a huge advantage of Elasticsearch. With our legacy search system, we were unable to get that kind of performance when the indices grew larger.
To work around their legacy search system's performance problems, Xoom had to maintain two different versions of each index — one with the last nine months of data, and the other with all historical data. To speed up processing, the risk engine would query the nine-month index.
"The performance was so bad with our legacy search system that we were actually keeping a separate index with the most recent data," Holtzman explained.
With Elasticsearch, Xoom can now index the same data in multiple indices, using different analyzers. This flexibility gives Xoom the ability to make very targeted queries and perform highly specific matching. "Not having those limitations with Elasticseach is a big win," said Holtzman. "Elasticsearch has given us a lot more flexibility, and we don't need a bunch of code to build different indices for each use case." Less code meant the company's developers could spend their time on more productive tasks, another advantage of using Elasticsearch.
"We also use Elasticsearch's Java APIs," he added. "They are a pleasure to work with. It is such a simple, nice model to use."