MercadoLibre: Powering Latin America's premier online marketplace

Based in Argentina, MercadoLibre is Latin America's largest online marketplace, eBay's Latin America sales partner, and operates similarly to eBay, bringing millions of buyers and sellers together. The company has 4 million active sellers with 20 million live product listings.

"Before using Elasticsearch, we used a Redis database," explains Federico Ocampo, Project Leader at MercadoLibre. "But Redis is not meant to be a search engine. We had to reindex all of our data every time we wanted to add a new attribute. Redis did not scale and it did not provide the high availability we required."

Today, MercadoLibre uses Elasticsearch for the "My Account" section of the site, where sellers manage their products for sale. When sellers log into the system, they can view a list of every product they are selling – called "My Items" – which is populated and searchable via Elasticsearch. From that list, the seller can edit attributes, check quantities and change prices.

Sometimes a seller must search through thousands of items. A typical MercadoLibre user will sell a few thousand products, but some sell as many as 50,000 items. Elasticsearch enables sellers to find products by filtering based on a variety of parameters, such as product type, price, quantity or date the item was published.

"The My Account section of the system is critical," says Ocampo. "Without the ability to search through their items, sellers would not be able to view their products, add new products, make changes to the descriptions, and adjust prices – sellers would not be able to sell. The entire system would be paralyzed if we did not have an easy way for sellers to search and manage their items."

For a user selling thousands of products, search is obviously essential. But even the user selling only a few items still depends on Elasticsearch in order to manage those products. Elasticsearch not only empowers sellers to manage thousands of products, but also helps millions of users conduct their business regardless of the number of product listings. The company's revenue depends directly upon the seller's ability to manage the products they are selling, and Elasticsearch makes this possible.

All MercadoLibre front ends call APIs on the backend. The APIs are available to anyone who wants to build an application based on MercadoLibre. These APIs are can be used for many different purposes, both internal and external. Because Elasticsearch worked so well, MercadoLibre began to use it in other ways via APIs. First, they implemented an API over Elasticsearch to help users to search through questions asked by buyers and answers provided by sellers – the key method of communication between buyers and sellers prior to a sale. Then, they created a second API that helps users to search orders.

"We develop our search APIs very quickly and easily with Elasticsearch," says Silvina Lucero, Senior Software Engineer at MercadoLibre. " These were two new projects with very different requirements and Elasticsearch has adapted well to both of them."

"Real-time search is important for us, because as soon as the seller publishes a new item, they want it to appear in the list," Ocampo explains. "So we need that process to be as quick as possible. With Elasticsearch, items are searchable within one or two seconds after being published."

The performance of query response is also important to MercadoLibre. The My Account search receives 6,000 requests per minute, while the other functions receive up to 2,000 requests per minute, and all the results must be returned quickly because e-commerce users have low tolerance for performance issues.

An Elasticsearch feature called "Custom Routing" enables MercadoLibre to route user data to a single shard. Under normal operation, searches in Elasticsearch are broadcast to all shards in the index. Multi-tenant indices often contain many shards to support future growth, but this can impact query latency due to the overhead of searching many, many shards simultaneously.

By enabling Custom Routing, customer documents are automatically co-located to a single shard. Elasticsearch can now search a single shard instead of broadcasting. This approach limits the search request to the one server, reduces the overhead of conducting searches, and produces instant query responses. Custom Routing enables multi-tenant scalability with a large number of shards without sacrificing query latency.

"Originally, for searching Orders, we used MongoDB," Waisgold recalls. "But faceting and other search features did not work very well.

When we migrated to Elasticsearch, the Order search started to work very well, especially for large sellers. We previously had response time of two seconds for very big sellers, and with Elasticsearch it was reduced to 100 milliseconds."

"Because we work in many different countries, with different time zones, we have to be up 24/7," Ocampo points out. "For MercadoLibre, reliability is critical. We cannot let the system go down if a server crashes. If a server goes down, sellers will not be able to sell, and that is a real problem. So we need to be sure we have high availability."

"Elasticsearch offers replicas, and dynamic replicas which help us increase or decrease the number of replicas in runtime," he continues. "We have found a balance between high availability and performance and the amount of servers that are ideal for us."

Availability was a problem before, when MercadoLibre used Redis. Every time they had to re-index or add a new server, they spent a day with the My Account system down. Elasticsearch continually re-indexes in real time, so now the system is never down.

"In the past, we would spend several hours per week of developer time maintaining the system," Ocampo adds. " With Elasticsearch, we do not need to do that. Elasticsearch does not require any maintenance at all, and we are very sure it works well. Elasticsearch delivers a savings in hours and money."

Two years ago, MercadoLibre hosted 12 million active listings. Today, they have 20 million. So the growth is significant, and scalability is critical.

"Before Elasticsearch, adding a new server required re-indexing the whole database, resulting in major downtime," Ocampo concludes."

That was not an option. Elasticsearch offers the possibility of adding or removing a new server in a second. We have more and more data every month, and we need to add new servers – and Elasticsearch is great for that."