Elasticsearch Enables 400 Criteo Engineers to Search 4 TB of Log Data Per Week
Amsterdam, The Netherlands - 30 June 2016
Elastic, the company behind the world’s most popular distributed real-time search engine Elasticsearch, today announces that Criteo (NASDAQ: CRTO) has developed a new internal log management tool using the Elastic Stack, allowing Criteo to analyze between one and eight billion event logs every week, diagnose potential problems before they occur and maintain optimum efficiency in its datacenters.
As a global leader in digital performance marketing, Criteo operates in over 80 countries across the Americas, EMEA and Asia-Pacific, serves over 11,000 advertisers and has a direct relationship with close to 16,000 publishers to buy inventory space and serve display ads in real-time. In 2015, Criteo delivered 710 billion targeted online ads and each bid request must be processed in under 100 milliseconds, making the performance of Criteo's server infrastructure vital to the company's success.
"We are currently running eight datacenters in the US, Europe and Asia with 17,000 servers creating event logs continuously. Each week we store 4 TB of data — that's about 8 billion documents. We needed a solution which could search the event logs quickly and be able to cope with the high levels of data," explains Nicolas Helleringer, Site Reliability Engineering Director at Criteo.
Event logs are files which record incidents that occur on the servers or within the applications. Every time an incident occurs, event logs are created to enable administrators to troubleshoot problems. As Criteo experienced hyper growth, it became clear that Criteo needed a new tool to cope with huge amounts of data and quickly pick up event log issues. Elasticsearch was selected based on its ability to scale horizontally across huge volumes of data, ease of use for developers to setup, and because it automatically works with Kibana for real-time analytics. This enables Criteo's 400 engineers to identify system issues and errors, and take action immediately.
"Using the Elastic Stack as our log management system, not only are we able to diagnose the problems quickly but we can look at trends, for example which errors occur the most," further explains Helleringer. "The next stage will be to foresee problems before they arise, potentially using more advanced features of Elasticsearch."
"Criteo is at the forefront of the digital economy, dealing up millions of online ads in real-time across the world every day," explains Gaurav Gupta, VP Products from Elastic, "We love it that Criteo has been able to build an internal log management tool using the Elastic Stack that meets the needs of its engineering, devops and IT teams."
Criteo (NASDAQ: CRTO) delivers personalized performance marketing at an extensive scale. Measuring return on post-click sales, Criteo makes ROI transparent and easy to measure. Criteo has 2,000 employees in 31 offices across the Americas, EMEA and Asia-Pacific, serving 11,000 advertisers worldwide and with direct relationships with 16,000 publishers.
For more information, please visit www.criteo.com.Media Contact:
Elastic is the world's leading software provider for making structured and unstructured data usable in real-time for use cases like search, logging, and analytics. Founded in 2012 by the people behind the Elasticsearch, Kibana, Logstash, and Beats open source projects, the Elastic Stack, X-Pack and Elastic Cloud have had more than 50 million cumulative downloads to date. Elastic is backed by Benchmark Capital, Index Ventures, and NEA with headquarters in Amsterdam and Mountain View, California, and offices and employees around the world. To learn more, visit www.elastic.co.
- Dan Reidy
- Reidy Communications