Reinventing Fermilab's Scientific Computing Grid Accounting with the Elastic Stack

In 2017, Fermilab launched a project to re-engineer an aging MySQL-based grid accounting system in their scientific computing division to a distributed system based around Elasticsearch, RabbitMQ, and Grafana. Leveraging this suite of open source tools, Fermilab was able to deploy a more flexible, scalable, and responsive system, all while saving resources and man hours.

Today, their scientific computing division is using the Elastic Stack to collect, search through, and analyze data on the state and usage of their distributed computing system that connects thousands of researchers with hundreds of computing resources around the world. Discover what motivated this huge change, get details on the migration, and hear lessons learned along the way.