Global 2000 energy company boosts application and infrastructure performance, saves employee time and operations costs with Elastic Observability

Monitoring application and infrastructure across countries and organizations presents a number of challenges. As the company grew, it deployed a complex array of monitoring tools and integrations, but this created many potential points of failure. Small issues could sometimes crash the entire system.

Incident reporting was also inconsistent. In some cases, the IT team only became aware of a problem when an employee reported it. Team members often worked overtime to fix issues, which prevented them from focusing on work that added value to the business.

One IT Manager at the company responsible for the health of the company’s information systems, was determined to improve the monitoring process as well as the collaboration between the AIOps and DevOps teams. In addition to accelerating incident resolution, he wanted to reduce the pressure on his team caused by a steady influx of incident tickets. "Our first goal was to improve observability by finding new ways to identify anomalies and take action to correct them," says the IT Manager. "Another was to reduce manual work, using automated processes to solve certain problems, which would reduce the burden on the DevOps team as well as lower costs."

A complete rethink of the monitoring architecture was needed. The IT Manager and several members of his team were already familiar with Elastic from previous roles and quickly agreed to deploy Elastic Observability for application and infrastructure monitoring. The configuration includes Elasticsearch for data storage and search, plus Kibana for visualization.

Currently, the Elasticsearch clusters contain 5.5 billion indexed documents and 5.2 TB of storage with index search latency below one millisecond.

"The performance is really impressive," says the IT Manager. "Our Logstash environment currently handles around 1,500 events per second. The latency rate of events is less than two milliseconds."

At the time of the Elastic deployment, most employees had begun working from home due to the Covid-19 pandemic, which also presented an observability challenge. The IT Manager decided to leverage Elastic to monitor the corporate VPN, including the health and performance of VPN apps and data transmission. This enabled the company to move more smoothly to a hybrid working environment while protecting business security and supporting employee productivity.

Elastic Observability has since been deployed to monitor dozens of other systems including the hundreds of Oracle databases that underpin the company’s ERP systems. Elastic also monitors the APIs that support integration between different business applications.

Elastic provides the IT Manager and his team with the flexibility to address all aspects of monitoring. "For some databases we just read the logs. For others we can execute a query inside the database. We can also monitor our infrastructure to collect health and performance data from servers, databases, and other components in our tech stack," he says.

Elastic was also initially integrated with the company’s ServiceNow technical support management platform to open and assign incident tickets to the support team. Some of those processes are now automated using the Alert and Actions feature in Kibana. When a threshold is reached, Kibana invokes a Jenkins API to execute commands in the database and resolve anomalies without needing to open a ticket.

Another example of process automation is autoscaling, which automatically increases database storage before capacity is reached. Previously, database storage overruns resulted in about 150 incident tickets per month. An IT Architecture Analyst at the company explains that it could take a support agent an hour to resolve the issue. With Elastic, it now takes less than half a minute.

The IT Architecture Analyst also cites a process automation example involving an invoicing application that would stop working due to activity lapses. Inactivity would lock the database and prevent access to data. "Prior to using Elastic Observability and its automation tools, inactivity locks had to be removed manually. Now they are detected and fixed automatically before they can impact operations," he says.

With automated monitoring capabilities, the IT team can get ahead of incidents by proactively resolving many issues before they become noticeable to users. The IT Managersays, "With Elastic, we have fewer tickets, better detection, faster resolution, and more satisfied colleagues. It’s the ideal outcome."

He also sees Elastic playing a role in the company’s digital transformation initiative. The company is making a significant investment in the modernization and consolidation of its IT systems to increase efficiency and productivity while expanding growth. "Elastic is an important part of our digital strategy," says the IT Manager. "It’s helping us to reduce IT incidents, manual work, and overtime costs, which bring significant benefits to the overall organization."