Secure services start with searchable data

As cyber threats increase, state and local governments remain prime targets. Fragmented systems limit visibility, and traditional approaches struggle to scale. Without timely insight, agencies risk falling short of both mission outcomes and public trust.

The pandemic accelerated this complexity. Remote work and digital services expanded rapidly, exposing gaps in infrastructure and increasing operational fragmentation. Agencies were forced to adapt their operations to create accessible services for citizens and employees. As systems became more decentralized, agencies lost a unified view of their environment, making it harder to monitor activity, detect issues, and respond effectively. 

Without modern approaches, visibility breaks down — and so does response.

To keep pace, agencies are moving toward a different model: one platform that unifies how data is collected, searched, and acted on.

Modern cybersecurity starts with search. It’s not just about storing data. It’s about making it instantly accessible, contextual, and actionable in real time.

State and local government agencies use the Elasticsearch Platform to unify search, observability, and security in one place. This creates a single, real-time view across systems, without needing to move or duplicate data.

With this foundation, agencies can detect anomalies earlier, investigate faster, and respond with confidence. Teams move from reactive workflows to proactive operations, using search to connect signals across environments and surface what matters most.

The result is not just better security — it’s better decision-making. Faster insights. Stronger resilience. And the ability to deliver reliable, secure services at scale.

For this US State Agency, shifting to remote work during the COVID-19 pandemic came with significant challenges. Moving to a distributed model exposed vulnerabilities in service delivery, monitoring, and cybersecurity in its previously centralized IT environment.

The agency urgently needed to strengthen its IT resilience, increase visibility, and meet the rising expectations for remote and digital services. To do this, the agency needed to modernize its IT infrastructure by consolidating monitoring tools and improving visibility into operational and cybersecurity data through a single pane of glass.

Supporting multiple departments and services across a complex IT environment, this US State Agency handles large volumes of sensitive data and depends on secure, streamlined performance.

However, its systems had been designed for a centralized workforce. Therefore, the pivot to a fully distributed environment prompted an urgent need to restructure its technology networks, systems, and infrastructure.

The primary obstacle facing this US State Agency was the lack of a unified view across operations. As digital demands surged and systems decentralized, different teams struggled to maintain a consistent understanding of operational data and performance. Without a comprehensive view, monitoring and troubleshooting became more complex, limiting the agency's ability to respond quickly to emerging issues.

Traditional transactional dashboards fell short, leaving teams reactive rather than proactive. Issues were often identified only after they had impacted employees or service delivery. The lack of visibility made it challenging to pinpoint problems and implement timely solutions, frustrating staff and constituents alike. Additionally, handling sensitive data increased pressure to maintain system reliability and trust.

“We looked at the technology issues we were trying to solve and realized that we needed to move away from transactional data dashboards, which make it hard to find issues, to ones that could handle more complex and streaming data, moving us toward being proactive,” said the chief digital and data officer.

In response to these challenges, the US State Agency recognized the need to modernize its IT infrastructure by consolidating monitoring tools and improving visibility into operational and cybersecurity data. It needed a solution that could integrate complex, dynamic data and provide a comprehensive operational view.

Faced with the challenges of a rapidly changing operational landscape, the US State Agency deployed Elastic Observability as its monitoring platform.

Migrating applications to Elastic Cloud Hosted on AWS helped break down silos that previously hindered data access and analytics. This migration made applications accessible to remote teams while providing the scalability needed to support a distributed work environment.

With Elastic Observability, the agency established a single pane of glass across its systems. Teams can now seamlessly search operational data, visualize system performance, and monitor systems in real time.

After struggling with reactive monitoring, Elastic allowed the agency to adopt a more proactive approach to problem-solving. Elastic enabled the agency’s teams to visualize performance metrics and gain insights into system health for quicker identification of potential issues.

Elastic’s comprehensive monitoring capabilities covered all aspects of the US State Agency’s IT processes. From data replication when transferring data from mainframes to various applications, to real-time performance tracking, Elastic provided the context needed to make faster, more informed decisions with end-to-end monitoring and streamlined operations.

The agency's partnership with Elastic Consulting proved invaluable during this transition. Through strategic guidance and support, the agency effectively designed its architecture to collect and analyze the critical data required for a successful deployment and continuous performance.

“Every month, we gain a deeper understanding of IT functions across the agency and can clearly see trends and anomalies in our systems. In these ways, the observability the Elastic platform provides has been a game-changer for our infrastructure resilience,” says the US State Agency CIO.

As a result of its modernization efforts, the US State Agency has significantly improved its operational efficiency and service reliability. It reported up to 80% improvement in efficiency thanks to automation, freeing up resources for more strategic initiatives.

For example, implementing Elastic Observability into the agency’s internal Total Cost of Ownership (TCO) app created a data dashboard that completes financial data analysis that would have taken over a week in a single day.

“With Elastic, the finance department is saving about a week’s worth of time each month on spending analysis,” says a consultant and technical lead with the US State Agency.  “With the transparency of data, questions can now be answered from information we couldn't see before.”

Thanks to Elastic Observability, the agency’s operational efficiency upgrades include:

• Centralized visibility across infrastructure and applications: Elastic Observability provides a unified view into infrastructure and applications, letting teams identify performance issues and opportunities for improvement.
• Increased efficiency through automation and shared dashboards: The agency transformed its operational processes with automation and shared dashboards to decrease manual reporting.
• Significant time and cost savings: Automation helped the State Agency save as much as a week of staff time per month.

By achieving centralized visibility across its IT landscape, the agency enhanced its ability to monitor and respond to service demands and focus on its core mission of delivering high-quality services to the public.

The Arizona Department of Homeland Security is a state agency that, charged with protecting private citizens and enforcing public safety, must maintain robust cybersecurity. AZDOHS plays a pivotal role in safeguarding the state's vital assets and ensuring the security of its information technology infrastructure.

AZDOHS supports multiple departments and services to ensure the resilience of Arizona's infrastructure against evolving threats. Dedicated to protecting the state’s critical infrastructure and enhancing public safety, AZDOHS implements many comprehensive cybersecurity measures and advanced technologies, all while attempting to safeguard large volumes of sensitive data at scale.

AZDOHS generates over 12 terabytes of daily logs. As data increases exponentially, so do threats. Identifying meaningful threats within massive volumes of daily security telemetry from multiple sources is overwhelming — especially when relying on traditional tools.

Monitoring a vast array of data points made it difficult to create efficient processes to handle the high volumes of data being ingested every day. Unlike other agencies, AZDOHS couldn’t simply store all its data. As a state agency in the public sector, timely analysis of data is crucial to identifying legitimate threats.

AZDOHS needed to centralize data ingestion to make its data searchable and actionable in real time and to identify and respond to cyber threats faster and more efficiently.

Looking to centralize security data ingestion, AZDOHS turned to Elastic Security for AI-driven security analytics.

By leveraging Elastic's prebuilt alerts, AZDOHS streamlined anomaly detection amid billions of data events. Their teams could customize the prebuilt models for their specific public sector needs for simple implementation with a large impact. AZDOHS can now focus on high-priority activities instead of being overwhelmed by irrelevant data.

Integrating Elastic’s AI-driven security analytics enabled AZDOHS to centralize data management and move toward more proactive security posture. With a more efficient data storage system, AZDOHS can more precisely search through billions of data events processed through the platform. Centralized data and AI insights help rapidly identify malicious activities and reduce alert fatigue from false positives.

"AI-driven insights were instrumental in elevating our cybersecurity posture,” said Austin Eakin, senior information technology engineer at AZDOHS. “The enrichment and correlation Elastic provided enabled us to leap into action immediately, shifting from mainly reactive measures to a more proactive stance."

Implementing Elastic’s solutions equipped the AZDOHS with the tools and insights needed to:

• Gain unified visibility of high data volumes: Deploying Elastic centralized AZDOHS’s security data, providing unified visibility across various data sources — enabling faster threat identification.
• Reduce alert noise and speed up investigations: Streamlined alert management means fewer irrelevant alerts, minimizing alert fatigue and allowing analysts to focus on genuine threats.
• Increase proactive threat detection and response: With enhanced data visibility, AZDOHS can proactively take preventive measures to address vulnerabilities before they escalate into critical issues, ultimately safeguarding Arizona's critical infrastructure.

By leveraging Elastic, AZDOHS has strengthened its ability to monitor, detect, and respond to threats at scale — even as cyber threats continue to evolve and more data funnels through its systems.

As an employment agency, Arbetsförmedlingen efficiently matches job seekers with employers and provides services such as information, analysis, and employment forecasts. As a major public sector agency, it relies on over 12,000 employees, including 1,000 dedicated IT specialists. Its operations span labor market services, unemployment benefits, and employment support programs for millions of Swedish citizens and employers.

Arbetsförmedlingen assists both job seekers and hiring organizations in navigating the complexities of the labor market. By analyzing employment trends and offering insights, Arbetsförmedlingen supports informed decision-making for stakeholders across Sweden.

The agency focuses on innovation to create a more user-friendly experience for individuals seeking employment. By continually adapting to changing regulations and new technologies, Arbetsförmedlingen supports the nation's workforce in a dynamic economy.

As a government agency responding to employment and market trends, Arbetsförmedlingen must adapt to evolving government regulations and operations. To meet these requirements, Arbetsförmedlingen digitalized its IT infrastructure to help communicate with citizens and businesses and connect job seekers with potential employment opportunities.

However, the agency faced significant obstacles related to data volume, system efficiency, and the need for improved visibility.

One of the most pressing challenges is managing large volumes of structured and unstructured data across multiple systems. As job market dynamics fluctuated, the agency needed to respond quickly, but its existing legacy systems struggled to keep pace with the increasing demands for accurate and timely data.

The agency also faced considerable technical debt due to deploying solutions from numerous vendors over the years. This fragmentation complicated integration, reduced operational efficiency, and increased costs.

"Our observability solution had long response times and was out of date. The APM platform was expensive and complicated, and we wanted to consolidate several search tools into one solution," said Tobias Ström, product owner, operation center.

The rising expectation for faster search capabilities and better analytics intensified these challenges. Analysts required quicker access to insights to inform decision-making, but traditional methods fell short in providing the speed and reliability necessary for effective responses.

Arbetsförmedlingen recognized the need for a modern IT infrastructure. The challenge was not just technical; it was also cultural, as employees accustomed to older systems had to adapt to new ways of working that could potentially streamline operations.

In light of these challenges, Arbetsförmedlingen had to find a solution that could simplify and enhance its operations. During the search for a resolution, Ronny Bergh, an IT specialist, discovered that some Java developers had transitioned to an open source version of Elasticsearch for system logging.

The agility of this deployment prompted the agency to reconsider its data management approach, ultimately enabling a significant increase in user capacity. By implementing Elastic Observability and Elasticsearch, the agency streamlined its data management capabilities, allowing teams to quickly analyze data, reducing the time spent on data retrieval.

Transforming data management was especially beneficial for internal analysts who rely on accurate, timely data to make informed decisions regarding job market trends and employment forecasts. With the ability to index and search through large datasets quickly, the agency increased its overall productivity without sacrificing security or quality.

Elastic also helped Arbetsförmedlingen overcome challenges posed by technical debt from its prior reliance on multiple vendor solutions. The unified Elastic platform streamlined its technology ecosystem and provided significant cost savings by reducing reliance on previous costly solutions while enhancing performance.

For a holistic view of its IT operations, the agency implemented an integrated platform for observability, search, and application performance monitoring (APM) to track and analyze data more effectively, improving decision-making and streamlining processes.

With Elastic, Arbetsförmedlingen reduced APM license and storage costs by 75%. Searchable snapshots enabled Arbetsförmedlingen to comply with data archiving standards while reducing dependency on costly storage.

"The use of searchable snapshots capability means that we can search infrequently accessed and read-only data in the frozen data tier, which reduces our dependency on expensive solid-state storage," says Stefan Jonsson, product owner at Arbetsförmedlingen.

With Elastic's machine learning capabilities, the agency could identify trends and behaviors in the job market, assisting policymakers in making informed decisions that impact employment strategies.

With the implementation of Elastic, the agency has made significant strides in improving its operations and service delivery and transformed its approach to data management with:

• Improved data access for analysts and decision-makers: Real-time access to comprehensive datasets allows analysts to draw insights and make informed decisions quickly.
• Faster, more flexible search: The agency can now execute faster searches across various complex datasets for higher productivity and efficiency.
• Data-driven insights to improve employment services: The ability to analyze trends effectively helps the agency proactively address labor market challenges and adapt to the needs of job seekers and employers alike.

The partnership with Elastic has brought about a transformative shift in Arbetsförmedlingen's operations, moving it away from fragmented legacy systems to a cohesive and integrated data management platform.

As one of the largest state departments in California, EDD administers essential benefit programs, including unemployment insurance, disability benefits, and Paid Family Leave. These programs provide crucial support to millions of Californians — from recovering from illness to navigating job transitions.

Millions of citizens mean millions of datasets, creating an ever-growing database across digital systems that support residents, employers, and partner agencies statewide.

The department was undergoing a multiyear modernization effort to help improve both customer and employee experiences. However, digitization could not come at the cost of ensuring that customers remained supported and protected across its systems and applications.

In a rapidly evolving cybersecurity landscape, EDD needed a solution that could quickly process, visualize, and analyze data.

The complexity of monitoring nearly 3,000 servers across various programs came with additional challenges. The varying systems and applications created data silos that obscured critical insights necessary for informed decision-making. Teams at EDD required faster investigation capabilities without introducing additional operational complexity. The challenge is compounded by the need for real-time insights, making centralized monitoring increasingly important.

EDD also needed improved visibility into application performance and system behavior. Teams must quickly identify service disruptions, performance bottlenecks, and security incidents to respond effectively and maintain operations. Traditional tools often could not analyze large-scale data efficiently.

Balancing accessibility and security was a key challenge. While it’s essential to have robust security measures in place, overly stringent protections can hinder genuine beneficiaries from accessing the aid they need. EDD understood that security mechanisms should not impede the provision of critical services, particularly in urgent situations where timely assistance is key.

This is especially important during periods of economic disruption when claims surge. Events such as economic downturns, natural disasters, or health crises can lead to an influx of claims, putting an immense strain on the department’s resources. As demand increases, maintaining service availability and security becomes crucial. The risks included potential exploitation by bad actors seeking to access personal information or submit fraudulent claims, necessitating multiple layers of protection.

To address the challenges of data management in complex, high-demand systems, EDD implemented Elastic Security, running on Elastic Cloud and AWS, to centralize observability and security insights across its environments. Elastic provided real-time visibility into application performance, critical threats, and suspicious activity.

By consolidating data from multiple sources, Elastic provides EDD with a comprehensive view essential for swift decision-making and effective incident management. With Elastic’s powerful infrastructure hosted on AWS, the department can leverage the scalability and reliability needed to support its operations, particularly during surges in service demand.

"Moving to Elastic Cloud on AWS speeds up performance for the security team, eliminating downtime and providing faster search and analysis of data,” says Douglas Leone, chief information security officer. “EDD currently has over 850 billion records in Elastic, and even as data volumes grow, performance remains strong."

Advanced dashboards provide clear visibility into traffic patterns, alerts, and overall system behavior, making it easier to identify potential vulnerabilities and prioritize action, shifting from reactive measures to a more proactive stance.

Elastic's AI-driven features, such as Attack Discovery, help teams prioritize alerts based on their severity. Security teams can focus on genuine threats while significantly reducing the time taken to detect and respond to security incidents.

The partnership with Elastic consulting provided customized support, including training and custom visualizations, helping teams fully adopt and optimize the platform. This collaboration facilitated a seamless onboarding experience and reinforced the department's commitment to maintaining security and efficiency across its operations.

By implementing Elastic alongside AWS, EDD improved operational efficiency while strengthening data protection and service reliability.

EDD improved its ability to manage high-demand systems and protect critical services, allowing the agency to better serve millions of Californians while protecting sensitive data.

Integrating Elastic Observability has helped:

• Unify visibility across distributed systems: EDD can now monitor applications and infrastructure components in real time, identifying potential issues and maintaining critical services.
• Speed up detection and resolution: Real-time insights into application performance and system health reduce the average time needed to identify and resolve anomalies.
• Strengthen monitoring and security: Advanced dashboards and AI-driven features enable proactive monitoring.
• Improve operational resilience during high demand: Elastic’s scalability, combined with AWS, allows EDD to handle surges in claims and inquiries.

By improving visibility, accelerating response times, and strengthening monitoring capabilities, EDD is better positioned to provide reliable and effective services to millions of residents.