Turning data into mission value in government and education

By

Leanne Link

elastic-blog-header-govt.png

Government and education leaders estimate that data volume at their organizations will increase by 59% over the next three years. Although having more information than you need is (arguably) better than not having it when you need it, the sheer volume of data can make it challenging for teams to pinpoint exactly what data will bring value to their mission goals.

Elastic and ThoughtLab recently released a global study that focuses on how organizations are extracting value from data — data from sources including cybersecurity (SIEM), logs, administrative, IoT, application traces, and more. Out of all industries included in the study, public sector ranks the lowest in what ThoughtLab refers to as “data maturity” — based on a framework of best practices, such as data operations, culture, and governance.

public sector bar graph

Public sector data challenges amid increasing volume

Government and education leaders say that the data-related challenges their organizations are facing include how to effectively store, access, and use data that’s meaningful to their missions.

  • Storing and managing mission data. According to the Elastic and ThoughtLab research, 45% of public sector organizations say they are struggling to manage the volume of data they collect. This issue is compounded by log storage requirements such as M-21-31 in the US, which outlines how long agencies must store certain types of data. Meeting M-21-31 requirements can become expensive, especially over time. 
  • Accessing mission data. Many government agencies are focusing on log storage, but another problem arises when they need to access this older (or “historical”) data from storage. Many data platforms make it difficult and/or time-consuming to access data in these historical storage tiers. In fact, half of public sector leaders surveyed say that availability of needed data is a top challenge at their organization.
  • Using mission data. In addition to storing and accessing large volumes of data, there’s also the pressing issue of how government and education organizations are putting that data to use. 31% of public sector leaders say they’re not utilizing data optimally across departments. Often that’s because data sits in siloed databases and technology solutions, and teams lack a single source of data truth for collaboration, analytics, and data sharing.
percentage

How government and education leaders can get more value from data

Unify data for AI use cases

Artificial intelligence (AI) is poised to reshape customer experiences, military intelligence, digital services, DecSecOps, and much more in the coming months and years. Collectively, public sector is still early in AI adoption: 80% of government organizations are at the initial or developing digital maturity stages. Even if your organization isn’t ready to incorporate AI into your systems and processes right now, it’s worth teeing up your data strategy to be ready when you are.

The value of generative AI for public sector comes when organizations integrate their own internal data with large language models (LLMs). Publicly accessible AI platforms like ChatGPT are trained on publicly available data, so they’ll only be as relevant as that information. On the other hand, generative AI output has the potential to become hyper-relevant and personalized when integrated with an organization’s own proprietary data. 

This means that in order to get the most value out of generative AI outputs, you don’t want to have to manually correlate disparate data sets in order to get a full picture of your information landscape. Having all your data in one place avoids silos, duplication, and makes it easier to quickly and accurately use data to extract value from generative AI.

Ask your data to do more for you

Many public sector agencies are being asked to do less with more and are prioritizing tool consolidation to increase efficiency and costs. If you’re working on gaining more value from your data — and unifying your data — this can provide a solid foundation for other use cases.  

Many organizations are leveraging the speed and scale that a search-based data platform can provide, without adding additional vendors that they then need to onboard, manage, and pay for.

For example, your data platform can also provide application and infrastructure performance monitoring. That same data foundation — especially when it operates quickly at scale — can also power your cybersecurity strategy, leveraging data logs to quickly detect and respond to threats.

Mission data done right

Data and analytics has the potential to create around $1.2 trillion in value per year in public sector, according to the McKinsey Global Institute. Government and education leaders say that becoming a “data-enabled enterprise” will lead to better customer experience, greater employee productivity, and increased revenue. Public sector organizations are headed in the right direction with their continued emphasis on reducing data silos, providing easy access to both historical and current data, and making their data actionable for their missions. 

Get more details from the ThoughtLab/Elastic study:

Sign up for Elastic Cloud free trial

Spin up a fully loaded deployment on the cloud provider you choose. As the company behind Elasticsearch, we bring our features and support to your Elastic clusters in the cloud.

Start free trial