Skip to main content

AI can do what now?! How AI is fixing security's worst bottleneck

By
Elastic Security Team
blog-bottleneck_(1).png

As a security professional, you probably didn’t get into this field because you love normalizing log formats or writing parsers. Yet, that’s exactly what many valuable defenders get stuck doing every day — wading through CSVs, flat text files, and homegrown log schemas just to get data into a usable format.

Anyone who’s worked in a security operations center (SOC) knows that onboarding data means dealing with a wide variety of sources, each with its own terminology and format. Whether it's structured or unstructured data, every technology speaks its own language, and analysts are expected to translate them all into a common, normalized format before any threat detection, anomaly hunting, or dashboard building can occur.

But the reality is that data onboarding is an essential prerequisite to effective cybersecurity operations. The good news is AI is finally alleviating teams of this manual, time-intensive, and error-prone process.

Reimagine data onboarding with AI

So, how exactly can AI help? In the recently released episode of AI can do what now?!, Anas Khatri, security solutions architect at Elastic, shares how large language models (LLMs) can now do much of the burdensome onboarding. They automatically normalize and enrich data across almost any format (e.g., CSV, JSON, flat file, and proprietary schemas) for ingestion into a security information and event management (SIEM) platform or other security analytics platform.

This shift isn’t theoretical; it’s happening today. Security platforms like Elastic are embedding these AI capabilities directly into the ingestion process, turning what used to take hours and days into a task that can be completed in minutes.

“Onboarding is really the first stage,” Khatri explains in the episode. “Dashboarding, visualization, detection engineering, threat hunting, or even custom queries that you’re going to make are going to be based on the data you’re ingesting. So, it’s very, very important for us that we ingest the data in the right way, and we make sure the data is enriched with the right information.”

There are always going to be in-house tools, niche platforms, legacy systems, and one-off integrations that create complexities beyond anything an AI feature can disentangle through — that’s just the reality of complex, diverse environments. But where diverse data sources can be normalized, actionable outcomes include:

  • Faster time to detection

  • Better data quality and consistency

  • More reliable dashboards and threat hunts

  • Quicker response to emerging attack surfaces

In short, if your log data is normalized, enriched, and available in near real time, then you can get back to the work of a security analyst — detecting intrusions and responding to incidents.

From manual to modern

If you’re spending hours writing custom scripts, mapping fields, or troubleshooting ingest issues, you don’t have to anymore. AI can do the heavy lifting, and that means you can get back to doing what you do best: protecting your organization.

Data onboarding isn’t just something that occurs during SIEM deployment or migration. It’s a continuous effort. Every time your business adds a new tool, replaces an old one, or creates a new function, there’s more data that needs to be brought in and normalized. It’s an ongoing task that evolves as the organization grows. This operational pain point persists beyond the initial SIEM setup, which is what makes features like Elastic Security’s Automatic Import so exciting.

“Gone are the days when we used to write regexes and program script to parse data,” says Khatri. “It’s time we jump directly into the fun part and start doing what we’re actually supposed to be doing."

Check out the AI Can Do What Now?! - Data onboarding episode to learn how AI is reshaping the most painful part of security workflows and how you can use it today.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

In this blog post, we may have used or referred to third party generative AI tools, which are owned and operated by their respective owners. Elastic does not have any control over the third party tools and we have no responsibility or liability for their content, operation or use, nor for any loss or damage that may arise from your use of such tools. Please exercise caution when using AI tools with personal, sensitive or confidential information. Any data you submit may be used for AI training or other purposes. There is no guarantee that information you provide will be kept secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative AI tools prior to use. 

Elastic, Elasticsearch, and associated marks are trademarks, logos, or registered trademarks of Elasticsearch N.V. in the United States and other countries. All other company and product names are trademarks, logos or registered trademarks of their respective owners.

Sign up for Elastic Cloud free trial

Spin up a fully loaded deployment on the cloud provider you choose. As the company behind Elasticsearch, we bring our features and support to your Elastic clusters in the cloud.

Start free trial