How our agent skills research, build, and test a complete Elastic integration

Elasticsearch has native integrations to industry leading Gen AI tools and providers. Check out our webinars on going Beyond RAG Basics, or building prod-ready apps Elastic Vector Database.

To build the best search solutions for your use case, start a free cloud trial or try Elastic on your local machine now.

Agents can now build an Elastic integration end to end. The Elastic Integration Skills encode Elastic’s best practices for research, package scaffolding, Common Expression Language (CEL) development, ingest pipelines, testing and review. They break the work into small units that each produce a validated result and combine to form a full, working integration.

We pointed them at the NIST National Vulnerability Database API and got two data streams with validated CEL programs, with pipeline and system tests passing. The package backfilled a week of data in a few minutes, collecting 7,408 CVE snapshots and 12,339 change events.

A previous post on the CEL input assumed you’d write the CEL yourself. That's no longer necessary, so you can focus on other parts of the process. You decide what gets built, and then you decide whether the evidence is good enough to trust it.

How our agent skills split the work between specialist agents

A full integration is built by orchestrating the work of specialist agents. An orchestration agent session dispatches subagents for each phase of development: research, CEL program, ingest pipeline, system tests, and review. Each starts by reading its instructions out of the skills before touching anything. The orchestrator delegates most work to subagents; it passes context in and reads back reports.

You work with the orchestrator to provide context for the subagents. As they work, you won't follow the individual steps they take, such as generating CEL code; instead, you receive a report that the code passed validation. Then you decide what to inspect in the output files.

For best results, use a top-tier reasoning model, such as Claude Opus or GPT-6 Astra. Install all the dependencies according to the instructions. Keep the Integration Skills up to date, as they'll be improved over time. 

Start with research into the API

There are two top-level skills: /research-integration and /create-integration. Comprehensive research at the outset sets you up for success in the creation phase. The research skill investigates the API's documentation and live behavior, collects sample data, and organizes what it learns into a research brief for the creation phase. The best way to shape the final result is to iterate on the research output.

A good research prompt briefly expresses your intent and refers to resources that are specific to the integration:

/research-integration NIST National Vulnerability Database (NVD) API.
API docs: https://nvd.nist.gov/developers/vulnerabilities
Data to collect: CVE records from the NVD.
The integration must backfill a configurable amount of history on first run,
then stay up to date by polling for new and changed records.
Use it without an API key by default, with an optional API key setting.
Integration slug: nist_nvd
Research output directory: research_results/nist_nvd/

That prompt produced various artifact files and a report (a short summary for you). The most important artifact is research-brief.md, which is the specification passed from the research stage to the creation stage. It captures API behavior, data semantics, proposed mappings, decisions already made, and outstanding questions.

The research brief also refers to another important artifact, test-api.py, which is a dependency-free script that exercises the exact proposed collection flow, built against a live API (when possible) or against a mock. You can run it yourself to validate assumptions about the API and to see the proposed collection logic in action. It serves as an executable description of the collection behavior that an agent will translate into CEL code. This tends to work well because, for current coding models (and most developers), Python is a much more familiar language than Mito CEL.

usage: test-api.py [-h] [--url URL] [--api-key API_KEY]
                   [--endpoint {cves,history}]
                   [--initial-interval INITIAL_INTERVAL]
                   [--backfill-filter {lastmod,pub}] [--page-size PAGE_SIZE]
                   [--max-pages MAX_PAGES] [--no-rejected] [--sleep SLEEP]
                   [--timeout TIMEOUT] [--proxy PROXY]
                   [--output-dir OUTPUT_DIR] [--mock]

NIST NVD API connectivity & flow test (mirrors the proposed CEL collection flow)

options:
  -h, --help            show this help message and exit
  --url URL             Base API URL including scheme (default:
                        https://services.nvd.nist.gov; env NVD_URL)
  --api-key API_KEY     Optional NVD API key, sent as `apiKey` header (env
                        NVD_API_KEY)
  --endpoint {cves,history}
                        cves = CVE API (lastModified windows); history = CVE
                        Change History API (default: cves)
  --initial-interval INITIAL_INTERVAL
                        Backfill look-back from now, e.g. 6h, 48h, 7d
                        (default: 24h). Sliced into <120-day windows.
  --backfill-filter {lastmod,pub}
                        cves only: date filter for the backfill windows
                        (default: lastmod). Incremental always uses lastmod.
  --page-size PAGE_SIZE
                        resultsPerPage (default: 2000; API max 2000 for cves,
                        5000 for history)
  --max-pages MAX_PAGES
                        Max pages per window (default: 5)
  --no-rejected         cves only: send the bare `noRejected` flag
  --sleep SLEEP         Seconds between requests (default: 7 without key, 1
                        with key)
  --timeout TIMEOUT     HTTP timeout in seconds (default: 60)
  --proxy PROXY         Proxy URL (env HTTPS_PROXY)
  --output-dir OUTPUT_DIR
                        Output directory (default: test-api-output)
  --mock                Mock mode: skip archiving, print output dir path

Although I asked for straightforward Common Vulnerabilities and Exposures (CVE) record collection, the research phase revealed some additional complexities to consider. A CVE record in the NVD is republished each time the CVE Numbering Authority (CNA), Cybersecurity and Infrastructure Security Agency (CISA), or NIST's own analysts change it, rather than just once, and those changes mostly arrive a day or more after publication. When I ran the script against a one-day time range, 78% of the records it collected were updates to CVEs published before the start of the range. The NVD also offers two APIs for keeping up to date: one returns the current state of each changed record, and the other returns a log of the individual changes.

The republication pattern needed more investigation. I discussed it further with the agent and asked it to:

• Describe the record lifecycle and the two APIs in more detail.

• Characterize how much records actually change over time. (It classified a day’s change events as either issues that require action from a security team or housekeeping tasks.)

• Check the viability of some integration design options.

Since many of the republications change something that a security team would act on, such as a Common Vulnerability Scoring System (CVSS) score, a Common Weakness Enumeration (CWE), or a Known Exploited Vulnerabilities (KEV) listing, I decided to collect comprehensive information: one data stream of record snapshots and another of change events. The two streams answer different questions:

The viability check showed that two streams polling without an API key would exceed the shared rate limit during a backfill, so the requests have to be paced. To explain the record lifecycle and the collection approach, I asked the agent to include a detailed discussion of them in the integration's README. It incorporated all of these decisions into the research brief and supporting artifacts. The research and my work at this stage had a big impact on my understanding of the API and the final form of the integration.

Mapping NVD data to the Elastic Common Schema

The research also produced an Elastic Common Schema (ECS) mapping analysis, which needed a decision of its own. ECS has a vulnerability field set, but it can represent only one CVSS assessment at a time, while an NVD record can carry CVSS 2.0, 3.0, 3.1, and 4.0 metrics from multiple sources that may disagree. The agent laid out the options and left the selection rule to me. The integration picks a single assessment in this order:

1. CVSS 3.1

2. CVSS 4.0

3. CVSS 3.0

4. CVSS 2.0

Within a version, Primary metrics are preferred over Secondary. All CVSS metric arrays are preserved under the integration’s own fields, so nothing is lost by the selection.

The agent recommended event.kind: event for the records. The existing cisa_kevs and first_epss integrations use enrichment for similar vulnerability reference data, so I chose that for consistency.

Run /create-integration to build the Elastic integration

With all the relevant information and decisions captured in the research artifacts, the project can be passed to the /create-integration skill. Since the collected sample events contain no sensitive data, they can be included as test examples:

/create-integration @research_results/nist_nvd/research-brief.md
@research_results/nist_nvd/references/sample-events/

The agent worked for a while and then reported back:

The nist_nvd integration is built, tested, and reviewed: both data streams pass their pipeline and system tests, and every review finding has been fixed or closed with evidence.

Did it really build what I wanted?

How to review a generated Elastic integration

After the creation phase, you likely have a working but imperfect integration. The agents did a lot of work, but it's up to you to determine whether they did it right.

Read the orchestrator’s report and the package layout 

Start by reading the orchestrator's report, which gives you a good overview of the work done. It links to subagent threads that you can use to ask topic-specific follow-up questions.

The integration review file covers a range of valuable observations, such as the tradeoffs of certain field mappings or the fact that the logo file is a placeholder. Have a look at the general package layout, as well, including the folders and files that make up the integration. In this example, there are two data streams, each with their own CEL programs and pipelines. The API URL and key are defined once at the package level rather than repeated in each data stream.

What the README reveals about the agent’s understanding

The README file is worth a look, too, because it gives you insight into what the agent thinks the integration means. In this case, the record lifecycle, the NVD's 2026 change to its enrichment policy, and what each data stream represents are explained concisely, along with the Elasticsearch Query Language (ES|QL) query that gets the current state of a CVE from a data stream of snapshots. The structured presentation of the relevant ideas with supporting details gives good insight into the domain and the operation of the integration, and it can easily be slimmed down if it’s suitable for the integration's users. Any conceptual misunderstandings in the README should prompt a closer review of the implementation logic.

Question the CEL code

The CEL programs are critical artifacts which are best understood by talking to the specialist subagent that produced them. While reviewing the code, you can ask questions such as:

• Why exactly is the backfill window calculated that way?

• Walk me through the requests made from a fresh start until it reaches steady state polling.

• What happens when a poll returns no events? Does the cursor still advance?

• On an error, does the integration retry the same window or skip it?

The CEL specialist builds CEL programs much like human programmers do, except that it's forced to take smaller steps and validate more rigorously as it progresses. First, it sets up a mock server. Then, the expression-writing subagent uses Mito to run the evolving CEL program against the server, adding functionality in predefined stages, including the basic skeleton and error handling, along with returning events. Only small amounts of code are generated between validation steps. The tools used by the agent during its development can also help it provide explanations or analysis of alternatives, if you challenge its approach during review.

What the tests prove, and what they don’t 

The tests built into the integration package are some of the strongest evidence that the integration works as it should, as long as they cover the cases you care about. By the time you review the result, the agents will have fixed any failing tests. The important question, then, is whether the tests exercise the cases you care about, not simply whether they pass. Tested behavior gives you good evidence of correctness, but untested behavior is uncertain.

Pipeline tests take raw events representative of those produced by the CEL input or another source and simulate ingesting them into Elasticsearch. They check whether  an output event matches an expected event document and the integration's field mappings after processing by the ingest pipeline. If you don't see the output you want or coverage of the event types you care about, talk to the pipeline builder subagent about improving the pipeline logic or extending the coverage.

The pipeline tests also enforce ECS. I had asked for event.type: creation for new vulnerabilities and change on the change events, but ECS only allows info alongside the vulnerability category. The pipeline specialist came back with the failing test rather than the mapping I asked for, and the create/update distinction moved to event.action, which holds the NVD's own action names.

System tests exercise the end-to-end flow; that is, the package is set up with test configuration and run in a real Elastic Agent, some events are collected, and they're indexed in Elasticsearch. The tests assert that an expected number of events are ingested without error. For most integrations, they should cover collecting data across multiple requests, including an initial backfill and polling for new data. If that's not covered or if your data source presents more complicated scenarios to handle, you should explore additional testing.

Currently, /create-integration relies on these two test types, although more isolated or more complicated testing is possible with policy tests and script tests. Be aware, however, that no automated tests can fully validate that:

• Your interpretation of the upstream API and its data is correct.

• The integration's default settings are sensible.

• The documentation answers the questions that users are likely to have.

• The real data is actually useful.

Another important way to validate a new integration is to install and configure it and then manually inspect the data it collects. The NVD integration backfilled a week of data (7,408 snapshot records and 12,339 change events) in a few minutes without an API key. The change events show the pattern that the research described, with new CVEs outnumbered by modifications to existing ones and with event volume dropping on weekends.

What to add after the agents finish

The NVD integration works and incorporates all of the lessons learned during research and further iteration on the brief. Getting to this point required only some prompting and processing time, a substantial productivity improvement over building the package manually. However, it's still not perfect and not necessarily complete.

I'd add a latest transform so that users get an index with one current document per CVE, rather than relying on a query over every stored version. The error path, which the API mocking already serves, could be covered by script tests. The generated package still has a placeholder logo and no dashboard. These issues don't undermine the correctness of what was built; instead, they represent potential refinements to improve the user and developer experience. The generated integration is a solid implementation, ready for those refinements. If more data needs to be collected, the /create-integration skill is the best way to add a data stream.

What agent skills automate, and what stays yours

The Integration Skills can increasingly automate implementation, but you still need to own the process. The workflow gives you two places to do that. Before the build, you decide what kind of integration you want and settle the design questions that matter in the research brief. After the build, you read the reports, the README, the CEL program, and the tests and then decide whether that evidence is enough to trust the result.