Evaluate and improve Knowledge Indicators

A useful Knowledge Indicator (KI) gives an agent accurate, relevant context without making it rediscover the same information from raw data. Inspect KIs after an automation runs and whenever the source data or intended use changes. When a generated KI needs improvement, refine its source or automation so later runs produce the improvement consistently.

You need an AI index that contains at least one generated KI, the Read privilege for the Context Engine feature, and the read and view_index_metadata index privileges on its backing index (ai-index-*). To refine an AI index's sources or automations, you need the All privilege for the Context Engine feature. To create an AI index and generate a KI, follow Get started with Context Engine.

You can inspect and test KIs in a managed AI index, but you cannot refine its sources or automations. The owning Elastic integration maintains that configuration.

Inspect KIs interactively in Kibana, or query them programmatically when you are evaluating an automated integration.

Follow these steps to inspect a generated KI:

1. Find Context in the navigation menu or use the global search field.
2. Open the AI index you want to inspect.
3. Select the Knowledge Indicators tab.
4. If the AI index contains several KI types, filter the list by type.
5. Select a KI to expand its JSON document.

Use the list AI indices API to find the AI index and its ES|QL query target. You can also use the describe AI index API to inspect its fields, KI types, tags, and example queries.

Run an ES|QL query with the query AI indices API to retrieve the KI or set of KIs you want to evaluate. Filter by fields such as id, type, or tags, and return the content, metadata, and provenance fields needed for the review.

The following fields help you determine what a KI represents, where it came from, and how an agent can use it:

For example, a KI derived from support cases might look like this:

{
  "id": "ki-support-feed-compaction-hot-tier",
  "type": "support.recurring_issue",
  "title": "Feed compaction stalls on the hot tier above 1,000 shards",
  "description": "A recurring cause of ingest backpressure in the reviewed cases.",
  "content": "The affected cases show a rising bulk rejection rate before compaction stalls.",
  "tags": ["tier:hot", "area:ingest"],
  "attributes": {
    "case_count": 14
  },
  "references": [
    {
      "uri": "index://support-cases",
      "relation": "derived_from",
      "description": "The support cases analyzed by the automation"
    }
  ],
  "expires_at": "2026-12-10T00:00:00Z",
  "governance": {
    "provenance": {
      "created_by": {
        "uri": "workflow://support-case-distiller",
        "metadata": {
          "version": 3,
          "run_id": "<workflow-run-id>"
        }
      }
    }
  }
}
		

KI types are free-form values. Use names that consistently identify what the documents represent. The generation strategy is a separate choice that controls which source records become KIs and how much information each KI contains. Refer to Select a generation strategy for common strategies.

Review the KI as context for an agent, not only as a valid document. Confirm that it:

• Answers a recurring question or helps an agent decide what to query next.
• Adds business meaning, derived findings, or usage guidance that a mapping alone cannot provide.
• Distinguishes facts derived from the complete dataset from observations based on a sample.
• States important limitations, missing data, and the questions it cannot answer.
• Includes references or verified query patterns when an agent might need current details.
• Contains enough information to be useful without repeating an entire source document.

Syntax and runtime verification show that a generated query parses and runs. They do not prove that its fields, grouping, and calculations answer the intended question. Review the query logic and its results before relying on it as guidance for an agent.

Use Kibana or the APIs to verify that later automation runs update the intended KI:

1. Record the KI's id, updated_at, and provenance run ID.
2. Run its automation again from the Workflows UI or with the run workflow API.
3. Open the KI from the Knowledge Indicators tab, or retrieve it again with the query AI indices API.
4. Confirm that the automation updated the KI with the same id instead of creating an unintended duplicate.
5. Confirm that updated_at, provenance, and any source-dependent content reflect the latest run.

If the automation intentionally creates one KI per event or source document, multiple KIs are expected. Stable IDs are most important when an automation maintains one KI for a dataset, entity, or other long-lived subject.

After you make the AI index available to an agent or application:

1. Ask a question the KI can answer from its generated content.
2. Ask a question that requires current details from the source.
3. Inspect the retrieval, source queries, and other tool calls made for each response.
4. Confirm that the agent retrieves the relevant KI, answers from it when appropriate, and queries the source only when it needs additional detail.

With Elastic Agent Builder, inspect the reasoning and tool calls shown with each response. If trace collection is available, you can also view the trace for the conversation round. For another agent framework or an application, inspect the equivalent client logs, traces, and API requests. Refer to Use Context Engine with agents and applications for the available integration routes.

This test shows whether the KI reduces repeated source exploration. If the agent ignores the KI, retrieves an unrelated KI, or still performs broad source exploration, revise the AI index description, KI content and tags, source selection, or automation instructions.

Compare traces for repeated or related questions. Look for broad source searches or repeated sequences of tool calls that better KI content or query guidance can avoid. These patterns show where moving recurring interpretation into the automation can save time and model tokens on later questions.

Do not rely on one-off edits to a generated KI. Update the source or automation so future runs reproduce the correction. Depending on what you find:

• Refine the source when it omits relevant data or includes unrelated data.
• Refine the automation instructions when the KI lacks meaning, boundaries, or useful retrieval guidance.
• Select a different generation strategy when each KI represents too much or too little source material.
• Adjust the schedule or expiration time when the KI becomes stale before its next refresh.
• Add verification when a generated query or structured attribute can be checked automatically.

Run the automation again, then repeat the content, refresh, and retrieval checks until the KI supports its intended questions. Treat this as a feedback loop: agent use exposes gaps, automation changes address them, and later runs regenerate the context with those improvements.