Troubleshoot field mapping conflicts

A cross-project search datafeed merges search results across linked projects, so a field name must represent the same analytical type everywhere. When mappings disagree, Elasticsearch can log a report-only warning after a confirmed scope change, exclude one project from the current run, or refuse to start the datafeed, depending on whether the field is optional or the required time field.

Use these sources to gather diagnostic information:

• Anomaly detection job job messages in Kibana: Open Machine Learning → Anomaly Detection, select the job, and review the Job messages tab for audit entries and warnings about linked projects, credentials, or scope changes. On the Datafeed tab, View datafeed counts opens the datafeed chart flyout for extraction timing.

  The same entries are stored in .ml-notifications-*.

• GET _ml/datafeeds/{datafeed_id}: Shows the effective project_routing value and, when an internal cloud API key exists, authorization.cloud_api_key.id.

• GET _ml/datafeeds/{datafeed_id}/_stats: While the datafeed runs, shows remote_cluster_stats with total_clusters, available_clusters, skipped_clusters, availability_ratio, stabilized_cluster_aliases, and per_cluster_consecutive_skips. The object is absent until the first search cycle establishes a baseline.

• .ml-annotations-read: Scope-change annotations for the job. The annotation event field carries search_scope_changed (not the separate type field).

• GET /_project/tags: Lists linked projects and their tags so you can compare them with a routing expression.

• Elastic Cloud console: Review linked projects in Link and manage projects.

If extraction failures are ongoing, check Job messages first. remote_cluster_stats from get datafeed stats only updates after a cycle completes.

After project scope stabilizes following a link or routing change, Elasticsearch re-checks field capabilities and logs a report-only warning. The datafeed continues:

Cross-project field conflict for datafeed [my-datafeed]: field [status] has incompatible types across linked projects: keyword in [prod-us], long in [prod-eu]. Align index mappings across projects or narrow project_routing to projects with a consistent schema.
		

The bracketed datafeed id, field name, project aliases, and type detail vary with your configuration.

For optional fields, Elasticsearch groups types into analytical families and warns when a field's types span more than one family:

Types within the same family do not trigger a warning (for example long in one project and integer in another). Types from different families do (for example keyword and long). If any type is outside these families, Elasticsearch does not warn for that field.

Standardize the conflicting field to a compatible type in every linked project. Update index templates or reindex where needed.

When mappings cannot be aligned immediately, narrow project_routing to projects with a consistent schema. For routing syntax and stale-alias problems, see Project scope problems. Stop the datafeed and close the anomaly detection job before running this update.

				POST _ml/datafeeds/{datafeed_id}/_stop
				POST _ml/anomaly_detectors/{job_id}/_close
				POST _ml/datafeeds/{datafeed_id}/_update
					{
  "project_routing": "_alias:prod-us"
}
		

Replace the routing expression with one that omits the conflicting project while keeping the anomaly detection job's intended coverage elsewhere.

Job messages no longer report optional-field conflict warnings. _field_caps shows one compatible type per field across every project in scope.

The datafeed fails immediately when Elasticsearch builds the extractor (at start or preview time):

Cannot run datafeed [my-datafeed]: required time field [@timestamp] has conflicting types across projects in scope: date in [prod-us], long in [prod-eu]. Fix mappings so [@timestamp] uses the same type in every project in scope, or exclude the conflicting project(s) via project_routing.
		

For the time field, only date and date_nanos are treated as compatible.

Standardize the time field to a compatible type in every linked project (use date with a consistent format, or date_nanos everywhere). Update index templates or reindex where needed.

To exclude a conflicting project immediately, narrow project_routing as shown in Optional-field warnings after scope change.

Preview the datafeed in Kibana or with POST _ml/datafeeds/{datafeed_id}/_preview and confirm data returns from every project still in scope. The datafeed starts without the fail-fast error.

Index templates, ingest pipelines, or explicit mapping updates in one linked project can introduce conflicts that were not present at create time. A mapping rollout alone does not trigger Elasticsearch's field-conflict recheck. Symptoms of mapping drift include:

• Job messages report a new extraction error even though routing and credentials are unchanged.
• The datafeed fails to start or preview after a restart because the time field now conflicts at extractor build time.

Optional-field conflict warnings and time-field project exclusions appear only when Elasticsearch re-checks field capabilities after project scope stabilizes (for example after a project is linked or project_routing changes), not from a mapping change by itself.

Mid-run project exclusion after a scope change:

Datafeed [my-datafeed] excluded project [prod-eu] from this run: required time field [@timestamp] has conflicting types: date in [prod-us], long in [prod-eu]. Fix mappings in [prod-eu] to resume searching it, or remove it from project_routing.
		

Example extraction error after mapping drift:

Datafeed is encountering errors extracting data: Cannot parse field [status] of type [long] in document with id 'abc123'
		

The text after the colon is the underlying cause and varies (parse failure, missing field, incompatible type, and so on).

Run field capabilities on the indices the datafeed queries, one call per project in scope:

				GET _origin:logs-*/_field_caps?fields=@timestamp,status&include_unmapped
				GET prod-us:logs-*/_field_caps?fields=@timestamp,status&include_unmapped
		

Use the _origin: qualifier for the origin project and the linked project's alias for linked projects. In each response, inspect the field entry's type keys and compare them across calls using the compatibility families in Optional-field warnings after scope change.

Align mappings across projects as described in Optional-field warnings after scope change and Required time-field fail-fast.

After mappings stabilize:

1. Stop the datafeed.
2. Update the job query or aggregations if field names or types changed.
3. Preview the datafeed and confirm data returns from every project still in scope.
4. Start the datafeed.

• _field_caps shows one compatible type per field across every project in scope.
• Job messages no longer report field conflicts or extraction errors caused by mapping drift.
• Preview returns documents from each project matched by project_routing.