Troubleshoot cross-project search datafeeds

Anomaly detection datafeeds on Elastic Cloud Serverless can search data across linked projects when cross-project search is configured. These topics help you diagnose and resolve problems with project scope (project_routing), internal cloud credentials, linked-project availability, and field mappings.

Before you troubleshoot, confirm that projects are linked and that users have access. See Link and manage projects and the Cross-project search overview.

Tip

If you can't find your issue here, explore the other troubleshooting topics or contact us.

Use these sources to gather diagnostic information:

  • Anomaly detection job job messages in Kibana: Open Machine Learning → Anomaly Detection, select the job, and review the Job messages tab for audit entries and warnings about linked projects, credentials, or scope changes. On the Datafeed tab, View datafeed counts opens the datafeed chart flyout for extraction timing.

    The same entries are stored in .ml-notifications-*.

  • GET _ml/datafeeds/{datafeed_id}: Shows the effective project_routing value and, when an internal cloud API key exists, authorization.cloud_api_key.id.

  • GET _ml/datafeeds/{datafeed_id}/_stats: While the datafeed runs, shows remote_cluster_stats with total_clusters, available_clusters, skipped_clusters, availability_ratio, stabilized_cluster_aliases, and per_cluster_consecutive_skips. The object is absent until the first search cycle establishes a baseline.

  • .ml-annotations-read: Scope-change annotations for the job. The annotation event field carries search_scope_changed (not the separate type field).

  • GET /_project/tags: Lists linked projects and their tags so you can compare them with a routing expression.

  • Elastic Cloud console: Review linked projects in Link and manage projects.

If extraction failures are ongoing, check Job messages first. remote_cluster_stats from get datafeed stats only updates after a cycle completes.

Start with the symptom that best matches what you see:

Symptom Start here Notes
The datafeed returns no results Project scope problems —
Results come only from the origin project Project scope problems If project_routing is _alias:_origin or the job has no stored routing, this is expected legacy behavior.

If authorization.cloud_api_key.id is missing or job messages report a cleared or never-minted key, refer to Cloud credential problems instead.
Extraction cycles are suddenly slower after you linked projects Project scope problems —
The datafeed keeps failing with extraction errors Linked project unavailable Use this page when job messages report a skipped linked project. For authorization failures, refer to Cloud credential problems. For field type conflicts, refer to Field mapping conflicts.
Elasticsearch or Kibana rejected a project scope change Project scope changes —
Some jobs failed during a bulk Change project scope update Project scope changes —
Anomaly scores spiked after a scope change Project scope changes —
A field is missing, a project is excluded from a run, or mappings conflict across projects Field mapping conflicts —
Authorization errors after the datafeed had been working Cloud credential problems —

If you already have an error or audit string from Job messages, the API, or Kibana, use this index:

Message (substring match) Page
matched no linked project / cannot search any project Project scope problems
remote clusters out of / were skipped when performing datafeed search Linked project unavailable
Cannot update project_routing / while its status is started Project scope changes
Rollback model snapshot retained before project_routing scope change Project scope changes
CPS migration: project_routing defaulted Run a job > Legacy jobs
Datafeed search scope changed / Elevated anomaly scores detected after search scope change Project scope changes
Internal cloud API key / Datafeed search probe failed / User lacks the required permissions Cloud credential problems
Failed to revoke internal cloud API key Cloud credential problems
Cross-project field conflict / conflicting types across projects / excluded project from this run Field mapping conflicts
Cannot run datafeed + required time field Field mapping conflicts
Datafeed has recovered data extraction / started retrieving data again Linked project unavailable

For valid project_routing values and wildcard rules, refer to Project routing in CPS.

Contact Elastic support when:

  • A linked project or region appears unavailable across multiple jobs and you have confirmed project linking in Elastic Cloud.
  • The same datafeed fails repeatedly after you apply the fixes in these topics.
  • The origin project reports memory pressure or out-of-memory errors while CPS datafeeds are running, and narrowing project_routing does not relieve the symptoms.
Edit page