Machine learning rules

Machine learning rules generate alerts when a machine learning anomaly detection job discovers an anomaly that exceeds a defined score threshold. Unlike other rule types, machine learning rules do not require you to write a query. Instead, they rely on machine learning jobs that continuously model normal behavior and flag deviations.

Machine learning rules are the right fit when:

• You want to detect behavioral anomalies that are difficult to express as static queries, such as unusual login times, atypical data transfer volumes, or rare process executions for a given user or host.
• A machine learning job is already active (or you plan to enable one) that models the relevant behavior.
• You want adaptive detection that automatically adjusts to changing baselines without manual threshold tuning.

Machine learning rules are not the best fit when:

• You can define the exact pattern you're looking for. Use a custom query rule or EQL rule instead.
• You need a count-based threshold. Use a threshold rule instead.
• You need to compare events against threat intelligence. Use an indicator match rule instead.

Machine learning rules require the appropriate Elastic Stack subscription or Serverless project feature tier, and you must have the machine_learning_admin role in Elastic Stack or the appropriate user role in Serverless. You also need at least one machine learning anomaly detection job active or configured to start.

For an overview of using machine learning with Elastic Security, refer to Anomaly detection.

Machine learning rules depend on their associated machine learning jobs running continuously. If a job stops, the rule cannot evaluate anomaly results and won't generate alerts. You can manage job status in two ways: from the machine learning jobs setting UI or the rule's details page.

Machine learning jobs associated with Elastic Security prebuilt rules are managed in a dedicated interface:

1. Find Detection rules (SIEM) in the navigation menu or use the global search field.
2. Select the settings menu in the upper-right corner of the page, then select ML job settings.

From there you can view, start, and stop all machine learning jobs associated with prebuilt detection rules.

You can also check and control job status for a specific rule:

1. Find Detection rules (SIEM) in the navigation menu or use the global search field, and select the rule name in the Rules table.
2. In the Definition section of the rule's details page, check whether the required machine learning jobs are running.
3. Use the toggles to start or stop each job.

If a required machine learning job isn't running, an error indicator appears in the Last response column of the Rules table. Select the indicator to see which jobs are affected.

The following examples use the detections API request format to show how machine learning rules are defined. Each example is followed by a breakdown of the machine learning-specific fields. For common fields like name, severity, and interval, refer to the detections API documentation.

This rule monitors a rare-process machine learning job for anomalies on Linux hosts.

{
  "type": "machine_learning",
  "name": "Anomalous process execution on Linux",
  "description": "Alerts when a machine learning job detects a rare process execution on a Linux host.",
  "machine_learning_job_id": "v3_linux_anomalous_process_all_hosts",
  "anomaly_threshold": 75,
  "severity": "low",
  "risk_score": 21,
  "interval": "15m",
  "from": "now-16m"
}
		

This rule monitors several login-related machine learning jobs and suppresses repeated alerts by user.

{
  "type": "machine_learning",
  "name": "Unusual login activity detected",
  "description": "Alerts when any login-related ML job detects anomalous authentication behavior.",
  "machine_learning_job_id": [
    "auth_high_count_logon_events",
    "auth_rare_source_ip_for_a_user",
    "auth_high_count_logon_fails"
  ],
  "anomaly_threshold": 50,
  "severity": "medium",
  "risk_score": 47,
  "alert_suppression": {
    "group_by": ["user.name"],
    "duration": { "value": 1, "unit": "h" },
    "missing_fields_strategy": "suppress"
  },
  "interval": "15m",
  "from": "now-16m"
}
		

The following settings appear in the Define rule section when creating a machine learning rule. For settings shared across all rule types, refer to Rule settings reference.

Machine learning jobs

The anomaly detection jobs whose results the rule evaluates. Select one or more jobs. If a selected job is not active, it starts automatically when the rule is enabled.

Anomaly score threshold

The minimum anomaly score (0-100) above which the rule generates alerts. Only anomalies that meet or exceed this score trigger alerts.

Suppress alerts by (optional)

Reduce repeated or duplicate alerts by grouping them on one or more fields. Only anomaly fields are available for suppression because machine learning alerts do not contain source event fields. For details, refer to Alert suppression.