Appendix H: Security anomaly detection configurationsedit

These anomaly detection jobs automatically detect file system and network anomalies on your hosts. They appear in the Anomaly Detection interface of the Elastic Security app in Kibana when you have data that matches their configuration. For more information, refer to Anomaly detection with machine learning.

Security: Auditbeatedit

Detect suspicious network activity and unusual processes in Auditbeat data.

In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the manifest file. In the Elastic Security app, it looks in the data view specified in the securitySolution:defaultIndex advanced setting for data that matches the query.

In 7.11 or later versions, use the Security: Linux jobs instead.[1]

Name Description Job Datafeed

linux_anomalous_network_activity_ecs

Looks for unusual processes using the network which could indicate command-and-control, lateral movement, persistence, or data exfiltration activity.

A link icon

A link icon

linux_anomalous_network_port_activity_ecs

Looks for unusual destination port activity that could indicate command-and-control, persistence mechanism, or data exfiltration activity. NOTE: This job is available only when you use Auditbeat to ship data. [2]

A link icon

A link icon

linux_anomalous_network_service

Looks for unusual listening ports that could indicate execution of unauthorized services, backdoors, or persistence mechanisms. NOTE: This job is available only when you use Auditbeat to ship data.[2]

A link icon

A link icon

linux_anomalous_network_url_activity_ecs

Looks for an unusual web URL request from a Linux instance. Curl and wget web request activity is very common but unusual web requests from a Linux server can sometimes be malware delivery or execution.

A link icon

A link icon

linux_anomalous_process_all_hosts_ecs

Looks for processes that are unusual to all Linux hosts. Such unusual processes may indicate unauthorized services, malware, or persistence mechanisms.

A link icon

A link icon

linux_anomalous_user_name_ecs

Rare and unusual users that are not normally active may indicate unauthorized changes or activity by an unauthorized user which may be credentialed access or lateral movement.

A link icon

A link icon

linux_network_configuration_discovery

Looks for commands related to system network configuration discovery from an unusual user context. This can be due to uncommon troubleshooting activity or due to a compromised account. A compromised account may be used by a threat actor to engage in system network configuration discovery in order to increase their understanding of connected networks and hosts. This information may be used to shape follow-up behaviors such as lateral movement or additional discovery.

A link icon

A link icon

linux_network_connection_discovery

Looks for commands related to system network connection discovery from an unusual user context. This can be due to uncommon troubleshooting activity or due to a compromised account. A compromised account may be used by a threat actor to engage in system network connection discovery in order to increase their understanding of connected services and systems. This information may be used to shape follow-up behaviors such as lateral movement or additional discovery.

A link icon

A link icon

linux_rare_kernel_module_arguments

Looks for unusual kernel modules which are often used for stealth.

A link icon

A link icon

linux_rare_metadata_process

Looks for anomalous access to the metadata service by an unusual process. The metadata service may be targeted in order to harvest credentials or user data scripts containing secrets.

A link icon

A link icon

linux_rare_metadata_user

Looks for anomalous access to the metadata service by an unusual user. The metadata service may be targeted in order to harvest credentials or user data scripts containing secrets.

A link icon

A link icon

linux_rare_sudo_user

Looks for sudo activity from an unusual user context.

A link icon

A link icon

linux_rare_user_compiler

Looks for compiler activity by a user context which does not normally run compilers. This can be ad-hoc software changes or unauthorized software deployment. This can also be due to local privilege elevation via locally run exploits or malware activity.

A link icon

A link icon

linux_system_information_discovery

Looks for commands related to system information discovery from an unusual user context. This can be due to uncommon troubleshooting activity or due to a compromised account. A compromised account may be used to engage in system information discovery in order to gather detailed information about system configuration and software versions. This may be a precursor to selection of a persistence mechanism or a method of privilege elevation.

A link icon

A link icon

linux_system_process_discovery

Looks for commands related to system process discovery from an unusual user context. This can be due to uncommon troubleshooting activity or due to a compromised account. A compromised account may be used to engage in system process discovery in order to increase their understanding of software applications running on a target host or network. This may be a precursor to selection of a persistence mechanism or a method of privilege elevation.

A link icon

A link icon

linux_system_user_discovery

Looks for commands related to system user or owner discovery from an unusual user context. This can be due to uncommon troubleshooting activity or due to a compromised account. A compromised account may be used to engage in system owner or user discovery in order to identify currently active or primary users of a system. This may be a precursor to additional discovery, credential dumping or privilege elevation activity.

A link icon

A link icon

rare_process_by_host_linux_ecs

Detect unusually rare processes on Linux.

A link icon

A link icon

Security: Auditbeat authenticationedit

Detect suspicious authentication events in Auditbeat data.

In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the manifest file. In the Elastic Security app, it looks in the data view specified in the securitySolution:defaultIndex advanced setting for data that matches the query.

Name Description Job Datafeed

suspicious_login_activity_ecs

Identifies an unusually high number of authentication attempts.

A link icon

A link icon

Security: Authenticationedit

Detect anomalous activity in your ECS-compatible authentication logs.

In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the manifest file. In the Elastic Security app, it looks in the data view specified in the securitySolution:defaultIndex advanced setting for data that matches the query.

By default, when you create these job in the Elastic Security app, it uses a data view that applies to multiple indices. To get the same results if you use the Machine Learning app, create a similar data view then select it in the job wizard.

Name Description Job Datafeed

auth_high_count_logon_events

Looks for an unusually large spike in successful authentication events. This can be due to password spraying, user enumeration or brute force activity.

A link icon

A link icon

auth_high_count_logon_events_for_a_source_ip

Looks for an unusually large spike in successful authentication events from a particular source IP address. This can be due to password spraying, user enumeration or brute force activity.

A link icon

A link icon

auth_high_count_logon_fails

Looks for an unusually large spike in authentication failure events. This can be due to password spraying, user enumeration or brute force activity and may be a precursor to account takeover or credentialed access.

A link icon

A link icon

auth_rare_hour_for_a_user

Looks for a user logging in at a time of day that is unusual for the user. This can be due to credentialed access via a compromised account when the user and the threat actor are in different time zones. In addition, unauthorized user activity often takes place during non-business hours.

A link icon

A link icon

auth_rare_source_ip_for_a_user

Looks for a user logging in from an IP address that is unusual for the user. This can be due to credentialed access via a compromised account when the user and the threat actor are in different locations. An unusual source IP address for a username could also be due to lateral movement when a compromised account is used to pivot between hosts.

A link icon

A link icon

auth_rare_user

Looks for an unusual user name in the authentication logs. An unusual user name is one way of detecting credentialed access by means of a new or dormant user account. A user account that is normally inactive, because the user has left the organization, which becomes active, may be due to credentialed access using a compromised account password. Threat actors will sometimes also create new users as a means of persisting in a compromised web application.

A link icon

A link icon

Security: CloudTrailedit

Detect suspicious activity recorded in your CloudTrail logs.

In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the manifest file. In the Elastic Security app, it looks in the data view specified in the securitySolution:defaultIndex advanced setting for data that matches the query.

Name Description Job Datafeed

high_distinct_count_error_message

Looks for a spike in the rate of an error message which may simply indicate an impending service failure but these can also be byproducts of attempted or successful persistence, privilege escalation, defense evasion, discovery, lateral movement, or collection activity by a threat actor.

A link icon

A link icon

rare_error_code

Looks for unusual errors. Rare and unusual errors may simply indicate an impending service failure but they can also be byproducts of attempted or successful persistence, privilege escalation, defense evasion, discovery, lateral movement, or collection activity by a threat actor.

A link icon

A link icon

rare_method_for_a_city

Looks for AWS API calls that, while not inherently suspicious or abnormal, are sourcing from a geolocation (city) that is unusual. This can be the result of compromised credentials or keys.

A link icon

A link icon

rare_method_for_a_country

Looks for AWS API calls that, while not inherently suspicious or abnormal, are sourcing from a geolocation (country) that is unusual. This can be the result of compromised credentials or keys.

A link icon

A link icon

rare_method_for_a_username

Looks for AWS API calls that, while not inherently suspicious or abnormal, are sourcing from a user context that does not normally call the method. This can be the result of compromised credentials or keys as someone uses a valid account to persist, move laterally, or exfil data.

A link icon

A link icon

Security: Linuxedit

Detect suspicious activity using ECS Linux events.

In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the manifest file. In the Elastic Security app, it looks in the data view specified in the securitySolution:defaultIndex advanced setting for data that matches the query.

In 7.11 or later versions, use these jobs instead of the Security: Auditbeat jobs.[1]

Name Description Job Datafeed

v2_linux_anomalous_network_port_activity_ecs

This is a new refactored job which works on ECS compatible events across multiple indices. Looks for unusual destination port activity that could indicate command-and-control, persistence mechanism, or data exfiltration activity.

A link icon

A link icon

v2_linux_anomalous_process_all_hosts_ecs

This is a new refactored job which works on ECS compatible events across multiple indices. Looks for processes that are unusual to all Linux hosts. Such unusual processes may indicate unauthorized services, malware, or persistence mechanisms.

A link icon

A link icon

v2_linux_anomalous_user_name_ecs

This is a new refactored job which works on ECS compatible events across multiple indices. Rare and unusual users that are not normally active may indicate unauthorized changes or activity by an unauthorized user which may be credentialed access or lateral movement.

A link icon

A link icon

v2_linux_rare_metadata_process

This is a new refactored job which works on ECS compatible events across multiple indices. Looks for anomalous access to the metadata service by an unusual process. The metadata service may be targeted in order to harvest credentials or user data scripts containing secrets.

A link icon

A link icon

v2_linux_rare_metadata_user

This is a new refactored job which works on ECS compatible events across multiple indices. Looks for anomalous access to the metadata service by an unusual user. The metadata service may be targeted in order to harvest credentials or user data scripts containing secrets.

A link icon

A link icon

v2_rare_process_by_host_linux_ecs

This is a new refactored job which works on ECS compatible events across multiple indices. Looks for processes that are unusual to a particular Linux host. Such unusual processes may indicate unauthorized services, malware, or persistence mechanisms.

A link icon

A link icon

Security: Networkedit

Detect anomalous network activity in your ECS-compatible network logs.

In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the manifest file. In the Elastic Security app, it looks in the data view specified in the securitySolution:defaultIndex advanced setting for data that matches the query.

By default, when you create these jobs in the Elastic Security app, it uses a data view that applies to multiple indices. To get the same results if you use the Machine Learning app, create a similar data view then select it in the job wizard.

Name Description Job Datafeed

high_count_by_destination_country

Looks for an unusually large spike in network activity to one destination country in the network logs. This could be due to unusually large amounts of reconnaissance or enumeration traffic. Data exfiltration activity may also produce such a surge in traffic to a destination country which does not normally appear in network traffic or business work-flows. Malware instances and persistence mechanisms may communicate with command-and-control (C2) infrastructure in their country of origin, which may be an unusual destination country for the source network.

A link icon

A link icon

high_count_network_denies

Looks for an unusually large spike in network traffic that was denied by network ACLs or firewall rules. Such a burst of denied traffic is usually either 1) a misconfigured application or firewall or 2) suspicious or malicious activity. Unsuccessful attempts at network transit, in order to connect to command-and-control (C2), or engage in data exfiltration, may produce a burst of failed connections. This could also be due to unusually large amounts of reconnaissance or enumeration traffic. Denial-of-service attacks or traffic floods may also produce such a surge in traffic.

A link icon

A link icon

high_count_network_events

Looks for an unusually large spike in network traffic. Such a burst of traffic, if not caused by a surge in business activity, can be due to suspicious or malicious activity. Large-scale data exfiltration may produce a burst of network traffic; this could also be due to unusually large amounts of reconnaissance or enumeration traffic. Denial-of-service attacks or traffic floods may also produce such a surge in traffic.

A link icon

A link icon

rare_destination_country

Looks for an unusual destination country name in the network logs. This can be due to initial access, persistence, command-and-control, or exfiltration activity. For example, when a user clicks on a link in a phishing email or opens a malicious document, a request may be sent to download and run a payload from a server in a country which does not normally appear in network traffic or business work-flows. Malware instances and persistence mechanisms may communicate with command-and-control (C2) infrastructure in their country of origin, which may be an unusual destination country for the source network.

A link icon

A link icon

Security: Packetbeatedit

Detect suspicious network activity in Packetbeat data.

In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the manifest file. In the Elastic Security app, it looks in the data view specified in the securitySolution:defaultIndex advanced setting for data that matches the query.

Name Description Job Datafeed

packetbeat_dns_tunneling

Looks for unusual DNS activity that could indicate command-and-control or data exfiltration activity.

A link icon

A link icon

packetbeat_rare_dns_question

Looks for unusual DNS activity that could indicate command-and-control activity.

A link icon

A link icon

packetbeat_rare_server_domain

Looks for unusual HTTP or TLS destination domain activity that could indicate execution, persistence, command-and-control or data exfiltration activity.

A link icon

A link icon

packetbeat_rare_urls

Looks for unusual web browsing URL activity that could indicate execution, persistence, command-and-control or data exfiltration activity.

A link icon

A link icon

packetbeat_rare_user_agent

Looks for unusual HTTP user agent activity that could indicate execution, persistence, command-and-control or data exfiltration activity.

A link icon

A link icon

Security: Windowsedit

Detects suspicious activity using ECS Windows events.

In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the manifest file. In the Elastic Security app, it looks in the data view specified in the securitySolution:defaultIndex advanced setting for data that matches the query.

If there are additional requirements such as installing the Windows System Monitor (Sysmon) or auditing process creation in the Windows security event log, they are listed for each job.

In 7.11 or later versions, use these jobs instead of the Security: Winlogbeat jobs.[3]

Name Description Job Datafeed

v2_rare_process_by_host_windows_ecs

This is a new refactored job which works on ECS compatible events across multiple indices. Detects unusually rare processes on Windows hosts.

A link icon

A link icon

v2_windows_anomalous_network_activity_ecs

This is a new refactored job which works on ECS compatible events across multiple indices. Looks for unusual processes using the network which could indicate command-and-control, lateral movement, persistence, or data exfiltration activity.

A link icon

A link icon

v2_windows_anomalous_path_activity_ecs

This is a new refactored job which works on ECS compatible events across multiple indices. Looks for activity in unusual paths that may indicate execution of malware or persistence mechanisms. Windows payloads often execute from user profile paths.

A link icon

A link icon

v2_windows_anomalous_process_all_hosts_ecs

This is a new refactored job which works on ECS compatible events across multiple indices. Looks for processes that are unusual to all Windows hosts. Such unusual processes may indicate execution of unauthorized services, malware, or persistence mechanisms.

A link icon

A link icon

v2_windows_anomalous_process_creation

This is a new refactored job which works on ECS compatible events across multiple indices. Looks for unusual process relationships which may indicate execution of malware or persistence mechanisms.

A link icon

A link icon

v2_windows_anomalous_user_name_ecs

This is a new refactored job which works on ECS compatible events across multiple indices. Rare and unusual users that are not normally active may indicate unauthorized changes or activity by an unauthorized user which may be credentialed access or lateral movement.

A link icon

A link icon

v2_windows_rare_metadata_process

This is a new refactored job which works on ECS compatible events across multiple indices. Looks for anomalous access to the metadata service by an unusual process. The metadata service may be targeted in order to harvest credentials or user data scripts containing secrets.

A link icon

A link icon

v2_windows_rare_metadata_user

This is a new refactored job which works on ECS compatible events across multiple indices. Looks for anomalous access to the metadata service by an unusual user. The metadata service may be targeted in order to harvest credentials or user data scripts containing secrets.

A link icon

A link icon

Security: Winlogbeatedit

Detect unusual processes and network activity in Winlogbeat data.

In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the manifest file. In the Elastic Security app, it looks in the data view specified in the securitySolution:defaultIndex advanced setting for data that matches the query.

In 7.11 or later versions, use the Security: Windows jobs instead.[3]

Name Description Job Datafeed

rare_process_by_host_windows_ecs

Detect unusually rare processes on Windows.

A link icon

A link icon

windows_anomalous_network_activity_ecs

Looks for unusual processes using the network which could indicate command-and-control, lateral movement, persistence, or data exfiltration activity.

A link icon

A link icon

windows_anomalous_path_activity_ecs

Looks for activity in unusual paths that may indicate execution of malware or persistence mechanisms. Windows payloads often execute from user profile paths.

A link icon

A link icon

windows_anomalous_process_all_hosts_ecs

Looks for processes that are unusual to all Windows hosts. Such unusual processes may indicate execution of unauthorized services, malware, or persistence mechanisms.

A link icon

A link icon

windows_anomalous_process_creation

Looks for unusual process relationships which may indicate execution of malware or persistence mechanisms.

A link icon

A link icon

windows_anomalous_script

Looks for unusual powershell scripts that may indicate execution of malware, or persistence mechanisms.

A link icon

A link icon

windows_anomalous_service

Looks for rare and unusual Windows services which may indicate execution of unauthorized services, malware, or persistence mechanisms.

A link icon

A link icon

windows_anomalous_user_name_ecs

Rare and unusual users that are not normally active may indicate unauthorized changes or activity by an unauthorized user which may be credentialed access or lateral movement.

A link icon

A link icon

windows_rare_metadata_process

Looks for anomalous access to the metadata service by an unusual process. The metadata service may be targeted in order to harvest credentials or user data scripts containing secrets.

A link icon

A link icon

windows_rare_metadata_user

Looks for anomalous access to the metadata service by an unusual user. The metadata service may be targeted in order to harvest credentials or user data scripts containing secrets.

A link icon

A link icon

windows_rare_user_runas_event

Unusual user context switches can be due to privilege escalation.

A link icon

A link icon

Security: Winlogbeat authenticationedit

Detect suspicious authentication events in Winlogbeat data.

In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the manifest file. In the Elastic Security app, it looks in the data view specified in the securitySolution:defaultIndex advanced setting for data that matches the query.

Name Description Job Datafeed

windows_rare_user_type10_remote_login

Unusual RDP (remote desktop protocol) user logins can indicate account takeover or credentialed access.

A link icon

A link icon

[1] If you cannot upgrade all your Beats to version 7.11 or later and you have both Security: Linux and Security: Auditbeat jobs running, you can avoid duplication by stopping the following jobs: linux_anomalous_network_activity_ecs, linux_anomalous_network_port_activity_ecs, linux_anomalous_process_all_hosts_ecs, linux_anomalous_user_name_ecs, linux_rare_metadata_process, linux_rare_metadata_user, rare_process_by_host_linux_ecs.
[2] Some jobs use fields that are not ECS-compliant. These jobs are available only when you use Beats or the Elastic Agent to ship data.
[3] If you cannot upgrade all your Beats to version 7.11 or later and you have both Security:Windows jobs and Security:Winlogbeat jobs running, you can avoid duplication by stopping the following jobs: rare_process_by_host_windows_ecs, windows_anomalous_network_activity_ecs, windows_anomalous_path_activity_ecs, windows_anomalous_process_all_hosts_ecs, windows_anomalous_process_creation, windows_anomalous_user_name_ecs, windows_rare_metadata_process, windows_rare_metadata_user