Anomaly Detection with Machine Learning
editAnomaly Detection with Machine Learning
editFor Free Trial, Cloud
and Platinum License deployments,
Machine Learning functionality is available
throughout the SIEM app. You can view the details of detected anomalies within
the Anomalies
table widget shown on the Hosts, Network and associated Details
pages, or even narrow to the specific date range of an anomaly from the Max
Anomaly Score
details in the overview of the Host and IP Details pages. Each
of these interfaces also offer the ability to drag and drop details of the
anomaly to Timeline, such as the Entity
itself, or any of the associated
Influencers
.

Manage machine learning jobs
editFor users with the ml_admin
role, the Anomaly Detection
interface within
the main navigation header can be used for for viewing, starting, and stopping
SIEM machine learning jobs.
To add a custom job to the Anomaly Detection
interface, add a SIEM
tag
to the job’s Group
field (Kibana → Machine learning → Create/Edit job → Job
details).
Prebuilt jobs
editThe SIEM app comes with prebuilt machine learning anomaly detection jobs for automatically detecting
host and network anomalies. The jobs are displayed in the Anomaly Detection
interface. They are available when:
-
You ship data using Beats, and
Kibana is configured with the required index patterns
(
auditbeat-*
,filebeat-*
,packetbeat-*
, orwinlogbeat-*
via Kibana → Management → Index Patterns). - Your shipped data is ECS-compliant, and Kibana is configured with the data’s index patterns.
Machine learning jobs look back and analyse two weeks of historical data prior to the time they are enabled. After jobs are enabled, they continuously analyse incoming data. When jobs are stopped and restarted within the two week timeframe, previously analysed data is not processed again.
Prebuilt job reference describes all available machine learning jobs and lists which beats are required on your hosts for each job. For information on tuning anomaly results to reduce the number of false positive, see Optimizing anomaly results.
View detected anomalies
editTo view the Anomalies
table widget and Max Anomaly Score By Job
details,
the user must have the ml_admin
or ml_user
role.
To adjust the score
threshold that determines which
anomalies are shown, you can modify Kibana →
Management → Advanced Settings → siem:defaultAnomalyScore
.