Create sample anomaly detection jobs in Kibanaedit

The Kibana sample data sets include some pre-configured anomaly detection jobs for you to play with. You can use either of the following methods to add the jobs:

  • After you load the sample web logs data set on the Kibana home page, click View data > ML jobs.
  • In the Machine Learning app, when you select the kibana_sample_data_logs index pattern in the Data Visualizer or the Anomaly Detection job wizards, it recommends that you create a job using its known configuration. Select the Kibana sample data web logs configuration.
Create jobs for the sample web logs

Accept the default values and click Create Jobs.

The wizard creates three jobs and three datafeeds.

If you want to see all of the configuration details for your jobs and datafeeds, you can do so on the Machine Learning > Anomaly Detection > Job Management page. For the purposes of this tutorial, however, here’s a quick overview of the goal of each job:

  • low_request_rate uses the low_count function to find unusually low request rates
  • response_code_rates uses the count function and partitions the analysis by response.keyword values to find unusual event rates by HTTP response code
  • url_scanning uses the high_distinct_count function and performs population analysis on the clientip field to find client IPs accessing an unusually high distinct count of URLs

The next step is to view the results and see what types of insights these jobs have generated!