Create forecastsedit

In addition to detecting anomalous behavior in your data, you can use the machine learning features to predict future behavior.

To create a forecast in Kibana:

  1. View your job results (for example, for the low_request_rate job) in the Single Metric Viewer. To find that view, follow the link in the Actions column on the Anomaly Detection page.
  2. Click Forecast.

    Create a forecast from the Single Metric Viewer
  3. Specify a duration for your forecast. This value indicates how far to extrapolate beyond the last record that was processed. You must use time units. In this example, the duration is one week (1w):

    Specify a duration of 1w
  4. View the forecast in the Single Metric Viewer:

    View a forecast from the Single Metric Viewer

    The yellow line in the chart represents the predicted data values. The shaded yellow area represents the bounds for the predicted values, which also gives an indication of the confidence of the predictions. Note that the bounds generally increase with time (that is to say, the confidence levels decrease), since you are forecasting further into the future. Eventually if the confidence levels are too low, the forecast stops.

  5. Optional: Compare the forecast to actual data.

    View a forecast over actual data in the Single Metric Viewer

    As the job processes more data, you can click the Forecast button again and choose to see one of your forecasts overlaid on the actual data. The chart then contains the actual data values, the bounds for the expected values, the anomalies, the forecast data values, and the bounds for the forecast. This combination of actual and forecast data gives you an indication of how well the machine learning features can extrapolate the future behavior of the data.

    If you want to see this type of comparison for the Kibana sample data, which has a finite number of documents, you can reset the job and analyze only a subset of the data before you create a forecast. For example, reset one of your anomaly detection jobs from the Job Management page in Kibana or use the reset anomaly detection jobs API. When you restart the datafeed for this job, choose a date part way through your sample data as the search end date. By default, the datafeed stops and the anomaly detection job closes when it reaches that date. Create the forecast. You can then restart the datafeed to process the remaining data and generate the type of results shown here.

    The Kibana sample data sets have timestamps that are relative to when you added the data sets. However, some of these dates are in the future. Therefore, for the purposes of this tutorial, when you restart your datafeed do not use the No end time (Real-time search) option. Specify the appropriate end dates so that it processes all of the data immediately.

Now that you have seen how easy it is to create forecasts with the sample data, consider what type of events you might want to predict in your own data. For more information and ideas, as well as a list of limitations related to forecasts, see Forecasting the future.