Put Datafeed API

The Put Datafeed API can be used to create a new machine learning datafeed in the cluster. The API accepts a PutDatafeedRequest object as a request and returns a PutDatafeedResponse.

Put Datafeed Request

A PutDatafeedRequest requires the following argument:

PutDatafeedRequest request = new PutDatafeedRequest(datafeedBuilder.build()); 

The configuration of the machine learning datafeed to create

Datafeed Configuration

The DatafeedConfig object contains all the details about the machine learning datafeed configuration.

A DatafeedConfig requires the following arguments:

DatafeedConfig.Builder datafeedBuilder = new DatafeedConfig.Builder(id, jobId) 
        .setIndices("index_1", "index_2");  

The datafeed ID and the job ID

The indices that contain the data to retrieve and feed into the job

Optional Arguments

The following arguments are optional:

datafeedBuilder.setChunkingConfig(ChunkingConfig.newAuto()); 

Specifies how data searches are split into time chunks.

datafeedBuilder.setFrequency(TimeValue.timeValueSeconds(30)); 

The interval at which scheduled queries are made while the datafeed runs in real time.

datafeedBuilder.setQuery(QueryBuilders.matchAllQuery()); 

A query to filter the search results by. Defaults to the match_all query.

datafeedBuilder.setQueryDelay(TimeValue.timeValueMinutes(1)); 

The time interval behind real time that data is queried.

datafeedBuilder.setDelayedDataCheckConfig(DelayedDataCheckConfig
    .enabledDelayedDataCheckConfig(TimeValue.timeValueHours(1))); 

Sets the delayed data check configuration. The window must be larger than the Job’s bucket size, but smaller than 24 hours, and span less than 10,000 buckets. Defaults to null, which causes an appropriate window span to be calculated when the datafeed runs. The default check_window span calculation is the max between 2h or 8 * bucket_span. To explicitly disable, pass DelayedDataCheckConfig.disabledDelayedDataCheckConfig().

datafeedBuilder.setScriptFields(scriptFields); 

Allows the use of script fields.

datafeedBuilder.setScrollSize(1000); 

The size parameter used in the searches.

Synchronous Execution

When executing a PutDatafeedRequest in the following manner, the client waits for the PutDatafeedResponse to be returned before continuing with code execution:

PutDatafeedResponse response = client.machineLearning().putDatafeed(request, RequestOptions.DEFAULT);

Synchronous calls may throw an IOException in case of either failing to parse the REST response in the high-level REST client, the request times out or similar cases where there is no response coming back from the server.

In cases where the server returns a 4xx or 5xx error code, the high-level client tries to parse the response body error details instead and then throws a generic ElasticsearchException and adds the original ResponseException as a suppressed exception to it.

Asynchronous Execution

Executing a PutDatafeedRequest can also be done in an asynchronous fashion so that the client can return directly. Users need to specify how the response or potential failures will be handled by passing the request and a listener to the asynchronous put-datafeed method:

client.machineLearning().putDatafeedAsync(request, RequestOptions.DEFAULT, listener); 

The PutDatafeedRequest to execute and the ActionListener to use when the execution completes

The asynchronous method does not block and returns immediately. Once it is completed the ActionListener is called back using the onResponse method if the execution successfully completed or using the onFailure method if it failed. Failure scenarios and expected exceptions are the same as in the synchronous execution case.

A typical listener for put-datafeed looks like:

ActionListener<PutDatafeedResponse> listener = new ActionListener<PutDatafeedResponse>() {
    @Override
    public void onResponse(PutDatafeedResponse response) {
        
    }

    @Override
    public void onFailure(Exception e) {
        
    }
};

Called when the execution is successfully completed.

Called when the whole PutDatafeedRequest fails.

Response

The returned PutDatafeedResponse returns the full representation of the new machine learning datafeed if it has been successfully created. This will contain the creation time and other fields initialized using default values:

DatafeedConfig datafeed = response.getResponse(); 

The created datafeed