Put Rollup Job API

edit

This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

The Put Rollup Job API can be used to create a new Rollup job in the cluster. The API accepts a PutRollupJobRequest object as a request and returns a PutRollupJobResponse.

Put Rollup Job Request

edit

A PutRollupJobRequest requires the following argument:

PutRollupJobRequest request = new PutRollupJobRequest(config); 

The configuration of the Rollup job to create as a RollupJobConfig

Rollup Job Configuration

edit

The RollupJobConfig object contains all the details about the rollup job configuration. See Rollup configuration to learn more about the various configuration settings.

A RollupJobConfig requires the following arguments:

RollupJobConfig config = new RollupJobConfig(id, 
    indexPattern,  
    rollupIndex,  
    cron,  
    pageSize,  
    groups,  
    metrics,  
    timeout);  

The name of the Rollup job

The index (or index pattern) to rollup

The index to store rollup results into

A cron expression which defines when the Rollup job should be executed

The page size to use for the Rollup job

The grouping configuration of the Rollup job as a GroupConfig

The metrics configuration of the Rollup job as a list of MetricConfig

The timeout value to use for the Rollup job as a TimeValue

Grouping Configuration

edit

The grouping configuration of the Rollup job is defined in the RollupJobConfig using a GroupConfig instance. GroupConfig reflects all the configuration settings that can be defined using the REST API. See Grouping Config to learn more about these settings.

Using the REST API, we could define this grouping configuration:

"groups" : {
  "date_histogram": {
    "field": "timestamp",
    "calendar_interval": "1h",
    "delay": "7d",
    "time_zone": "UTC"
  },
  "terms": {
    "fields": ["hostname", "datacenter"]
  },
  "histogram": {
    "fields": ["load", "net_in", "net_out"],
    "interval": 5
  }
}

Using the GroupConfig object and the high level REST client, the same configuration would be:

DateHistogramGroupConfig dateHistogram =
    new DateHistogramGroupConfig("timestamp", DateHistogramInterval.HOUR, new DateHistogramInterval("7d"), "UTC"); 
TermsGroupConfig terms = new TermsGroupConfig("hostname", "datacenter"); 
HistogramGroupConfig histogram = new HistogramGroupConfig(5L, "load", "net_in", "net_out"); 

GroupConfig groups = new GroupConfig(dateHistogram, histogram, terms); 

The date histogram aggregation to use to rollup up documents, as a DateHistogramGroupConfig

The terms aggregation to use to rollup up documents, as a TermsGroupConfig

The histogram aggregation to use to rollup up documents, as a HistogramGroupConfig

The grouping configuration as a GroupConfig

Metrics Configuration

edit

After defining which groups should be generated for the data, you next configure which metrics should be collected. The list of metrics is defined in the RollupJobConfig using a List<MetricConfig> instance. MetricConfig reflects all the configuration settings that can be defined using the REST API. See Metrics Config to learn more about these settings.

Using the REST API, we could define this metrics configuration:

"metrics": [
    {
        "field": "temperature",
        "metrics": ["min", "max", "sum"]
    },
    {
        "field": "voltage",
        "metrics": ["avg", "value_count"]
    }
]

Using the MetricConfig object and the high level REST client, the same configuration would be:

List<MetricConfig> metrics = new ArrayList<>(); 
metrics.add(new MetricConfig("temperature", Arrays.asList("min", "max", "sum"))); 
metrics.add(new MetricConfig("voltage", Arrays.asList("avg", "value_count"))); 

The list of MetricConfig to configure in the RollupJobConfig

Adds the metrics to compute on the temperature field

Adds the metrics to compute on the voltage field

Execution

edit

The Put Rollup Job API can be executed through a RollupClient instance. Such instance can be retrieved from a RestHighLevelClient using the rollup() method:

AcknowledgedResponse response = client.rollup().putRollupJob(request, RequestOptions.DEFAULT);

Response

edit

The returned PutRollupJobResponse indicates if the new Rollup job has been successfully created:

boolean acknowledged = response.isAcknowledged(); 

acknowledged is a boolean indicating whether the job was successfully created

Asynchronous Execution

edit

This request can be executed asynchronously:

client.rollup().putRollupJobAsync(request, RequestOptions.DEFAULT, listener); 

The PutRollupJobRequest to execute and the ActionListener to use when the execution completes

The asynchronous method does not block and returns immediately. Once it is completed the ActionListener is called back using the onResponse method if the execution successfully completed or using the onFailure method if it failed.

A typical listener for PutRollupJobResponse looks like:

ActionListener<AcknowledgedResponse> listener = new ActionListener<AcknowledgedResponse>() {
    @Override
    public void onResponse(AcknowledgedResponse response) {
        
    }

    @Override
    public void onFailure(Exception e) {
        
    }
};

Called when the execution is successfully completed. The response is provided as an argument

Called in case of failure. The raised exception is provided as an argument