Evaluates the machine learning algorithm that ran on a data frame.
The API accepts an
EvaluateDataFrameRequest object and returns an
EvaluateDataFrameRequest request = new EvaluateDataFrameRequest( indexName, new QueryConfig(QueryBuilders.termQuery("dataset", "blue")), new BinarySoftClassification( "label", "p", // Evaluation metrics PrecisionMetric.at(0.4, 0.5, 0.6), RecallMetric.at(0.5, 0.7), ConfusionMatrixMetric.at(0.5), AucRocMetric.withCurve()));
Constructing a new evaluation request
Reference to an existing index
The query with which to select data from indices
Kind of evaluation to perform
Name of the field in the index. Its value denotes the actual (i.e. ground truth) label for an example. Must be either true or false
Name of the field in the index. Its value denotes the probability (as per some ML algorithm) of the example being classified as positive
The remaining parameters are the metrics to be calculated based on the two fields described above.
Precision calculated at thresholds: 0.4, 0.5 and 0.6
Recall calculated at thresholds: 0.5 and 0.7
Confusion matrix calculated at threshold 0.5
AuC ROC calculated and the curve points returned
When executing a
EvaluateDataFrameRequest in the following manner, the client waits
EvaluateDataFrameResponse to be returned before continuing with code execution:
EvaluateDataFrameResponse response = client.machineLearning().evaluateDataFrame(request, RequestOptions.DEFAULT);
Synchronous calls may throw an
IOException in case of either failing to
parse the REST response in the high-level REST client, the request times out
or similar cases where there is no response coming back from the server.
In cases where the server returns a
5xx error code, the high-level
client tries to parse the response body error details instead and then throws
ElasticsearchException and adds the original
ResponseException as a
suppressed exception to it.
EvaluateDataFrameRequest can also be done in an asynchronous fashion so that
the client can return directly. Users need to specify how the response or
potential failures will be handled by passing the request and a listener to the
asynchronous evaluate-data-frame method:
The asynchronous method does not block and returns immediately. Once it is
ActionListener is called back using the
if the execution successfully completed or using the
onFailure method if
it failed. Failure scenarios and expected exceptions are the same as in the
synchronous execution case.
A typical listener for
evaluate-data-frame looks like:
EvaluateDataFrameResponse contains the requested evaluation metrics.
List<EvaluationMetric.Result> metrics = response.getMetrics(); PrecisionMetric.Result precisionResult = response.getMetricByName(PrecisionMetric.NAME); double precision = precisionResult.getScoreByThreshold("0.4"); ConfusionMatrixMetric.Result confusionMatrixResult = response.getMetricByName(ConfusionMatrixMetric.NAME); ConfusionMatrix confusionMatrix = confusionMatrixResult.getScoreByThreshold("0.5");