Evaluate ranked search results | Elasticsearch API documentation

Evaluate ranked search results Generally available; Added in 6.2.0

GET /_rank_eval

Evaluate the quality of ranked search results over a set of typical search queries.

Required authorization

Index privileges: read

More about ranking evaluation

Query parameters

allow_no_indices boolean

A setting that does two separate checks on the index expression. If false, the request returns an error (1) if any wildcard expression (including _all and *) resolves to zero matching indices or (2) if the complete set of resolved indices, aliases or data streams is empty after all expressions are evaluated. If true, index expressions that resolve to no indices are allowed and the request returns an empty result.
expand_wildcards string | array[string]

Whether to expand wildcard expression to concrete indices that are open, closed or both.

Values are all, open, closed, hidden, or none.
ignore_unavailable boolean

If false, the request returns an error if it targets a concrete (non-wildcarded) index, alias, or data stream that is missing, closed, or otherwise unavailable. If true, unavailable concrete targets are silently ignored.
search_type string

Search operation type

Values are query_then_fetch or dfs_query_then_fetch.

application/json

Body Required

requests array[object] Required

A set of typical search requests, together with their provided ratings.
Hide requests attributes Show requests attributes object
- id string Required
  
  The search request’s ID, used to group result details later.
- request object
  
  The query being evaluated.
  Hide request attributes Show request attributes object
  
  query object Required
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  External documentation
  
  Hide query attributes Show query attributes object
  
  common object Deprecated
  
  fuzzy object
  
  Returns documents that contain terms similar to the search term, as measured by a Levenshtein edit distance.
  
  geo_grid object
  
  Matches geo_point and geo_shape values that intersect a grid cell from a GeoGrid aggregation.
  
  intervals object
  
  Returns documents based on the order and proximity of matching terms.
  
  match object
  
  Returns documents that match a provided text, number, date or boolean value. The provided text is analyzed before matching.
  
  match_bool_prefix object
  
  Analyzes its input and constructs a bool query from the terms. Each term except the last is used in a term query. The last term is used in a prefix query.
  
  match_phrase object
  
  Analyzes the text and creates a phrase query out of the analyzed text.
  
  match_phrase_prefix object
  
  Returns documents that contain the words of a provided text, in the same order as provided. The last term of the provided text is treated as a prefix, matching any words that begin with that term.
  
  prefix object
  
  Returns documents that contain a specific prefix in a provided field.
  
  range object
  
  Returns documents that contain terms within a provided range.
  
  regexp object
  
  Returns documents that contain terms matching a regular expression.
  
  span_term object
  
  Matches spans containing a term.
  
  term object
  
  Returns documents that contain an exact term in a provided field. To return a document, the query term must exactly match the queried field's value, including whitespace and capitalization.
  
  terms_set object
  
  Returns documents that contain a minimum number of exact terms in a provided field. To return a document, a required number of terms must exactly match the field values, including whitespace and capitalization.
  
  text_expansion object Deprecated Generally available; Added in 8.8.0
  
  Uses a natural language processing model to convert the query text into a list of token-weight pairs which are then used in a query against a sparse vector or rank features field.
  
  weighted_tokens object Deprecated Generally available; Added in 8.13.0
  
  Supports returning text_expansion query results by sending in precomputed tokens with the query.
  
  wildcard object
  
  Returns documents that contain terms matching a wildcard pattern.
  
  size number
- ratings array[object] Required
  
  List of document ratings
  Hide ratings attributes Show ratings attributes object
  
  _id string Required
  
  The document ID.
  
  _index string Required
  
  The document’s index. For data streams, this should be the document’s backing index.
  
  rating number Required
  
  The document’s relevance with regard to this search request.
- template_id string
  
  The search template Id
- params object
  
  The search template parameters.
  Hide params attribute Show params attribute object
  
  * object Additional properties
metric object

Definition of the evaluation metric to calculate.
Hide metric attributes Show metric attributes object
- precision object
  
  Precision at K (P@k)
  Hide precision attributes Show precision attributes object
  
  k number
  
  Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
  
  Default value is 10.0.
  
  relevant_rating_threshold number
  
  Sets the rating threshold above which documents are considered to be "relevant".
  
  Default value is 1.0.
  
  ignore_unlabeled boolean
  
  Controls how unlabeled documents in the search results are counted. If set to true, unlabeled documents are ignored and neither count as relevant or irrelevant. Set to false (the default), they are treated as irrelevant.
  
  Default value is false.
- recall object
  
  Recall at K (R@k)
  Hide recall attributes Show recall attributes object
  
  k number
  
  Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
  
  Default value is 10.0.
  
  relevant_rating_threshold number
  
  Sets the rating threshold above which documents are considered to be "relevant".
  
  Default value is 1.0.
- mean_reciprocal_rank object
  
  Mean Reciprocal Rank
  Hide mean_reciprocal_rank attributes Show mean_reciprocal_rank attributes object
  
  k number
  
  Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
  
  Default value is 10.0.
  
  relevant_rating_threshold number
  
  Sets the rating threshold above which documents are considered to be "relevant".
  
  Default value is 1.0.
- dcg object
  
  Discounted cumulative gain (DCG)
  Hide dcg attributes Show dcg attributes object
  
  k number
  
  Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
  
  Default value is 10.0.
  
  normalize boolean
  
  If set to true, this metric will calculate the Normalized DCG.
  
  Default value is false.
- expected_reciprocal_rank object
  
  Expected Reciprocal Rank (ERR)
  Hide expected_reciprocal_rank attributes Show expected_reciprocal_rank attributes object
  
  k number
  
  Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
  
  Default value is 10.0.
  
  maximum_relevance number Required
  
  The highest relevance grade used in the user-supplied relevance judgments.

Responses

200 application/json
Hide response attributes Show response attributes object
- metric_score number Required
  
  The overall evaluation quality calculated by the defined metric
- details object Required
  
  The details section contains one entry for every query in the original requests section, keyed by the search request id
  
  Hide details attribute Show details attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  metric_score number Required
  
  The metric_score in the details section shows the contribution of this query to the global quality metric score
  
  unrated_docs array[object] Required
  
  The unrated_docs section contains an _index and _id entry for each document in the search result for this query that didn’t have a ratings value. This can be used to ask the user to supply ratings for these documents
  
  Hide unrated_docs attributes Show unrated_docs attributes object
  
  _id string Required
  
  _index string Required
  
  hits array[object] Required
  
  The hits section shows a grouping of the search results with their supplied ratings
  
  Hide hits attributes Show hits attributes object
  
  hit object Required
  
  rating number | string | null
  
  One of:
  number-1 number string-2 string | null
  
  metric_details object Required
  
  The metric_details give additional information about the calculated quality metric (e.g. how many of the retrieved documents were relevant). The content varies for each metric but allows for better interpretation of the results
  
  Hide metric_details attribute Show metric_details attribute object
  
  * object Additional properties
  
  Hide * attribute Show * attribute object
  
  * object Additional properties
- failures object Required
  
  Hide failures attribute Show failures attribute object
  
  * object Additional properties

GET /_rank_eval

curl \
 --request GET 'http://api.example.com/_rank_eval' \
 --header "Content-Type: application/json" \
 --data '"{\n  \"requests\": [\n    {\n      \"id\": \"JFK query\",\n      \"request\": { \"query\": { \"match_all\": {} } },\n      \"ratings\": []\n    } ],\n  \"metric\": {\n    \"precision\": {\n      \"k\": 20,\n      \"relevant_rating_threshold\": 1,\n      \"ignore_unlabeled\": false\n    }\n  }\n}"'

Request example

An example body for a `GET /my-index-000001/_rank_eval` request.

{
  "requests": [
    {
      "id": "JFK query",
      "request": { "query": { "match_all": {} } },
      "ratings": []
    } ],
  "metric": {
    "precision": {
      "k": 20,
      "relevant_rating_threshold": 1,
      "ignore_unlabeled": false
    }
  }
}

Evaluate ranked search results Generally available; Added in 6.2.0

Required authorization

Query parameters

Body Required

Responses

rating number | string | null