Introducing query rules in Elasticsearch 8.10

By

Kathleen DeRusso

intro-query-elasticsearch-8-10-0.png

We’re excited to announce query rules in Elasticsearch 8.10! Query rules allow you to change a query based on the query terms they are searching for, or based on context information provided as part of the search query.

What is a query rule?

Query rules allow customization of search results outside search relevance, which gives more control of results for targeted queries based on contextual information you provide. This enables more targeted search results for marketing campaigns, personalization, and segment-specific search results, all built into Elasticsearch®!

What types of rules are supported?

To start, we are supporting pinned query rules, which allow you to identify documents that you want to promote at the top of search results based on the context in a particular query.

  • Using pinned query rules, you can: 
  • Pin a result to preorder the newest iPhone when people search for iPhone
  • Pin promoted articles from other brands when people search for iPhone
  • Pin different results for users searching for “football” if they are located in the US or the UK
  • Pin important organizational announcements so it’s at the top of everyone’s search results
  • Pin information about upcoming employee performance reviews for all non-manager employees

… and so on!

What types of criteria are supported?

Query rule matching criteria may be any of the following: 

  • exact: Matches the specified value exactly
  • fuzzy: Matches the specified value within an allowed Levenshtein edit distance
  • prefix: Starts with the specified value
  • suffix: Ends with the specified value
  • contains: Contains the specified value
  • lt: Less than the specified value
  • lte: Less than or equal to the specified value
  • gt: Greater than the specified value
  • gte: Greater than or equal to the specified value
  • always: Always matches for all rule queries

What types of actions are supported?

For pinned query rules, actions can be ids to pin corresponding to indexed _id fields, or docs to pin corresponding _id fields in specified indices.

How do query rules work?

Under the hood, the process to create and use query rules is as follows: 

  1. An administrator creates a query ruleset containing one or more contextual query rules.

  2. Using the query rules management API, we store these query rules inside Elasticsearch.

  3. Searches use the rule_query, which includes a search query as well as a query ruleset and match criteria. 

  4. Elasticsearch identifies all rules in the ruleset that match the criteria specified in the query.

  5. Each matching rule is applied in the order in which it appears in the ruleset.

  6. During the query rewrite phase, this query is rewritten into a pinned query pinning the IDs or documents identified in the rules. 

  7. The new pinned query runs and results are returned with matching promoted results on top.

This diagram shows the lifecycle:

query-rules-sketch

Example

We’ll go through a short example. You can also follow along in Elasticsearch Labs!

A global ecommerce site wants to promote its new wireless charger, the PureJuice Pro.

The index includes the following documents:

POST /products/_doc/us1
{
  "name": "PureJuice Pro",
  "description": "PureJuice Pro: Experience the pinnacle of wireless charging. Blending rapid charging tech with sleek design, it ensures your devices are powered swiftly and safely. The future of charging is here.",
  "price": 15.00,
  "currency": "USD",
  "plug_type": "B",
  "voltage": "120v"
}

POST /products/_doc/uk1
{
  "name": "PureJuice Pro - UK Compatible",
  "description": "PureJuice Pro: Redefining wireless charging. Seamlessly merging swift charging capabilities with a refined aesthetic, it guarantees your devices receive rapid and secure power. Welcome to the next generation of charging.",
  "price": 20.00,
  "currency": "GBP",
  "plug_type": "G",
  "voltage": "230V"
}

POST /products/_doc/eu1
{
  "name": "PureJuice Pro - Wireless Charger suitable for European plugs",
  "description": "PureJuice Pro: Elevating wireless charging. Combining unparalleled charging speeds with elegant design, it promises both rapid and dependable energy for your devices. Embrace the future of wireless charging.",
  "price": 18.00,
  "currency": "EUR",
  "plug_type": "C",
  "voltage": "230V"
}

Searching this index for a wireless charger using the following query:

POST /products/_search
{
  "query": {
    "multi_match": {
      "query": "wireless charger",
      "fields": [ "name^5", "description" ]
    }
  }
}

This query will return the European charger first due to a title match — but we may want to promote different versions based on the location of our searchers.

Managing query rules

First, an administrator with manage_search_query_rules privilege defines and stores a ruleset using the query rules management API.

PUT /_query_rules/promotion-rules
{
  "rules": [
    {
      "rule_id": "us-charger",
      "type": "pinned",
      "criteria": [
        {
          "type": "contains",
          "metadata": "my_query",
          "values": ["wireless charger"]
        },
        {
          "type": "exact",
          "metadata": "country",
          "values": ["us"]
        }
      ],
      "actions": {
        "ids": [
          "us1"
        ]
      }
    },
    {
      "rule_id": "uk-charger",
      "type": "pinned",
      "criteria": [
        {
          "type": "contains",
          "metadata": "my_query",
          "values": ["wireless charger"]
        },
        {
          "type": "exact",
          "metadata": "country",
          "values": ["uk"]
        }
      ],
      "actions": {
        "ids": [
          "uk1"
        ]
      }
    }
  ]
}

In this ruleset, we define two rules: 

  • If a user’s search term contains the phrase “wireless charger” and they’re located in the US, we want to pin the US product at the top of the result set.
  • If a user performs the same search but is located in the UK, we want to pin the UK version of this product at the top of the result set. 

Tip: Because we’re transforming this into a pinned query under the hood, we need to respect a couple of ground rules: 

  • We can define either ids or docs to pin but not both. To save yourself pain, it’s a best practice to choose one or the other and be consistent in your usage in the query ruleset. 
  • Pinned queries can only support up to 100 pinned documents. Any given rule enforces this limit.

Searching query rules

When we’re searching against query rules, we want to use the rule_query. An example rule query may look like this:

POST /products/_search
{
  "query": {
    "rule_query": {
      "organic": {
        "multi_match": {
          "query": "reliable wireless charger for iPhone",
          "fields": [
            "name^5",
            "description"
          ]
        }
      },
      "match_criteria": {
        "my_query": "reliable wireless charger for iPhone",
        "country": "us"
      },
      "ruleset_id": "promotion-rules"
    }
  }
}

This query is composed of the following parts: 

  • An organic query, which is the search we want to run and rank documents. If no query rules are applied, this query is executed without any modifications.
  • Match criteria, which defines the criteria we want to match on. In this example, we have two pieces of match_criteria: 
    • my_query, which in this case is the same string that the user entered in the search box 
    • country, which was determined through using geolocation on the user’s IP address 
  • The ruleset ID, which determines the ruleset in which we look for matching rules. 

We validate each rule to determine whether it should be applied to the query. When we process the rule query, we find a matching rule for the query “wireless charger” coming from the US:

{
      "rule_id": "us-charger",
      "type": "pinned",
      "criteria": [
        {
          "type": "contains",
          "metadata": "my_query",
          "values": ["wireless charger"]
        },
        {
          "type": "exact",
          "metadata": "country",
          "values": ["us"]
        }
      ],
      "actions": {
        "ids": [
          "id1-us"
        ]
      }
    }

As a reminder, in order for the query rule to match, all criteria defined in the query rule must match the rule query input.

Before the query is executed, it is rewritten into a pinned query:

{
  "query": {
    "pinned": {
      "organic": {
        "query_string": {
          "query": "reliable wireless charger for iPhone"
        }
      },
      "ids": [
        "id1-us"
      ]
    }
  }
}

Elasticsearch searches using this new pinned query and returns the result with the pinned US version of the product at the top of the result set.

Conclusion

We’ve shown you how to define query rules to promote results based on contextual information like user-entered queries or personalization data, and how to search using these rules. 

Read about this capability and more in the Elastic® 8.10 release notes, and try searching with query rules for yourself with a free, 14-day trial of Elastic Cloud. We’d love to hear from you on GitHub and our Discuss forums.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

Sign up for Elastic Cloud free trial

Spin up a fully loaded deployment on the cloud provider you choose. As the company behind Elasticsearch, we bring our features and support to your Elastic clusters in the cloud.

Start free trial