You are looking at preliminary documentation for a future release. Not what you want? See the current release documentation.
Elastic Docs ›Elasticsearch Guide [master] ›REST APIs ›Connector APIs
« Update connector error API Update connector index name API »

Update connector filtering APIedit

This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

Updates the draft filtering configuration of a connector and marks the draft validation state as edited. The filtering configuration can be activated once validated by the Elastic connector service.

The filtering property is used to configure sync rules (both basic and advanced) for a connector. Learn more in the sync rules documentation.

Requestedit

PUT _connector/<connector_id>/_filtering PUT _connector/<connector_id>/_filtering/_activate

Prerequisitesedit

  • To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for native connectors.
  • The connector_id parameter should reference an existing connector.
  • To activate filtering rules, the draft.validation.state must be valid.

Path parametersedit

<connector_id>
(Required, string)

Request bodyedit

rules

(Optional, array of objects) An array of basic sync rules, each with the following sub-attributes:

  • id (Required, string)
    A unique identifier for the rule.
  • policy (Required, string)
    Specifies the policy, such as include or exclude.
  • field (Required, string)
    The field in the document to which this rule applies.
  • rule (Required, string)
    The type of rule, such as regex, starts_with, ends_with, contains, equals, <, >, etc.
  • value (Required, string)
    The value to be used in conjunction with the rule for matching the contents of the document’s field.
  • order (Required, number)
    The order in which the rules are applied. The first rule to match has its policy applied.
  • created_at (Optional, datetime)
    The timestamp when the rule was added. Defaults to now UTC timestamp.
  • updated_at (Optional, datetime)
    The timestamp when the rule was last edited. Defaults to now UTC timestamp.
advanced_snippet

(Optional, object) Used for advanced filtering at query time, with the following sub-attributes:

  • value (Required, object or array)
    A JSON object/array passed directly to the connector for advanced filtering.
  • created_at (Optional, datetime)
    The timestamp when this JSON object was created. Defaults to now UTC timestamp.
  • updated_at (Optional, datetime)
    The timestamp when this JSON object was last edited. Defaults to now UTC timestamp.

Response codesedit

200
Connector draft filtering was successfully updated.
400
The connector_id was not provided or the request payload was malformed.
404 (Missing resources)
No connector matching connector_id could be found.

Examplesedit

The following example updates the draft basic sync rules for a Google Drive connector with ID my-g-drive-connector. All Google Drive files with .txt extension will be skipped:

PUT _connector/my-g-drive-connector/_filtering
{
    "rules": [
         {
            "field": "file_extension",
            "id": "exclude-txt-files",
            "order": 0,
            "policy": "exclude",
            "rule": "equals",
            "value": "txt"
        },
        {
            "field": "_",
            "id": "DEFAULT",
            "order": 1,
            "policy": "include",
            "rule": "regex",
            "value": ".*"
        }
    ]
}
{
    "result": "updated"
}

The following example updates the draft advanced sync rules for a MySQL connector with id my-sql-connector. Advanced sync rules are specific to each connector type. Refer to the references for connectors that support advanced sync rules for syntax and examples.

PUT _connector/my-sql-connector/_filtering
{
    "advanced_snippet": {
        "value": [{
            "tables": [
                "users",
                "orders"
            ],
            "query": "SELECT users.id AS id, orders.order_id AS order_id FROM users JOIN orders ON users.id = orders.user_id"
        }]
    }
}
{
    "result": "updated"
}

Note, you can also update draft rules and advanced_snippet in a single request.

Once the draft is updated, its validation state is set to edited. The connector service will then validate the rules and report the validation state as either invalid or valid. If the state is valid, the draft filtering can be activated with:

PUT _connector/my-sql-connector/_filtering/_activate
{
    "result": "updated"
}

Once filtering rules are activated, they will be applied to all subsequent full or incremental syncs.

« Update connector error API Update connector index name API »