API logs APIedit

Displays API request and response data at the Engine level.

Account-level operations, such as requests to Credentials, do not appear...

... What does appear? Everything else!

API Endpoint Logged?

engines

Yes

documents

Yes

search

Yes

synonyms

Yes

clickthrough

Yes

analytics

Yes

curations

Yes

logs/api

Yes

credentials

No

Every authenticated GET, PUT, POST or DELETE received during a search query, analytics request, document creation - any Engine event of any kind - will be recorded within the API Log.

Even requests against the API Log itself are recorded within the API Log.

Read more:

API Log data is robust. The examples each show one result in full, for clarity.

Log retentionedit

Since 7.9.0, App Search retains API event data according to your Enterprise Search log retention settings.

See Log retention in the Enterprise Search documentation.

Log settingsedit

Since 7.9.0, you can disable API event logging using the App Search log settings.

See Log settings guide.

Authenticationedit

For authentication, the API Logs endpoint requires...

  1. A Host Identifier: [HOST_IDENTIFIER]
  2. The name of your Engine: [ENGINE]
  3. A Private API Key: [PRIVATE_API_KEY]
curl -X GET 'https://[instance id].ent-search.[region].[provider].cloud.es.io/api/as/v1/engines/[ENGINE]/logs/api' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer [PRIVATE_API_KEY]'

Response Objectedit

By default, you will receive 10 results per page in ascending order.

A result will include:

  • Time-stamp
  • HTTP method
  • Full request path
  • Status code
  • Request body and response
  • User Agent queried
// An example JSON payload from the logs/api endpoint.
{
  "results": [
    {
      "timestamp": string,
      "http_method": string,
      "path": string,
      "full_request_path": string,
      "status": number,
      "request_body": string,
      "response_body": string,
      "user_agent": string
    }
  ],
  "meta": {
    "query": string,
    "filters": {
      "date": {
        "from": string,
        "to": string
      }
    },
    "page": {
      "current": number,
      "total_pages": number,
      "total_results": number,
      "size": number
    }
  }
}
Searching the API Logedit
GET /api/as/v1/engines/[ENGINE]/logs/api
POST /api/as/v1/engines/[ENGINE]/logs/api
query (optional)
You can search over the full_request_path of an API Log event. Use this to specify a particular endpoint, like analytics, search, curations and so on.
sort_direction (optional)
Would you like to have your results ascending, oldest to newest, or descending, newest to oldest? The key accepts asc or desc. Defaults to ascending.
page (optional)
Can provide... current for current page total_pages for the net number of pages total_results for the overall number of results size for the amount of results per page. Expects integers within all fields.

Example - A GET request for 5 pages of search events between October 15th and 16th with 20 results on each page, in descending order.

curl -X GET 'https://[instance id].ent-search.[region].[provider].cloud.es.io/api/as/v1/engines/national-parks-demo/logs/api' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer private-xxxxxxxxxxxxxxxxxxxx' \
-d '{
  "filters": {
    "date": {
      "from": "2018-10-15T00:00:00+00:00",
      "to": "2018-10-16T00:00:00+00:00"
    }
  },
  "page": {
    "total_results": 100,
    "size": 20
  },
  "query": "search",
  "sort_direction": "desc"
}'

Example Response

{
  "results": [
    {
      "timestamp": "2018-10-15T20:42:14Z",
      "http_method": "POST",
      "path": "/api/as/v1/engines/national-parks-demo/search",
      "full_request_path": "/api/as/v1/engines/national-parks-demo/search",
      "status": 200,
      "request_body": "{\n  \"query\": \"everglade\"\n}",
      "response_body": "{\"meta\":{\"warnings\":[],\"page\":{\"current\":1,\"total_pages\":1,\"total_results\":3,\"size\":10},\"request_id\":\"164c8262b4a82039def310f8139f68d2\"},\"results\":[{\"nps_link\":{\"raw\":\"https://www.nps.gov/ever/index.htm\"},\"title\":{\"raw\":\"Everglades\"},\"date_established\":{\"raw\":\"1934-05-30T05:00:00+00:00\"},\"world_heritage_site\":{\"raw\":\"true\"},\"states\":{\"raw\":[\"Florida\"]},\"description\":{\"raw\":\"The Everglades are the largest tropical wilderness in the United States. This mangrove and tropical rainforest ecosystem and marine estuary is home to 36 protected species, including the Florida panther, American crocodile, and West Indian manatee. Some areas have been drained and developed; restoration projects aim to restore the ecology.\"},\"visitors\":{\"raw\":930907.0},\"id\":{\"raw\":\"park_everglades\"},\"location\":{\"raw\":\"25.32,-80.93\"},\"square_km\":{\"raw\":6106.6},\"acres\":{\"raw\":1508968.1},\"_meta\":{\"score\":1.0028129}},{\"nps_link\":{\"raw\":\"https://www.nps.gov/voya/index.htm\"},\"title\":{\"raw\":\"Voyageurs\"},\"date_established\":{\"raw\":\"1971-01-08T06:00:00+00:00\"},\"world_heritage_site\":{\"raw\":\"false\"},\"states\":{\"raw\":[\"Minnesota\"]},\"description\":{\"raw\":\"This park protecting four lakes near the Canada\\u2013US border is a site for canoeing, kayaking, and fishing. The park also preserves a history populated by Ojibwe Native Americans, French fur traders called voyageurs, and gold miners. Formed by glaciers, the region features tall bluffs, rock gardens, islands, bays, and several historic buildings.\"},\"visitors\":{\"raw\":241912.0},\"id\":{\"raw\":\"park_voyageurs\"},\"location\":{\"raw\":\"48.5,-92.88\"},\"square_km\":{\"raw\":883.0},\"acres\":{\"raw\":218200.15},\"_meta\":{\"score\":0.0062864386}},{\"nps_link\":{\"raw\":\"https://www.nps.gov/wrst/index.htm\"},\"title\":{\"raw\":\"Wrangell\\u2013St. Elias\"},\"date_established\":{\"raw\":\"1980-12-02T06:00:00+00:00\"},\"world_heritage_site\":{\"raw\":\"true\"},\"states\":{\"raw\":[\"Alaska\"]},\"description\":{\"raw\":\"An over 8 million acres (32,375 km2) plot of mountainous country\\u2014the largest National Park in the system\\u2014protects the convergence of the Alaska, Chugach, and Wrangell-Saint Elias Ranges, which include many of the continent's tallest mountains and volcanoes, including the 18,008-foot Mount Saint Elias. More than a quarter of the park is covered with glaciers, including the tidewater Hubbard Glacier, piedmont Malaspina Glacier, and valley Nabesna Glacier.\"},\"visitors\":{\"raw\":79047.0},\"id\":{\"raw\":\"park_wrangell\\u2013st.-elias\"},\"location\":{\"raw\":\"61,-142\"},\"square_km\":{\"raw\":33682.6},\"acres\":{\"raw\":8323146.48},\"_meta\":{\"score\":0.00437395}}]}",
      "user_agent": "curl/7.54.0"
    },

    ## ... Truncated!

  ],
  "meta": {
    "query": "search",
    "filters": {
      "date": {
        "from": "2018-10-15T00:00:00+00:00",
        "to": "2018-10-16T00:00:00+00:00"
      }
    },
    "sort_direction": "desc",
    "page": {
      "current": 1,
      "total_pages": 1,
      "total_results": 9,
      "size": 20
    }
  }
}
Filtering the API Logedit

Filter the API Log.

A time-frame must be provided.

You can combine all parameters within the filters object.

GET /api/as/v1/engines/[ENGINE]/logs/api
POST /api/as/v1/engines/[ENGINE]/logs/api
filters/date (required)
Requires a nested date object under filters containing from and to keys. The keys must contain dates formatted in accordance with RFC3339. You may omit the time: YYYY-MM-DD.
filters/status (optional)
Filter based on a particular status code: 400, 401, 403, 429, 200.
filters/method (optional)
Filter based on a particular HTTP method: GET, POST, PUT, PATCH, DELETE.
Date Filteredit

Example - A GET request to receive all API Log events between October 15th and 16th.

curl -X GET 'https://[instance id].ent-search.[region].[provider].cloud.es.io/api/as/v1/engines/national-parks-demo/logs/api' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer private-xxxxxxxxxxxxxxxxxxxx' \
-d '{
  "filters": {
    "date": {
      "from": "2018-10-15T00:00:00+00:00",
      "to": "2018-10-16T00:00:00+00:00"
    }
  }
}'

Example Response

{
  "results": [
    {
      "timestamp": "2018-10-15T17:01:33Z",
      "http_method": "GET",
      "path": "/api/as/v1/engines/national-parks-demo/search",
      "full_request_path": "/api/as/v1/engines/national-parks-demo/search",
      "status": 200,
      "request_body": "{\"result_fields\": {\"states\": {\"raw\": {}}, \"location\": {\"raw\": {}}, \"title\": {\"raw\": {}}}, \"query\": \"california\", \"group\": {\"field\": \"states\"}}",
      "response_body": "{\"meta\":{\"warnings\":[],\"page\":{\"current\":1,\"total_pages\":1,\"total_results\":2,\"size\":10},\"request_id\":\"700de28b9ae89e05e7114353eed7fbee\"},\"results\":[{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Channel Islands\"},\"location\":{\"raw\":\"34.01,-119.42\"},\"id\":{\"raw\":\"park_channel-islands\"},\"_meta\":{\"score\":5.2469006},\"_group\":[{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Joshua Tree\"},\"location\":{\"raw\":\"33.79,-115.9\"},\"id\":{\"raw\":\"park_joshua-tree\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Kings Canyon\"},\"location\":{\"raw\":\"36.8,-118.55\"},\"id\":{\"raw\":\"park_kings-canyon\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Lassen Volcanic\"},\"location\":{\"raw\":\"40.49,-121.51\"},\"id\":{\"raw\":\"park_lassen-volcanic\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Pinnacles\"},\"location\":{\"raw\":\"36.48,-121.16\"},\"id\":{\"raw\":\"park_pinnacles\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Redwood\"},\"location\":{\"raw\":\"41.3,-124\"},\"id\":{\"raw\":\"park_redwood\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Sequoia\"},\"location\":{\"raw\":\"36.43,-118.68\"},\"id\":{\"raw\":\"park_sequoia\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Yosemite\"},\"location\":{\"raw\":\"37.83,-119.5\"},\"id\":{\"raw\":\"park_yosemite\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\",\"Nevada\"]},\"title\":{\"raw\":\"Death Valley\"},\"location\":{\"raw\":\"36.24,-116.82\"},\"id\":{\"raw\":\"park_death-valley\"},\"_meta\":{\"score\":3.3266773}}],\"_group_key\":\"California\"},{\"states\":{\"raw\":[\"California\",\"Nevada\"]},\"title\":{\"raw\":\"Death Valley\"},\"location\":{\"raw\":\"36.24,-116.82\"},\"id\":{\"raw\":\"park_death-valley\"},\"_meta\":{\"score\":3.3266773},\"_group\":[],\"_group_key\":\"Nevada\"}]}",
      "user_agent": "python-requests/2.19.1"
    },

    ## ... Truncated!

  ],
  "meta": {
    "query": "",
    "filters": {
      "date": {
        "from": "2018-10-15T00:00:00+00:00",
        "to": "2018-10-16T00:00:00+00:00"
      }
    },
    "sort_direction": "asc",
    "page": {
      "current": 1,
      "total_pages": 2,
      "total_results": 20,
      "size": 10
    }
  }
}
Status Filteredit

Example - A GET request to receive all API Log events between February 1st and 5th with a status code of 400.

curl -X GET 'https://[instance id].ent-search.[region].[provider].cloud.es.io/api/as/v1/engines/national-parks-demo/logs/api' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer private-xxxxxxxxxxxxxxxxxxxx' \
-d '{
  "filters": {
    "date": {
      "from": "2019-02-01T00:00:00+00:00",
      "to": "2019-02-05T00:00:00+00:00"
    },
    "status": "400"
  }
}'

Example Response

{
  "results": [
    {
      "timestamp": "2019-02-01T16:06:06Z",
      "http_method": "POST",
      "path": "/api/as/v1/engines/national-parks-demo/query_suggestion",
      "full_request_path": "/api/as/v1/engines/national-parks-demo/query_suggestion",
      "status": 400,
      "request_body": "{\"query\":\"z\",\"types\":{\"documents\":{\"fields\":[\"title\",\"state\"]}}}",
      "response_body": "{\"errors\":[\"Types documents fields contains invalid value: state\"]}",
      "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0.2 Safari/605.1.15"
    },

    # ... Truncated!

  ],
  "meta": {
    "query": "",
    "filters": {
      "date": {
        "from": "2019-02-01T00:00:00+00:00",
        "to": "2019-02-05T00:00:00+00:00"
      },
      "status": "400"
    },
    "sort_direction": "asc",
    "page": {
      "current": 1,
      "total_pages": 3,
      "total_results": 27,
      "size": 10
    }
  }
}
Method Filteredit

Example - A GET request to receive all API Log events between February 1st and 5th with a method of POST.

curl -X GET 'https://[instance id].ent-search.[region].[provider].cloud.es.io/api/as/v1/engines/national-parks-demo/logs/api' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer private-xxxxxxxxxxxxxxxxxxxx' \
-d '{
  "filters": {
    "date": {
      "from": "2019-02-01T00:00:00+00:00",
      "to": "2019-02-05T00:00:00+00:00"
    },
    "method": "POST"
  }
}'

Example Response

{
  "results": [
    {
      "timestamp": "2019-02-01T01:13:04Z",
      "http_method": "POST",
      "path": "/api/as/v1/engines/national-parks-demo/search.json",
      "full_request_path": "/api/as/v1/engines/national-parks-demo/search.json",
      "status": 200,
      "request_body": "{\"query\":\"example\",\"facets\":{\"states\":{\"type\":\"value\",\"size\":10},\"world_heritage_site\":{\"type\":\"value\",\"size\":10}},\"result_fields\":{\"description\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"nps_link\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"location\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"states\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"title\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"visitors\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"world_heritage_site\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"acres\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"square_km\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"date_established\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"id\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}}},\"search_fields\":{\"description\":{},\"nps_link\":{},\"location\":{},\"states\":{},\"title\":{},\"visitors\":{},\"world_heritage_site\":{},\"acres\":{},\"square_km\":{},\"date_established\":{},\"id\":{}},\"page\":{\"current\":1,\"size\":20},\"filters\":{\"all\":[]}}",

    # ... Truncated!

  ],
  "meta": {
    "query": "",
    "filters": {
      "date": {
        "from": "2019-02-01T00:00:00+00:00",
        "to": "2019-02-05T00:00:00+00:00"
      },
      "method": "POST"
    },
    "sort_direction": "asc",
    "page": {
      "current": 1,
      "total_pages": 18,
      "total_results": 173,
      "size": 10
    }
  }
}

What’s Next?edit

Your API Log is a well-spring of valuable Engine-level insight. If you want to start looking deeper into how visitors run queries against your Engines, the Analytics and Clickthrough endpoints are worth exploring. If you want to start improving the relevance of your queries, Synonyms are an excellent place to start.