Observability
editObservability
editThe client does not provide a default logger, but instead it offers an event
emitter interface to hook into internal events, such as request
and
response
.
Correlating those events can be hard, especially if your applications have a large codebase with many events happening at the same time.
To help you with this, the client offers you a correlation id system and other features. Let’s see them in action.
Events
editThe client is an event emitter, this means that you can listen for its event and
add additional logic to your code, without need to change the client internals
or your normal usage. You can find the events names by access the events
key
of the client.
const { events } = require('@elastic/elasticsearch') console.log(events)
The event emitter functionality can be useful if you want to log every request, response and error that is happening during the use of the client.
const logger = require('my-logger')() const { Client } = require('@elastic/elasticsearch') const client = new Client({ cloud: { id: '<cloud-id>' }, auth: { apiKey: 'base64EncodedKey' } }) client.diagnostic.on('response', (err, result) => { if (err) { logger.error(err) } else { logger.info(result) } })
The client emits the following events:
|
Emitted before starting serialization and compression. If you want to measure this phase duration, you should measure the time elapsed between this event and client.diagnostic.on('serialization', (err, result) => { console.log(err, result) }) |
|
Emitted before sending the actual request to Elasticsearch (emitted multiple times in case of retries). client.diagnostic.on('request', (err, result) => { console.log(err, result) }) |
|
Emitted before starting deserialization and decompression. If you want to measure this phase duration, you should measure the time elapsed between this event and client.diagnostic.on('deserialization', (err, result) => { console.log(err, result) }) |
|
Emitted once Elasticsearch response has been received and parsed. client.diagnostic.on('response', (err, result) => { console.log(err, result) }) |
|
Emitted when the client ends a sniffing request. client.diagnostic.on('sniff', (err, result) => { console.log(err, result) }) |
|
Emitted if the client is able to resurrect a dead node. client.diagnostic.on('resurrect', (err, result) => { console.log(err, result) }) |
The values of result
in serialization
, request
, deserialization
,
response
and sniff
are:
body: any; statusCode: number | null; headers: anyObject | null; warnings: string[] | null; meta: { context: any; name: string; request: { params: TransportRequestParams; options: TransportRequestOptions; id: any; }; connection: Connection; attempts: number; aborted: boolean; sniff?: { hosts: any[]; reason: string; }; };
While the result
value in resurrect
is:
strategy: string; isAlive: boolean; connection: Connection; name: string; request: { id: any; };
Events order
editThe event order is described in the following graph, in some edge cases, the
order is not guaranteed.
You can find in
test/acceptance/events-order.test.js
how the order changes based on the situation.
serialization │ │ (serialization and compression happens between those two events) │ └─▶ request │ │ (actual time spent over the wire) │ └─▶ deserialization │ │ (deserialization and decompression happens between those two events) │ └─▶ response
Correlation id
editCorrelating events can be hard, especially if there are many events at the same time. The client offers you an automatic (and configurable) system to help you handle this problem.
const { Client } = require('@elastic/elasticsearch') const client = new Client({ cloud: { id: '<cloud-id>' }, auth: { apiKey: 'base64EncodedKey' } }) client.diagnostic.on('request', (err, result) => { const { id } = result.meta.request if (err) { console.log({ error: err, reqId: id }) } }) client.diagnostic.on('response', (err, result) => { const { id } = result.meta.request if (err) { console.log({ error: err, reqId: id }) } }) client.search({ index: 'my-index', query: { match_all: {} } }).then(console.log, console.log)
By default the id is an incremental integer, but you can configure it with the
generateRequestId
option:
const { Client } = require('@elastic/elasticsearch') const client = new Client({ cloud: { id: '<cloud-id>' }, auth: { apiKey: 'base64EncodedKey' }, // it takes two parameters, the request parameters and options generateRequestId: function (params, options) { // your id generation logic // must be syncronous return 'id' } })
You can also specify a custom id per request:
client.search({ index: 'my-index', query: { match_all: {} } }, { id: 'custom-id' }).then(console.log, console.log)
Context object
editSometimes, you might need to make some custom data available in your events, you
can do that via the context
option of a request:
const { Client } = require('@elastic/elasticsearch') const client = new Client({ cloud: { id: '<cloud-id>' }, auth: { apiKey: 'base64EncodedKey' } }) client.diagnostic.on('request', (err, result) => { const { id } = result.meta.request const { context } = result.meta if (err) { console.log({ error: err, reqId: id, context }) } }) client.diagnostic.on('response', (err, result) => { const { id } = result.meta.request const { winter } = result.meta.context if (err) { console.log({ error: err, reqId: id, winter }) } }) client.search({ index: 'my-index', query: { match_all: {} } }, { context: { winter: 'is coming' } }).then(console.log, console.log)
The context object can also be configured as a global option in the client configuration. If you provide both, the two context objects will be shallow merged, and the API level object will take precedence.
const { Client } = require('@elastic/elasticsearch') const client = new Client({ cloud: { id: '<cloud-id>' }, auth: { apiKey: 'base64EncodedKey' }, context: { winter: 'is coming' } }) client.diagnostic.on('request', (err, result) => { const { id } = result.meta.request const { context } = result.meta if (err) { console.log({ error: err, reqId: id, context }) } }) client.diagnostic.on('response', (err, result) => { const { id } = result.meta.request const { winter } = result.meta.context if (err) { console.log({ error: err, reqId: id, winter }) } }) client.search({ index: 'my-index', query: { match_all: {} } }, { context: { winter: 'has come' } }).then(console.log, console.log)
Client name
editIf you are using multiple instances of the client or if you are using multiple
child clients (which is the recommended way to have multiple instances of the
client), you might need to recognize which client you are using. The name
options help you in this regard.
const { Client } = require('@elastic/elasticsearch') const client = new Client({ cloud: { id: '<cloud-id>' }, auth: { apiKey: 'base64EncodedKey' }, name: 'parent-client' // default to 'elasticsearch-js' }) const child = client.child({ name: 'child-client' }) console.log(client.name, child.name) client.diagnostic.on('request', (err, result) => { const { id } = result.meta.request const { name } = result.meta if (err) { console.log({ error: err, reqId: id, name }) } }) client.diagnostic.on('response', (err, result) => { const { id } = result.meta.request const { name } = result.meta if (err) { console.log({ error: err, reqId: id, name }) } }) client.search({ index: 'my-index', query: { match_all: {} } }).then(console.log, console.log) child.search({ index: 'my-index', query: { match_all: {} } }).then(console.log, console.log)
X-Opaque-Id support
editTo improve observability, the client offers an easy way to configure the
X-Opaque-Id
header. If you set the X-Opaque-Id
in a specific request, this
allows you to discover this identifier in the
deprecation logs,
helps you with identifying search slow log origin
as well as identifying running tasks.
The X-Opaque-Id
should be configured in each request, for doing that you can
use the opaqueId
option, as you can see in the following example. The
resulting header will be { 'X-Opaque-Id': 'my-search' }
.
const { Client } = require('@elastic/elasticsearch') const client = new Client({ cloud: { id: '<cloud-id>' }, auth: { apiKey: 'base64EncodedKey' } }) client.search({ index: 'my-index', body: { foo: 'bar' } }, { opaqueId: 'my-search' }).then(console.log, console.log)
Sometimes it may be useful to prefix all the X-Opaque-Id
headers with a
specific string, in case you need to identify a specific client or server. For
doing this, the client offers a top-level configuration option:
opaqueIdPrefix
. In the following example, the resulting header will be
{ 'X-Opaque-Id': 'proxy-client::my-search' }
.
const { Client } = require('@elastic/elasticsearch') const client = new Client({ cloud: { id: '<cloud-id>' }, auth: { apiKey: 'base64EncodedKey' }, opaqueIdPrefix: 'proxy-client::' }) client.search({ index: 'my-index', body: { foo: 'bar' } }, { opaqueId: 'my-search' }).then(console.log, console.log)