Node.js Agent version 3.xedit

3.40.1 2022/11/15edit

Bug fixesedit
  • Prevent a possible tight loop in central config fetching. (#3029)

3.40.0 2022/10/31edit

Featuresedit
  • Enable support for redis v4 (#2945)
  • [preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will apply best effort to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. Next.js server-side instrumentation. See the Get started with Next.js document.

    This adds instrumentation of the Next.js dev server (next dev) and prod server (next start). The APM transactions for incoming HTTP requests to the server will be named appropriately based on Next.js’s routing — both for user page routes (e.g. GET /a-dynamic-page/[id]) and for internal Next.js routes (e.g. Next.js _next/data route my-page, Next.js Rewrite route /foo -> /bar). As well, exceptions in server-side code (e.g. getServerSideProps, server-side run page handlers, API handlers) will be reported. (#2959)

    This is a technical preview to get feedback from Next.js users. The details on how exactly the instrumentation works may change in future versions.

  • Improve container-info gathering to support AWS ECS/Fargate environments. (#2914)
Bug fixesedit
  • Source lines of context in stacktraces is no longer reported for "*.min.js" files that do not have source-map information. These files are assumed to be minimized files, for which source line context won’t be useful. This change is to guard against excessively large stacktrace data.
Choresedit
  • Add guards to ensure that a crazy Cache-Control: max-age=... response header cannot accidentally result in inappropriate intervals for fetching central config. The re-fetch delay is clamped to [5 seconds, 1 day]. (#2941)

3.39.0 2022/10/17edit

Featuresedit
  • Improve the granularity of data captured about downstream services, e.g. databases, for spans that represent an external call (known as "exit spans"). This data is used for Service Maps and Dependencies in the Kibana APM app.

    This is handled via the new span service.target.* fields that replace the deprecated destination.service.resource field (spec). All instrumentations have been updated to set appropriate service target values. If necessary, e.g. for manual instrumentation, a new public span.setServiceTarget(type, name) API has been added to specify these values. (#2882)

    The never-public-but-available span.setDestinationContext() has been marked for removal (using it will process.emitWarning()). Users of this internal method should switch to the public span.setServiceTarget(type, name).

    As part of this change, improvements have been made to some module instrumentations:

    • redis and ioredis: span.type has changed from "cache" to "db" per spec
    • mongodb: span.action used to be "query", now it will be the mongodb command name, e.g. "find", "insert".
    • mongodb and mongodb-core: span.db.instance is now set to the database name (#1494)
    • mysql and mysql2: span.db.{instance,user} are now populated.
    • @elastic/elasticsearch: The cluster name is heuristically determined for Elastic Cloud deployments and used for the service target name.
    • sqs: span.destination.{address,port} are now populated.
    • pg: span.db.{instance,user} are now populated.
    • cassandra-driver: the Cassandra keyspace is captured for service target data, if available.
    • OpenTelemetry Bridge: OTel spans with kind PRODUCER and CLIENT are now handled as exit spans (e.g. span compression could apply).
  • Support instrumentation of @koa/router (and koa-router) versions 11 and 12. Contributed by @sibelius. (#2811)
  • Support instrumentation of tedious@15. (#2897)
  • Improve the captured information for Elasticsearch client instrumentation. For all outgoing Elasticsearch client requests, the full HTTP url is now captured (stored in the "url.original" field). For Elasticsearch requests that do a search, the outgoing request body is captured (to the "span.db.statement" field) as before, but the format has changed to only hold the request body. Before this change the "span.db.statement" would also hold any HTTP query parameters. These are now more naturally captured in "url.original". (#2019)

    This change also introduces the elasticsearchCaptureBodyUrls configuration option to enable controlling which Elasticsearch REST API paths are considered for request body capture. (#2873)

  • Support instrumenting core modules when require’d with the optional node:-prefix. For example require('node:http') will now be instrumented. (#2816)
  • Agent will delay loading of the error-callsites module until agent start time, and will not load the module if the agent is disabled/inactive. This prevents the setting of an Error.prepareStackTrace handler until necessary for stacktrace collection. (#2833 #2906)
  • Add *principal* pattern to default value for sanitizeFieldNames config var, so that it is more likely to redact authentication-related HTTP headers, e.g. on Azure. (#2938)
Bug fixesedit
  • Avoid a possible RangeError: Maximum call stack size exceeded in Span timer handler for exceedingly deep Span trees. (#2939)
  • Fix instrumentation of (very old) graphql module versions ⇐0.9.6. Instrumentation of these older graphql versions was broken in v3.36.0. (#2927)
Choresedit
  • Disable knex instrumentation when not collecting span stack traces (because there is no point). This is a performance improvement for Knex usage in the default configuration. (#2879)
  • Document and add types for parent option to apm.captureError(). (#2977)

3.38.0 2022/08/11edit

Featuresedit
  • Add instrumentation for the undici HTTP client library. This also adds instrumentation of Node.js v18’s fetch(), which uses undici under the hood. For the instrumentation to work one must be using node v14.17.0 or later, or have installed the diagnostics_channel polyfill. (#2383)
  • Added exitSpanMinDuration configuration field, allowing end users to set a time threshold for dropping exit spans. (#2843)
Bug fixesedit
  • Capturing an error would fail if the Error instance had an attribute that was an invalid date. (#2030)
  • Fix the span for an instrumented S3 ListBuckets API call to not be invalid for APM server intake. (#2866)
  • Fix an issue where the transaction name for a trace of a Lambda function implementing a GraphQL server (e.g. via apollo-server-lambda) would not get the GraphQL-specific naming. (#2832)

3.37.0 2022/07/18edit

Featuresedit
  • The agent will now use AsyncLocalStorage for run-context tracking in new enough versions of Node.js (versions >=14.5 and >=12.19). This can reduce overhead from using the APM agent, especially in Promise-heavy applications. (#2786)

    This also adds a new contextManager configuration option to control which mechanism the agent uses for run-context tracking. It replaces the, now deprecated, asyncHooks configuration option. If you experience problems with the new AsyncLocalStorage-based tracking, you can restore the older behavior with contextManager: "asynchooks".

Choresedit
  • The old "patch" mechanism that the APM agent uses for run-context tracking (enabled via contextManager: "patch", or previously enabled via asyncHooks: false) is now deprecated. It will be removed in a future major version (after an 18 month deprecation period).

3.36.0 2022/06/15edit

Featuresedit
  • Adds dropped span statistics to transaction payloads allowing APM Server to calculate more accurate throughput metrics. (#2302)
  • Improve the grouping of captured API errors from @elastic/elasticsearch instrumentation. When an Elasticsearch client API error is captured, if the response body includes a error.type, e.g. illegal_argument_exception, the captured error.exception.type will be ResponseError (illegal_argument_exception) rather than ResponseError. This means that API errors will be grouped separately in the Kibana APM app based on their client API error type. (#2770)
  • Graphql v16 support (#2508)
Bug fixesedit
  • Fix the automatic wrapping of Lambda handlers to support handler modules created by esbuild bundling — as is done in some Serverless Framework functions that use TypeScript. (#2753)
  • Fix Express route tracking (used for transaction.name) when an argument is passed to the next(arg) callback of a request handler. Before this change passing next(<some object not an instance of Error>) would be considered an error by Express, but not by the APM agent’s route tracking. (#2750)
  • Updated sanitizeFieldNames and ignoreMessageQueues environment variables to use ELASTIC_APM_ prefix. (previous variable names are still recgonized, but not documented) (#2636)

3.35.0 2022/06/01edit

Featuresedit
  • Add support for knex version v0.21 to v1 (#2699). Note that instrumentation of knex >=0.95.0 is not support when using the deprecated contextManager=patch configuration option.
  • Change the instrumentation of SQS- and SNS-triggered AWS Lambda invocations: The special-casing of triggers with a single message/record has been removed. That means that instead of a possible continued distributed trace (if a single received message has a traceparent), a span link will be added to the APM transaction for each message with a traceparent. transaction.context.message. fields are no longer collected. (#2708)
  • Enable support for ioredis v5 (#2714)
  • A Docker image with the APM agent will be published for each release to docker.elastic.co/observability/apm-agent-nodejs:VERSION, for example: docker.elastic.co/observability/apm-agent-nodejs:3.35.0. (#2742)
Bug fixesedit
  • Fixes automatic Lambda handler wrapping to work with handlers that point to subfolders (ex. _HANDLER=path/to/folder.methodName) (#2709)

3.34.0 2022/05/26edit

Featuresedit
  • Add support for tedious version v10 to v14 (#2517)
  • When automatically determining serviceName and serviceVersion by looking for a "package.json", the agent will now prefer to start looking from the directory of the script being executed, rather than the current working directory. (#2420)
  • Add an experimental OpenTelemetry bridge. Briefly, the OpenTelemetry Bridge allows one to use the vendor-neutral OpenTelemetry Tracing API (@opentelemetry/api) to manually instrument your code, and have the Elastic Node.js APM agent handle those API calls. (#2641)
  • Add Span Links support. (#2673)

    The transaction.startSpan() and apm.startTransaction() public APIs now accept a links option for specify links. The OpenTelemetry Bridge also supports specifying links during span creation (with the limitation that span link attributes are not supported).

  • Add a traceContinuationStrategy configuration option to allow some control over how the APM Agent uses incoming trace-context headers for context propagation. (#2592)
  • Add span links to AWS SQS messaging spans on ReceiveMessage, one for each message (up to 1000) which has a traceparent message attribute. (#2593)
  • Add "nodejs16.x" as one of the compatible runtimes for the Node.js APM agent Lambda layers now that this runtime is available on AWS.
Bug fixesedit
  • Fixes a bug where the the agent would not serialize the database context of a span. (#2715)
  • Fix a possible crash in span compression handling on a span that was manually created without a parent span (e.g. if created with a custom childOf option). (#2701)
Choresedit
  • Add a package-lock.json file to ensure repeatable builds of the AWS Lambda layer and to assist with security issue auditing. (#2626)
  • Deprecate instrumentation for the legacy "hapi" package. While the APM agent still supports it, that instrumentation is no longer tested and support will be dropped in the next major version of the agent. Note that the "@hapi/hapi" package is still fully supported. (#2698)
  • Deprecate instrumentation for the obsolete "jade" package. "jade" was renamed to "pug" in 2015. While the APM agent still supports "jade", that instrumentation is no longer tested. (#2711)

3.33.0 2022/05/05edit

Featuresedit
  • Add a parent option to agent.captureError(err[, options][, cb]) to allow passing in a Transaction or Span to use as the parent for the error. Before this change the current span or transaction, if any, was always used.

    This option is not documented in the user docs, nor added to the TypeScript types, because it is only expected to be useful for coming OTel Bridge work.

Bug fixesedit
  • Fix a possible crash in the instrumentation of an incoming HTTP/2 request: if the underlying Http2Session was destroyed before the APM transaction was ended (on Http2Stream end). This resulted in the instrumentation using the [stream.session.socket](https://nodejs.org/api/http2.html#http2sessionsocket) proxy, which can throw ERR_HTTP2_SOCKET_UNBOUND after the session is destroyed. (#2670)
Choresedit
  • The release process is slightly changed. CI (Jenkins) now handles npm publish ... when a tag is pushed. (#2667)
  • Pulled the traceparent NPM module into a local module and replaced the random-poly-fill module with the built in require('crypto').randomFillSync function call (#2669)

3.32.0 2022/04/27edit

Featuresedit
  • Add support for node v18. (#2652)
  • Add support for span compression. (#2100, #2604)

    By default, consecutive (sibling) exit spans of the same name, type, subtype, and destination with a duration of less than 50ms will be compressed into a single composite span. A possible case is the N+1 query problem. Traces with many consecutive matching spans will be represented — both in data and the APM UI — more efficiently.

    Span compression can be disabled or matching behavior configured with the spanCompression* configuration options.

  • Marks spans as "exit spans" across all instrumentations, preventing additional child spans from being added to the exit spans. See issue for a full list of spans types that will be treated as exit spans. (#2601)
  • Allow a new span to be created/started even if its transaction has ended. This is expected to be a very rare use case. (#2653)
  • The Trace Context headers are now propagated for http2 requests. (#2656)

3.31.0 2022/03/23edit

Featuresedit
  • Add captureBody support for Hapi. (#1905)
  • If a SNS or SQS single event trigger to an instrumented Lambda function includes message attributes with the name "traceparent" (and "tracestate"), case-insensitive, then those are used to continue the trace. This was already being done for API Gateway event headers.
Bug fixesedit
  • Fix a bug with Lambda instrumentation where the APM agent would result in an otherwise working Lambda function to respond with null if the Lambda was missing the Elastic APM Lambda Extension. (#2598)
  • Fix a bug in Lambda instrumentation in the capturing of SNS and SQS event message attributes. (#2605)

3.30.0 2022/03/10edit

Breaking changesedit
  • Added a new config option spanStackTraceMinDuration that replaces both captureSpanStackTraces and spanFramesMinDuration. The latter two are now deprecated, but still supported. If spanStackTraceMinDuration is specified, then any value for the deprecated two options will be ignored.

    There is a significant change in default behavior of the APM agent. If none of these configuration options is specified, then the default (spanStackTraceMinDuration: -1) is that stack traces are not collected and reported for any spans. This change in default behavior was made because the CPU performance impact of collecting span stack traces was found to be too high in practice for busy and/or complex applications. This is mentioned in the "Breaking changes" section to highlight the change, but it is not considered breaking in general. The impact is that the "Stack Trace" tab in the "Span details" view in the Kibana APM app will be empty. This was already the case for some spans based on span duration. (#2565)

  • Implement the explicit signaling of Lambda invocation completion to the Elastic AWS Lambda Extension. This improves data flushing in a Lambda environment to ensure tracing data is only sent when the Lambda is active. This avoids possible tracing data loss while a Lambda VM is frozen. (#2485)

    However, because this change triggers a bug in the extension, this version of the APM Node.js Agent must only be used with versions of the AWS Lambda Extension after v0.0.3.

Featuresedit
  • Add faas.name and faas.version fields to Lambda transactions. (#2587)
  • Added automatic wrapping of AWS Lambda handlers (#2577)
  • Improvements to AWS Lambda instrumentation: Better transaction.name for API Gateway-triggered lambdas. Respect explicitly set serviceName, serviceVersion, and usePathAsTransactionName config settings. Default cloudProvider: none and centralConfig: false to reduce required environment variables for setting up APM instrumentation of Lambdas. (#2531)

3.29.0 2022/02/10edit

  • Fix a bug in instrumentation of @elastic/elasticsearch that caused a memory leak. (#2569)

3.28.0 2022/02/08edit

Known issue: This release includes a memory leak in instrumentation of the @elastic/elasticsearch package. If you use that package, you should not use v3.28.0 of this APM agent. (#2569)

Breaking changesedit

The following changes are not considered breaking. However, they result in a change in behavior and trace output that might impact some users, so they are highlighted here.

  • Change the redis and mysql instrumentations to not patch at all if they are listed in disableInstrumentations. This means that an application that uses one of these packages and lists that package in disableInstrumentations could see changes in the async run-context of callbacks. See #2498 and the v3.26.0 release notes which has a similar change.
  • Elasticsearch spans (from elasticsearch, @elastic/elasticsearch, and @elastic/elasticsearch-canary instrumentation) will no longer have an HTTP child span(s) for the underlying HTTP request. This is listed in this section to provide awareness in case some users have custom analysis of APM trace data that expects those HTTP spans.

    Per the APM Agent spec for exit spans, Elasticsearch spans are now marked as exit spans and as a result, HTTP child spans are suppressed. (#2000)

    As part of this change, some HTTP context has been added to Elasticsearch spans, when available: the HTTP response status_code, and the size of the response body (encoded_body_size). (#2484)

Featuresedit
  • Drop unsampled transactions when sending to APM Server v8.0+. (#2455)
  • The default serviceName string (when it is not configured and cannot be inferred from a "package.json" file) has been changed from "nodejs_service" to "unknown-nodejs-service". This is a standardized pattern used across Elastic APM agents to allow the Kibana APM app to recognize when to provide help to the user on configuring the service name. (#2491)
  • Add transaction.name to captured APM errors. This will allow the Kibana APM app to correlate error groups and transaction groups. (#2456)
  • Mark S3 spans (from aws-sdk instrumentation) as exit spans (per https://github.com/elastic/apm/blob/main/specs/agents/tracing-spans.md#exit-spans). The result is that HTTP child spans of S3 spans are no longer captured. (#2125)
Bug fixesedit
  • Fixes for run context handling for @elastic/elasticsearch instrumentation. (#2430)
  • Fixes for run context handling for cassandra-driver instrumentation. (#2430)
  • Fixes for run context handling for mongodb-core instrumentation. (#2430)
  • Fixes for run context handling for elasticsearch instrumentation. (#2430)

3.27.0 2022/01/17edit

Featuresedit
  • Add support for express-graphql 0.10.0 - 0.12.0 inclusive. (#2511)
Bug fixesedit
  • Fix an issue where the agent’s async task tracking could cause the user’s application to use too much memory. In cases where an application is under sustained load and is running in a memory constrained container, this looked like a memory leak.

    This high memory usage could happen when application code starts async tasks (e.g. Promises, setTimeouts, async I/O) that outlive the APM Transaction (typically an HTTP request handler). The agent’s async task tracking keeps a reference to the APM Transaction (and any APM Spans) until the async task ends, thus extending the lifetime of those APM objects and the references they hold — in particular, HTTP request and response objects. This could lead to higher memory usage.

    With this change, those references are removed when APM Transactions and Spans are ended, and agent memory usage is now the same as what it was before v3.24.0 when this issue was introduced. (#2528, #2489)

  • Fixes for run context handling for graphql instrumentation. (#2430)
  • Fixes for run context handling for DynamoDB instrumentation (aws-sdk package) so that a span created after an AWS client command (in the same tick, in the command callback, or promise) is not a child of the automatic AWS span. This change also ensures captured errors from failing client commands are a child of the AWS span. (#2430)
  • Fixes for run context handling for pg instrumentation. (#2430)
  • Fixes for run context handling for mongodb instrumentation. (#2512)

3.26.0 2021/12/07edit

Breaking changesedit
  • In earlier versions, the agent would propagate run context in some packages even if instrumentation for that package was disabled via disableInstrumentations. Recent versions change the semantics of disableInstrumentations to mean the agent should not touch the listed packages at all. This means that an application that uses one of these packages and lists that package in disableInstrumentations could see changes in the async run-context of callbacks. This affects: pg (v3.24.0), redis (v3.25.0), mysql (v3.25.0), ioredis (v3.26.0), mysql2 (v3.26.0). See #2498 for details.
Featuresedit
  • Add *auth* pattern to default value for sanitizeFieldNames config var, so that it is more likely to redact authentication/authorization-related HTTP headers and form fields. This pattern replaces the authorization pattern in the set of defaults. (#2427)
Bug fixesedit
  • Fix run-context handling for tedious instrumentation so that automatically created mssql spans are never the currentSpan in user code. (#2430)
  • Fix http2 instrumentation for outgoing requests to not have the created HTTP span context be active in user code. (#2430)
  • Fix run-context handling in ws instrumentation so that the span created for a ws.send(...) isn’t the "current span" in subsequent code in the same tick. (#2481)
  • Fix run-context handling for memcached instrumentation so that the automatically created Memcached span is never the currentSpan in user code. (#2430)
  • Fix a possible crash when serializing a Transaction if the incoming req.socket is null (possible if the socket has been destroyed). (#2479)
  • Fixes for run context handling for aws-sdk instrumentation (S3, SQS, SNS) so that a span created after an AWS client command (in the same tick, in the command callback, or promise) is not a child of the automatic AWS span. This change also ensures captured errors from failing client commands are a child of the AWS span. (#2430)
  • Fix http and https instrumentation for outgoing requests to not have the http span context be active in user code. (#2470)
  • Fixes for ioredis instrumentation (#2460):

    • Fix run-context so that a span created in the same tick as an ioredis client command will no longer be a child of the redis span.
    • Capture an APM error and correctly set span.outcome to "failure" when a redis client command calls back with an error.
    • Avoid a rare possible double-instrumentation of redis commands internally-queued before the RedisClient is "ready". (#2459)
    • Add destination context so Redis shows up on the Service Map.
  • Fix run-context handling for mysql2 instrumentation to avoid accidental creation of child spans of the automatic mysql spans. (#2430})

3.25.0 2021/11/24edit

Bug fixesedit
  • Fixes for redis instrumentation:

    • Fix run-context so that a span created in the same tick as a redis client command will no longer be a child of the redis span. (#2430)
    • Capture an APM error and correctly set span.outcome to "failure" when a redis client command calls back with an error.
    • Avoid a rare possible double-instrumentation of redis commands internally-queued before the RedisClient is "ready". (#2446)
  • Avoid setting the tracestate header for outgoing HTTP requests to the empty string. This can happen for non-trace-root transactions. While the HTTP spec allows empty header values, some servers do not. (#2405)
  • Deprecate transaction.subtype and transaction.action. These fields were never used by APM server. This also deprecates the apm.startTransaction(...) call signatures that take subtype and action arguments. In the next major version these two fields will be removed. (#2356)
  • Fix run-context handling for mysql instrumentation to avoid accidental creation of child spans of the automatic mysql spans. (#2430})

3.24.0 2021/11/09edit

Breaking changesedit
  • Change disableSend to no longer skip internal processing work. It now only disables communication with APM Server. Use contextPropagationOnly if your use case is to limit the APM agent’s processing to the minimum to support context propagation and log correlation.

    This is listed under "Breaking changes" as a heads-up. The only possible negative result of this disableSend change is some extra CPU processing time by the agent. There is no outward functionality change.

Featuresedit
  • Gather AWS Lambda-specific metadata on first function invocation and ensure no intake requests to APM Server are started before that metadata is available. (#2404)
  • Add contextPropagationOnly configuration option. This supports the use case of using the APM agent to propagate HTTP trace-context and to support log-correlation (adding trace.id et al fields to log records) without an APM server, and to otherwise reduce the processing time of the agent. (#2393)

    This is similar to disableSend, but differs in that contextPropagationOnly: true tells the agent to skip unnecessary work.

  • The User-Agent header used for communication with APM Server now includes the serviceName and serviceVersion. For some users this can be helpful for APM Server log analysis. (#2364)
  • In a Lambda enviornment we now collect a number of additional data fields on the Transaction object. See the spec for more information on fields collected. https://github.com/elastic/apm/blob/main/specs/agents/tracing-instrumentation-aws-lambda.md (#2156)
  • Zero configuration support. The only required agent configuration option is serviceName. Normally the agent will attempt to infer serviceName for the "name" field in a package.json file. However, that could fail. With this version, the agent will cope with: a scoped package name (@scope/name is normalized to scope-name), a "name" that isn’t a valid serviceName, not being able to find a "package.json" file, etc. Ultimately it will fallback to "nodejs_service". (#1944)

    One consequence of this change is that apm.getServiceName() will return undefined until the agent is started (check with apm.isStarted()).

Bug fixesedit
  • Stop collecting transaction breakdown metrics (transaction.breakdown.count, transaction.duration.sum.us, transaction.duration.count), as they are not being used in APM UI. (#2370)
  • Wrap fs.realpath.native when configured with asyncHooks=false. This fixes using that function (which was undefined before this fix) and a crash when importing fs-extra@10. (#2401)
  • A significant change was made to internal run context tracking (a.k.a. async context tracking). There are no configuration changes or API changes for custom instrumentation. (#2181)

    One behavior change is that multiple spans created synchronously (in the same async task) will form parent/child relationships; before this change they would all be siblings. This fixes HTTP child spans of Elasticsearch and aws-sdk automatic spans to properly be children. (#1889)

    Another behavior change is that a span B started after having ended span A in the same async task will no longer be a child of span A. (#1964)

    This fixes an issue with context binding of EventEmitters, where removeListener would fail to actually remove if the same handler function was added to multiple events.

  • Fix pino’s deprecation warning when using a custom logger with pino@6 (#2332)

3.23.0 2021/10/25edit

Breaking changesedit
  • Remove support for instrumenting versions of fastify earlier than 1.0.0. This instrumentation might still work, but is no longer supported. Fastify v1.0.0 was released in 2018. All current users should be using fastify v2 or v3 at least. See https://www.fastify.io/docs/latest/LTS/ (#2387)
Featuresedit
  • Add initial support for version 8 of @elastic/elasticsearch, which is still in pre-release. (#2385)

3.22.0 2021/10/21edit

Featuresedit
  • Add support for node v17.
  • When an error is captured, the APM agent will only immediately flush it to APM server if it is an "unhandled" error. Unhandled errors are typically those captured via the uncaughtException process event. Before this change, a captured error (e.g. for a 4xx or 5xx response from an HTTP server) was always immediately flushed. This could negatively impact performance for a service that was generating frequent errors. (#686)
Bug fixesedit
  • Improve apm.flush([cb]) to wait for inflight spans and errors before flushing data to APM server. Before this change, a recently ended span or recently captured error might not yet have completed processing (for example, stacktrace collection is asynchronous) and might not be included in the flush call. (#2294)
  • AWS Lambda changes: Disable metrics collection during the experimental phase of (re)implementing Lambda support (#2363). Some fixes for better flushing of data at the end of a Lambda invocation.
  • span.toString() and transaction.toString() have been deprecated. The exact string output may change in v4 of the agent.
  • Add Span.ids and Transaction.ids to TypeScript types. (#2347)
  • Improve span.sync determination (fixes #1996) and stop reporting transaction.sync which was never used (#2292). A minor semantic change is that span.sync is not set to a final value until span.end() is called. Before span.end() the value will always by true.
  • Guard against a negative value of metricsInterval, which can lead to high CPU usage as metrics are collected as fast as possible. Also ensure no metrics collection can happen if metricsInterval="0s" as intended. Before this change it was possible for some metric collection to still happen, even though none would be reported. (#2330)

    This change also guards against negative and invalid values in the following configuration options: abortedErrorThreshold, apiRequestTime, and serverTimeout. If an invalid value is given, then will fallback to their default value.

3.21.1 2021/09/16edit

Bug fixesedit
  • Update types to avoid imports of @types/... modules (other than @types/node), so that TypeScript users of elastic-apm-node need not manually npm install @types/connect @types/pino @types/aws-lambda to compile. (#2331)

3.21.0 2021/09/15edit

Featuresedit
  • Add the longFieldMaxLength integer configuration option (default 10000). Specific transaction/span/error fields (see the list below) will be truncated at this number of unicode characters. (#2193, #1921)

    The errorMessageMaxLength configuration option is now deprecated, but still supported. Users should switch to using longFieldMaxLength. If errorMessageMaxLength is not specified, truncation of error messages will now use the longFieldMaxLength value.

    Note that ultimately the maximum length of any tracing field is limited by the max_event_size configured for the receiving APM server.

    The fields affected by longFieldMaxLength are:

    • transaction.context.request.body, error.context.request.body - Before this change these fields were not truncated.
    • transaction.context.message.body, span.context.message.body, error.context.message.body - Before this change these fields were not truncated.
    • span.context.db.statement - Before this change this field was truncated at 10000 bytes. Truncation is now a number of unicode characters.
    • error.exception.message, error.log.message - Before this change, the default 2kB errorMessageMaxLength would apply.
  • Improve the TypeScript types by exporting more of interfaces: AgentConfigOptions, Transaction, Span, TransactionOptions, SpanOptions. (#2118)
Bug fixesedit
  • Fix a bug in apm.removePatch(module, aHandler) that would remove the last registered handler if aHandler did not match any currently registered handlers. (#2315)
  • Fix a crash in instrumentation of the old Elasticsearch client (elasticsearch) for some rarer cases of client options — for example passing multiple hosts. (#2312)
  • Ensure the internal HTTP(S) client requests made by the APM agent to APM server are not themselves traced. (#1168, #1136)
  • Fix crashing error with agent.registerMetric and active:false configuration. (#1799, #2290)

3.20.0 2021/08/12edit

Bug fixesedit
  • Fix failing tests and a possible runtime crash in @elastic/elasticsearch@7.14.0 instrumentation. (#2187)

3.19.0 2021/08/05edit

Featuresedit
  • The agent now supports the 3.x branch of apollo-server-express. (#2155)
  • Add instrumentation support for mongodb@4.x. (#2171)
Bug fixesedit
  • The agent will no longer report counting metrics with a value of zero, and will remove these metrics from the registry. (#2163)

3.18.0 2021/07/20edit

Featuresedit
  • Trace an incoming HTTP/1.1 request to an HTTP/2 server using the allowHTTP1 option. Before this change only incoming requests supporting HTTP/2 would be traced. (#2143)
  • Add instrumentation of the AWS SNS publish method when using the JavaScript AWS SDK v2 (aws-sdk). (#2157)
Bug fixesedit

3.17.0 2021/07/05edit

Featuresedit
  • Add instrumentation of all AWS S3 methods when using the JavaScript AWS SDK v2 (aws-sdk).
  • Add disableSend configuration option. This supports some use cases using the APM agent without an APM server. (#2101)
  • Add instrumentation of all DynamoDB methods when using the JavaScript AWS SDK v2 (aws-sdk).
Bug fixesedit
  • Fix inconsistencies in HTTP spans from other APM agents. span.subtype will now be "http" for HTTP, HTTPS, and HTTP/2 outgoing spans — previously it was "http", "https", "http2", respectively. As well, span.action will now be the HTTP method (e.g. "GET", "PUT", "POST"), rather than "http". (#2075)
  • Fixed error where SQS messages sent without an active transactions could crash the agent. (#2113)
  • Fixed support for proxies in destination context (#1770)

3.16.0 - 2021/06/14edit

Featuresedit
  • Added spanFramesMinDuration configuration field, allowing users to set a time threshold value that spans must reach before the agent will add a stack trace to the span. As a result, many short spans that previously included stack traces by default no longer will.
  • Prefer W3C "traceparent" header over "elastic-apm-traceparent" for incoming requests. #2079
Bug fixesedit
  • Fix a crash (TypeError: lastPrepareStackTrace) in the agent when used with React v17 and later (#1980).
  • Performance improvements have been made in error and stacktrace capture (#2094). This also included in two bug fixes:

    • Before this change, some captured errors (for example a next(new Error('boom') from an Express handler) would mark the error as "unhandled" incorrectly. "Unhandled" exceptions are those caught by an uncaughtException handler.
    • Before this change, source context lines for a stacktrace would not properly use the "sourcesContext" field of a file’s source map.

3.15.0 - 2021/05/19edit

Featuresedit
  • Add support for Node.js v16. (This also drops testing of Node.js v13 releases.) #2055
Bug fixesedit
  • Update TypeScript typings for Agent.setLabel and Agent.addLabels to include the stringify argument that was added in v3.11.0.

3.14.0 - 2021/04/19edit

Featuresedit
  • Add apm.addMetadataFilter(fn) for filtering the metadata object sent to APM server.
  • The handling of sending events (transactions, spans, errors) to APM server has improved in a few ways. During temporary spikes in load and/or an APM server that is unresponsive, the agent will buffer a number of events and drop them above a certain limit (configurable via maxQueueSize). This helps ensure the agent does not overly consume memory and CPU. As well, the agent will now backoff when the APM server errors. Finally, improved error handling means it will terminate failing requests to the APM server more quickly.

    Note: v1 of this agent (EOL’d 2 years ago), included a maxQueueSize config variable with a different meaning. If you have a lingering usage of that setting (also possibly as the ELASTIC_APM_MAX_QUEUE_SIZE environment variable), then it should be removed.

  • Adds support for Amazon SQS queues via aws-sdk instrumentation that partially implements the APM messaging spec, and adds queue.latency.min.ms, queue.latency.max.ms, and queue.latency.avg.ms metrics for SQS queues.
Bug fixesedit
  • Fixed bug where the URL property for outgoing HTTP request spans was set with the server’s IP address rather than its hostname. The Agent now sets this property with the actual URL requested by Node.js. #2035
  • Fixed bug where external services were not listed under Dependencies on the APM Service Overview page due to the trace-context propagated sample_rate value not being set on either transactions or spans.

3.13.0 - 2021/04/06edit

Featuresedit
  • The APM agent’s own internal logging now uses structured JSON logging using the pino API, and formatted in ecs-logging format. The log records on stdout are now single-line JSON objects. For example:

    # Before
    APM Server transport error (ECONNREFUSED): connect ECONNREFUSED 127.0.0.1:8200
    
    # After
    {"log.level":"error","@timestamp":"2021-03-19T00:21:17.571Z","log":{"logger":"elastic-apm-node"},
    "ecs":{"version":"1.6.0"},"message":"APM Server transport error (ECONNREFUSED): connect ECONNREFUSED 127.0.0.1:8200"}

    Pretty formatting (and filtering) on the console may be done via the ecslog tool.

    A custom logger is still supported as before. However, a non-pino custom logger will only receive the "message" field, and not structured log fields as they are added over time.

  • Add support for setting the ELASTIC_APM_LOGGER=false environment variable to disable/ignore a given custom logger. This is to support easier Debug mode for troubleshooting.
Bug fixesedit
  • Lock package dependency "elastic-apm-http-client@9.6.0" to avoid using v9.7.0 for now, because it is breaking tests. A coming release will get back on the latest of this dependency. #2032
  • Remove the "ancestors" field from a log.trace message on startup. Its info is a duplicate of info in the "startTrace" field in the same log record. #2005
  • Remove the accidental nodejs.eventloop.delay.ns metric that was always reporting a zero value. The existing nodejs.eventloop.delay.avg.ms is the intended metric. #1993

3.12.1 - 2021/02/25edit

Bug fixesedit

3.12.0 - 2021/02/21edit

Featuresedit
  • feat: Set span outcome to success or failure depending on whether an error was captured during when the span was active. #1814
  • feat: Adds public setOutcome method to span and transaction APIs, and adds a top level setTransactionOutcome and setSpanOutcome to set outcome values for the current active transaction or active span.
  • Limit the transactionSampleRate value to 4 decimal places of precision according to the shared APM spec. This ensures that propagated sampling rate in the tracestate header is short and consistent. #1979
Bug fixesedit
  • fix: It was possible for fetching central config to result in the cloudProvider config value being reset to its default. #1976
  • fix: fixes bug where tedious could crash the agent on bulk inserts #1935
    Reported via the forum. The error symptom was: Cannot read property 'statement' of undefined

3.11.0 - 2021/02/08edit

Featuresedit
  • feat: add apm.getServiceName() #1949
    This will be used by ecs-logging packages to integrate with APM.
  • feat: support numeric and boolean labels #1909
    Add an optional stringify option to apm.setLabel(name, version, stringify = true) and apm.addLabels(labels, stringify = true), which can be set false to allow numeric and boolean labels. Stringify defaults to true for backwards compatibility — stringification will be removed in a future major version.
  • feat: added support for cloud metadata fetching #1937
    Agent now collects information about its cloud environment and includes this data in the APM Server’s metadata payload. See the spec for more information.

3.10.0 - 2021/01/11edit

Featuresedit
  • feat: Improve handling of raw body parsing The agent will now report raw/Buffer encoded post bodies as <Buffer>.
  • feat: Add support for api keys #1818
    This allows the usage of API keys for authentication to the APM server
  • feat: Add automatic instrumentation of the @elastic/elasticsearch package #1870

    The instrumentation of the legacy "elasticsearch" package has also changed slightly to commonalize:

    • "span.context.destination" is set on all Elasticsearch spans, not just a subset of query-like API endpoints.
    • For query-like API endpoints (e.g. /_search), the capturing of query details on "span.context.db.statement" has changed (a) to include both the query params and the request body if both exist (separated by \n\n) and (b) to URL encode the query params, rather than JSON encoding.
  • feat: Add captureAttributes boolean option to apm.captureError() to allow disabling the automatic capture of Error object properties. This is useful for cases where those properties should not be sent to the APM Server, e.g. for performance (large string fields) or security (PII data). #1912
  • feat: Add log_level central config support. #1908
    Spec: https://github.com/elastic/apm/blob/main/specs/agents/logging.md
  • feat: Add sanitize_field_names configuration option.
    Allows users to configure a list of wildcard patterns to remove items from the agent’s HTTP header and application/x-www-form-urlencoded payloads. #1898

Bug fixesedit
  • fix: Fix parsing of comma-separated strings for relevant config vars to allow whitespace around the commas. E.g.:

    export ELASTIC_APM_TRANSACTION_IGNORE_URLS='/ping, /metrics*'

    Config vars affected are: disableInstrumentations, transactionIgnoreUrls addPatch, and globalLabels.

  • fix: Correct the environment variable for setting transactionIgnoreUrl (added in v3.9.0) from ELASTIC_TRANSACTION_IGNORE_URLS to ELASTIC_APM_TRANSACTION_IGNORE_URLS.

3.9.0 - 2020/11/30edit

Featuresedit
  • feat: support fastify 3 #1891
    Adds .default and .fastify module.exports to instrumented fastify function for 3.x line, and prefers req.routerMethod and req.routerPath for transaction name
  • feat: Set "destination" context on spans for "mongodb". #1893
    This allows Kibana APM Service Maps to show a "mongodb" node for services using the mongodb package (which includes mongoose and mongojs).
  • feat: transactionIgnoreUrl wildcard matching #1870
    Allows users to ignore URLs using simple wildcard matching patterns that behave the same across language agents. See https://github.com/elastic/apm/issues/144
Bug fixesedit
  • fix: treat set-cookie in response headers as sensitive data #1886
  • fix: Synchronous spans would never have span.sync == true. #1879

3.8.0 - 2020/11/09edit

Featuresedit
  • feat: expand k8s pod ID discovery regex #1863
  • feat: implements tracestate #1828
    Expands support for the W3C Trace Context specification by adding a tracestate header implementation, and uses this new header to track the Elastic transaction sample rate across a trace’s service boundaries.
  • feat: add span and transaction outcome #1824
    This adds an "outcome" field to HTTP(S) transactions and spans.
Bug fixesedit
  • fix(pg): prevent unhandled promise rejection #1846
  • fix: redis@2.x instrumentation was broken #1852
  • A number of fixes to the test suite.

3.7.0 - 2020/8/10edit

  • feat(knex): add support for 0.21.x #1801
  • feat(redis): add support for v3.x #1641
  • feat(graphql): add support for 15.x #1795
  • feat(koa-router): add support for 9.x #1772
  • fix(elasticsearch): ensure requests can be aborted #1566
  • fix: end span if outgoing http request ends prematurely #1583
  • fix: don’t throw on invalid URL #1771
  • fix: patch apollo-server-core > 2.14 correctly #1796
  • fix: add currentTraceIds to typings #1733

3.6.1 - 2020/5/20edit

  • fix(package): bump elastic-apm-http-client to ^9.4.0 #1756

3.6.0 - 2020/5/18edit

  • feat: add destination metadata for db spans #1687
  • feat: add support for Node.js 14 #1742
  • feat(pg): add support for pg v8.x #1743
  • feat: add metrics for external memory #1724
  • feat: enrich spans with destination info #1685
  • fix(instrumentation): add .js to module path #1711

3.5.0 - 2020/3/9edit

  • feat(error): get stack trace from Error-like objects #1613
  • fix: add logUncaughtExceptions conf option to TypeScript typings #1668

3.4.0 - 2020/2/21edit

  • feat: support W3C TraceContext traceparent header #1587
  • feat: add custom metrics API (experimental) #1571
  • feat(koa-router): add support for v8.x #1642
  • fix(cassandra): improve support for cassandra-driver v4.4.0+ #1636
  • fix: support promisifying setTimeout and friends #1636

3.3.0 - 2019/12/13edit

  • feat(config): add serverCaCertFile config #1560
  • feat(config): add central config support for transactionMaxSpans and captureBody #1555

3.2.0 - 2019/11/19edit

  • fix(metrics): only register collectors if enabled #1520
  • fix(ioredis): prevent unhandled promise rejection #1523
  • chore: add Node 13 to supported engines #1524

3.1.0 - 2019/10/16edit

Featuresedit
  • feat(mongodb): instrumentation #1423
  • fix(package): update elastic-apm-http-client to version 9.0.0 #1419
  • perf: cache ids value of transactions and spans #1434
Bug fixesedit
  • fix: always end transaction when socket is closed prematurely #1439
  • fix: change logUncaughtExceptions default to false #1432
  • fix: write stack trace of uncaught exceptions to STDERR #1429

3.0.0 - 2019/9/30edit

Breaking changesedit
  • feat: allow manual instrumentation with instrument: false #1114
  • feat: allow setting span/transaction type, subtype, and action separately (the behavior of the old type has changed) #1292
  • feat: use external as span type instead of ext #1291
  • refactor(graphql): use custom transaction type graphql for graphql requests instead of request #1245
  • feat(http): add instrumentIncomingHTTPRequests config (disableInstrumentations now behaves differently) #1298
  • chore: remove deprecated APIs #1413
  • chore: drop support for older Node.js versions #1383