IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
Elastic Docs ›Observability Guide [8.2] ›Send data to Elasticsearch ›Elastic Serverless Forwarder for AWS ›Configuration options for Elastic Serverless Forwarder
« Use AWS Secrets Manager Use tags and filters »

Route AWS service logsedit

For S3 SQS Event Notifications inputs, the Elastic Serverless Forwarder supports automatic routing of several AWS service logs to the corresponding integration data streams for further processing and storage in the Elasticsearch cluster.

Automatic routingedit

Elastic Serverless Forwarder supports automatic routing of the following logs to the corresponding default integration data stream:

  • AWS CloudTrail (aws.cloudtrail)
  • Amazon CloudWatch (aws.cloudwatch_logs)
  • Elastic Load Balancing (aws.elb_logs)
  • AWS Network Firewall (aws.firewall_logs)
  • Amazon VPC Flow (aws.vpcflow)
  • AWS Web Application Firewall (aws.waf)

For these use cases, setting the es_datastream_name field in the configuration file is optional.

For most other use cases, you will need to set the es_datastream_name field in the configuration file to route the data to a specific data stream or index. This value should be set in the following use cases:

  • You want to write the data to a specific index, alias, or custom data stream, and not to the default integration data stream. This can help some users to use existing Elasticsearch assets like index templates, ingest pipelines, or dashboards, that are already set up and connected to business processes.
  • When using Kinesis Data Stream, CloudWatch Logs subscription filter or Direct SQS message payload inputs. Only the S3 SQS Event Notifications input method supports automatic routing to default integration data streams for several AWS service logs.
  • When using S3 SQS Event Notifications but where the log type is something other than AWS CloudTrail (aws.cloudtrail), Amazon CloudWatch Logs (aws.cloudwatch_logs), Elastic Load Balancing (aws.elb_logs), AWS Network Firewall (aws.firewall_logs), Amazon VPC Flow (aws.vpcflow), and AWS Web Application Firewall (aws.waf).

If the es_datastream_name is not specified, and the log cannot be matched with any of the above AWS services, then the dataset will be set to generic and the namespace set to default, pointing to the data stream name logs-generic-default.

« Use AWS Secrets Manager Use tags and filters »