Amazon Kinesis Data Firehose troubleshootingedit

Monitoringedit

You can use the monitoring tab in the Firehose console to ensure there are incoming records and the delivery success rate is 100%.

By default Firehose also logs to a Cloudwatch log group with the name /aws/kinesisfirehose/<delivery stream name>, which is automatically created when the delivery stream is created. Two log streams, DestinationDelivery and BackupDelivery, are created in this log group.

The backup settings in the delivery stream specify how failed delivery requests are handled. See Backup settings for details on configuring backups to S3.

Scalingedit

Firehose can automatically scale to handle very high throughput. If your Elastic deployment is not properly configured for the data volume coming from Firehose, it could cause a bottleneck, which may lead to increased ingest times or indexing failures.

There are several facets to optimizing the underlying Elasticsearch performance, but Elastic Cloud provides several ready-to-use hardware profiles which can provide a good starting point. Other factors which can impact performance are shard sizing, indexing configuration, and index lifecycle management (ILM).

Supportedit

If you encounter further problems, please contact Elastic support by following the instructions here.