Amazon Kinesis Data Firehose is a fully managed service for delivering real-time streaming data to destinations such as Amazon Simple Storage Service (Amazon S3), Amazon Redshift, Amazon Elasticsearch Service (Amazon ES), and Splunk. Kinesis Data Firehose is part of the Kinesis streaming data platform, including Kinesis Data Streams, Kinesis Video Streams, and Amazon Kinesis Data Analytics. With Kinesis Data Firehose, you do not need to write applications or manage resources. You configure your data producers to send data to Kinesis Data Firehose, and it automatically delivers the data to the destination that you specified. You can also configure Kinesis Data Firehose to transform your data before delivering it.

Use the AWS public cloud integration to discover and collect metrics against the AWS service.

External reference

What Is Amazon Kinesis Data Firehose?

Setup

To set up the AWS integration and discover the AWS service, go to AWS Integration Discovery Profile and select Firehose.

Event support

CloudTrail event support

  • Supported (CreateDeliveryStream, DeleteDeliveryStream)
  • Configurable in OpsRamp AWS Integration Discovery Profile.

CloudWatch alarm support

  • Supported
  • Configurable in OpsRamp AWS Integration Discovery Profile.

Supported metrics

OpsRamp MetricAWS MetricMetric Display NameUnitAggregation Type
aws_firehose_PutRecord_Bytes

Number of bytes put to the Amazon Kinesis stream using PutRecord over the specified time period.
PutRecord.BytesPutRecord.BytesBytesSum
aws_firehose_PutRecord_Latency

Time taken per PutRecord operation, measured over the specified time period.
PutRecord.LatencyPutRecord.LatencymsAverage
aws_firehose_IncomingBytes

Number of bytes successfully put to the Amazon Kinesis stream over the specified time period. This metric includes bytes from PutRecord and PutRecords.
IncomingBytesIncomingBytesBytesSum
aws_firehose_IncomingRecords

Number of records successfully put to the Amazon Kinesis stream over the specified time period. This metric includes record counts from PutRecord and PutRecords.
IncomingRecordsIncomingRecordsCountSum
aws_firehose_DeliveryToElasticsearch_Bytes

Number of bytes indexed to Amazon ES over the specified time period.
DeliveryToElasticsearch.BytesDeliveryToElasticsearch.Bytes.firehoseBytesSum
aws_firehose_DeliveryToElasticsearch_Records

Number of records indexed to Amazon ES over the specified time period.
DeliveryToElasticsearch.RecordsDeliveryToElasticsearch.Records.firehoseCountSum
aws_firehose_DeliveryToElasticsearch_Success

Sum of the successfully indexed records over the sum of records that were attempted.
DeliveryToElasticsearch.SuccessDeliveryToElasticsearch.Success.firehoseCountSum
aws_firehose_DeliveryToRedshift_Bytes

Number of bytes copied to Amazon Redshift over the specified time period.
DeliveryToRedshift.BytesDeliveryToRedshift.Bytes.firehoseBytesSum
aws_firehose_DeliveryToRedshift_Records

Number of records copied to Amazon Redshift over the specified time period.
DeliveryToRedshift.RecordsDeliveryToRedshift.Records.firehoseCountSum
aws_firehose_DeliveryToRedshift_Success

Sum of successful Amazon Redshift COPY commands over the sum of all Amazon Redshift COPY commands.
DeliveryToRedshift.SuccessDeliveryToRedshift.Success.firehoseCountSum
aws_firehose_DeliveryToS3_DataFreshness

Age (from getting into Kinesis Firehose to now) of the oldest record in Kinesis Firehose. Any record older than this age is delivered to the S3 bucket.
DeliveryToS3.DataFreshnessDeliveryToS3.DataFreshness.firehoseSecondsAverage
aws_firehose_DeliveryToS3_Records

Number of records delivered to Amazon S3 over the specified time period.
DeliveryToS3.RecordsDeliveryToS3.Records.firehoseCountSum
aws_firehose_DeliveryToS3_Success

Sum of successful Amazon S3 put commands over the sum of all Amazon S3 put commands.
DeliveryToS3.SuccessDeliveryToS3.Success.firehoseCountSum
aws_firehose_DescribeDeliveryStream_Latency

Time taken per DescribeDeliveryStream operation, measured over the specified time period.
DescribeDeliveryStream.LatencyDescribeDeliveryStream.Latency.firehoseMillisecondsAverage
aws_firehose_DescribeDeliveryStream_Requests

Total number of DescribeDeliveryStream requests.
DescribeDeliveryStream.RequestsDescribeDeliveryStream.Requests.firehoseCountAverage
aws_firehose_ListDeliveryStreams_Latency

Time taken per ListDeliveryStream operation, measured over the specified time period.
ListDeliveryStreams.LatencyListDeliveryStreams.Latency.firehoseMillisecondsAverage
aws_firehose_ListDeliveryStreams_Requests

Total number of ListFirehose requests.
ListDeliveryStreams.RequestsListDeliveryStreams.Requests.firehoseCountAverage
aws_firehose_PutRecord_Requests

Total number of PutRecord requests, which is equal to total number of records from PutRecord operations.
PutRecord.RequestsPutRecord.Requests.firehoseCountSum
aws_firehose_PutRecordBatch_Bytes

Number of bytes put to the Kinesis Firehose delivery stream using PutRecordBatch over the specified time period.
PutRecordBatch.BytesPutRecordBatch.Bytes.firehoseBytesSum
aws_firehose_PutRecordBatch_Latency

Time taken per PutRecordBatch operation, measured over the specified time period.
PutRecordBatch.LatencyPutRecordBatch.Latency.firehoseMillisecondsAverage
aws_firehose_PutRecordBatch_Records

Total number of records from PutRecordBatch operations.
PutRecordBatch.RecordsPutRecordBatch.Records.firehoseCountSum
aws_firehose_PutRecordBatch_Requests

Total number of PutRecordBatch requests.
PutRecordBatch.RequestsPutRecordBatch.Requests.firehoseCountSum
aws_firehose_UpdateDeliveryStream_Requests

Total number of UpdateDeliveryStream requests.
UpdateDeliveryStream.RequestsUpdateDeliveryStream.Requests.firehoseCountSum
aws_firehose_UpdateDeliveryStream_Latency

Time taken per UpdateDeliveryStream operation, measured over the specified time period.
UpdateDeliveryStream.LatencyUpdateDeliveryStream.Latency.firehoseMillisecondsAverage
aws_firehose_BackupToS3_Bytes

Number of bytes delivered to Amazon S3 for backup over the specified time period.
BackupToS3.BytesBackupToS3 BytesBytesSum
aws_firehose_BackupToS3_DataFreshness

Age (from getting into Kinesis Data Firehose to now) of the oldest record in Kinesis Data Firehose. Any record older than this age is delivered to the Amazon S3 bucket for backup. Kinesis Data Firehose emits this metric when data transformation is enabled for Amazon S3 or Amazon Redshift destinations.
BackupToS3.DataFreshnessBackupToS3 DataFreshnessSecondsAverage
aws_firehose_BackupToS3_Records

Number of records delivered to Amazon S3 for backup over the specified time period.
BackupToS3.RecordsBackupToS3 RecordsCountSum
aws_firehose_BytesPerSecondLimit

The current maximum number of bytes per second that a delivery stream can ingest before throttling.
BytesPerSecondLimitBytes Per Second LimitBytesAverage
aws_firehose_DataReadFromKinesisStream_Bytes

When the data source is a Kinesis data stream, this metric indicates the number of bytes read from that data stream.
DataReadFromKinesisStream.BytesData Read From Kinesis Stream BytesBytesSum
aws_firehose_DataReadFromKinesisStream_Records

When the data source is a Kinesis data stream, this metric indicates the number of records read from that data stream.
DataReadFromKinesisStream.RecordsData Read From Kinesis Stream RecordsCountSum
aws_firehose_DeliveryToElasticsearch_DataFreshness

Age (from getting into Kinesis Data Firehose to now) of the oldest record in Kinesis Data Firehose.
DeliveryToElasticsearch.DataFreshnessDelivery To Elasticsearch Data FreshnessSecondsAverage
aws_firehose_DeliveryToSplunk.Records

Number of records delivered to Splunk over the specified time period.
DeliveryToSplunk.RecordsDelivery To Splunk RecordsCountSum
aws_firehose_DeliveryToSplunk_Bytes

Number of bytes delivered to Splunk over the specified time period.
DeliveryToSplunk.BytesDelivery To Splunk BytesBytesSum
aws_firehose_DeliveryToSplunk_DataAckLatency

The approximate duration it takes to receive an acknowledgement from Splunk after Kinesis Data Firehose sends it data.
DeliveryToSplunk.DataAckLatencyDelivery To Splunk Data Ack LatencySecondsAverage
aws_firehose_DeliveryToSplunk_DataFreshness

Age (from getting into Kinesis Data Firehose to now) of the oldest record in Kinesis Data Firehose. Any record older than this age is delivered to Splunk.
DeliveryToSplunk.DataFreshnessDelivery To Splunk Data FreshnessSecondsAverage
aws_firehose_KinesisMillisBehindLatest

When the data source is a Kinesis data stream, this metric indicates the number of milliseconds that the last read record is behind the newest record in the Kinesis data stream.
KinesisMillisBehindLatestKinesis Millis Behind LatestMillisecondAverage
aws_firehose_RecordsPerSecondLimit

Current maximum number of records per second that a delivery stream can ingest before throttling.
RecordsPerSecondLimitRecords Per Second LimitBytesAverage
aws_firehose_ThrottledRecords

Number of records that were throttled because data ingestion exceeded one of the delivery stream limits.
ThrottledRecordsThrottled RecordsCountAverage
aws_firehose_IncomingPutRequests

Number of PutRecord and PutRecordBatch requests over the specified period of time.
IncomingPutRequestsIncoming Put RequestsCountAverage
aws_firehose_PutRequestsPerSecondLimit

Maximum number of put requests per second that a delivery stream can handle before throttling.
PutRequestsPerSecondLimitPut Requests Per Second LimitCountAverage
aws_firehose_ThrottledDescribeStream

Total number of times the DescribeStream operation is throttled when the data source is a Kinesis data stream.
ThrottledDescribeStreamThrottled Describe StreamCountAverage
aws_firehose_ThrottledGetRecords

Total number of times the GetRecords operation is throttled when the data source is a Kinesis data stream.
ThrottledGetRecordsThrottled Get RecordsCountAverage
aws_firehose_ThrottledGetShardIterator

Total number of times the GetShardIterator operation is throttled when the data source is a Kinesis data stream.
ThrottledGetShardIteratorThrottled Get Shard IteratorCountAverage
aws_firehose_KMSKeyAccessDenied

Number of times the service encounters a KMSAccessDeniedException for the delivery stream.
KMSKeyAccessDeniedKMS Key Access DeniedCountAverage
aws_firehose_KMSKeyDisabled

Number of times the service encounters a KMSDisabledException for the delivery stream.
KMSKeyDisabledKMS Key DisabledCountAverage
aws_firehose_KMSKeyInvalidState

Number of times the service encounters a KMSInvalidStateException for the delivery stream.
KMSKeyInvalidStateKMS Key Invalid StateCountAverage
aws_firehose_KMSKeyNotFound

Number of times the service encounters a KMSNotFoundException for the delivery stream.
KMSKeyNotFoundKMS Key Not FoundCountAverage