Introduction

You can stream AWS logs to OpsRamp using Kinesis Data Firehose Delivery Stream. If you are already using Amazon CloudWatch Logs, AWS allows you to push log data to a specific HTTP Endpoint using Kinesis Data Firehose.

Streaming Logs Using Kinesis Data Firehose

To forward AWS CloudWatch logs to OpsRamp, follow the steps below to create an Amazon Kinesis Data Firehose Delivery Stream:

  1. Navigate to the AWS console.

  2. Under the Analytics section, choose Kinesis Service.

  3. Click Create delivery stream.

  4. Select Direct PUT or other sources as the source using the Kinesis Data Firehose PutRecord API for Source Configuration.

  5. Select HTTP Endpoint as the destination for this Delivery Stream.

  6. OpsRamp recommends setting the batch size to 1MB. The delivery stream sends logs if the batch size of 1MB is reached or if the longer time (minimum 60 seconds) is reached.

  7. Enter a name for the Delivery Stream.

  8. Enter a name for the HTTP Endpoint (optional).

  9. Pass the OpsRamp HTTP endpoint URL for HTTP Endpoint URL configuration.

  10. For Content encoding, choose the Disabled option as the content encoding of your request.

  11. For S3 backup mode, select Failed data only.

  12. For the S3 bucket, enter an S3 bucket as a backup for the delivery stream to store data that failed delivery to the HTTP API endpoint.

  13. Review your settings and choose Create delivery stream.

Analyze Logs in OpsRamp

When the delivery stream is active, your source will be able to stream log data to OpsRamp. Once this configuration setup is completed at the AWS Console, you can analyze logs in OpsRamp Logs Explorer.

Supported Labels

When integrating AWS logs with OpsRamp, the following labels can be used to categorize and manage your log data effectively:

  • messageType
  • owner
  • logGroup
  • logStream
  • subscriptionFilters
  • id