Introduction
Fluent Bit is a super-fast, lightweight, and highly scalable logging and metrics processor and forwarder. It is the preferred choice for cloud and containerized environments. OpsRamp allows you to export logs from Fluent Bit, which is already configured to collect logs from various sources.
Note
Opsramp only supports ingestion of logs from Fluent Bit but not metrics.Configuration for exporting logs to OpsRamp
Add the following configuration to your Fluent Bit configuration file to export logs to OpsRamp:
[FILTER]
Name modify
Match *
Add source fluent-bit
Rename time timestamp
Rename log message
Rename _HOSTNAME host
[OUTPUT]
name http
match *
host
port 443
tls on
uri
format json
Note
The service label can be used to specify the source from which the log is collected.Example 1: Fluent Bit configuration for exporting logs to OpsRamp
Below is a complete configuration example for Fluent Bit to export logs to OpsRamp and specify the source of the logs using the service label:
[SERVICE]
# Flush
# =====
# set an interval of seconds before to flush records to a destination
flush 1
# Daemon
# ======
# instruct Fluent Bit to run in foreground or background mode.
daemon Off
# Log_Level
# =========
# Set the verbosity level of the service, values can be:
#
# - error
# - warning
# - info
# - debug
# - trace
#
# by default 'info' is set, that means it includes 'error' and 'warning'.
log_level debug
# Parsers File
# ============
# specify an optional 'Parsers' configuration file
parsers_file parsers.conf
# Plugins File
# ============
# specify an optional 'Plugins' configuration file to load external plugins.
plugins_file plugins.conf
# HTTP Server
# ===========
# Enable/Disable the built-in HTTP Server for metrics
http_server Off
http_listen 0.0.0.0
http_port 2020
# Storage
# =======
# Fluent Bit can use memory and filesystem buffering based mechanisms
#
# - https://docs.fluentbit.io/manual/administration/buffering-and-storage
#
# storage metrics
# ---------------
# publish storage pipeline metrics in '/api/v1/storage'. The metrics are
# exported only if the 'http_server' option is enabled.
#
storage.metrics on
# storage.path
# ------------
# absolute file system path to store filesystem data buffers (chunks).
#
# storage.path /tmp/storage
# storage.sync
# ------------
# configure the synchronization mode used to store the data into the
# filesystem. It can take the values normal or full.
#
# storage.sync normal
# storage.checksum
# ----------------
# enable the data integrity check when writing and reading data from the
# filesystem. The storage layer uses the CRC32 algorithm.
#
# storage.checksum off
# storage.backlog.mem_limit
# -------------------------
# if storage.path is set, Fluent Bit will look for data chunks that were
# not delivered and are still in the storage layer, these are called
# backlog data. This option configure a hint of maximum value of memory
# to use when processing these records.
#
# storage.backlog.mem_limit 5M
[INPUT]
name tail
path /var/log/syslog
key message
[FILTER]
Name record_modifier
Match *
Record host ${HOSTNAME}
Record source fluent-bit
[FILTER]
Name modify
Match *
Rename date timestamp
[OUTPUT]
name http
match *
host
port 443
tls on
uri
format json
Example 2: Fluent Bit configuration with resource attributes
Before exporting the log events, ensure that the following attributes or fields are set using Fluent Bit configuration:
Resource attributes:
- source
- host
- level (If not set, it is considered as “Unknown”)
Parsed labels:
- message # Mandatory field to be set
- timestamp ( If not set, the time the record received at Opsramp is considered as log record time )
- level ( If not set, it’s considered as “Unknown” )
Note
You can make additional attributes filterable by prefixing the required keys withopsramp_filter_
. OpsRamp supports up to 10 filterable fields.Note
Ensure that the cardinality of these filters is not too high, as higher cardinality can impact the performance of data searches.See Fluent-Bit configuration for more details.