Introduction
Syslog-ng, a flexible and scalable log management solution, can integrate seamlessly with OpsRamp for efficient log collection and monitoring. This integration supports two primary methods: Agent-based and Direct forwarding.
- Agent-Based Integration: With agent-based integration, Syslog-ng forwards logs to OpsRamp through an installed agent. The agent collects syslog-ng logs and transmits them securely to OpsRamp, where they are processed for monitoring, alerting, and analysis.
- Direct: Syslog-ng can directly forward logs to OpsRamp without requiring an intermediary agent. This method is beneficial for environments where direct communication is preferred or where agent deployment may be impractical.
Configuration
Below is the default configuration for integrating Syslog-ng with OpsRamp in an agent-based setup:
# supported version for this configuration is Syslog-NG 3.25.1-3
syslog_server:
type: syslog
source: syslog_server
address: ""
mode: tcp
protocol:
Note
Supported in agent version 13.1 and above.Syslog-ng configuration
- To forward logs using syslog-ng, add the following configuration to your
/etc/syslog-ng/syslog-ng.conf
file. - The following configuration specifies that all traffic coming in on port 515 should be forwarded to the OpsRamp agent, which is listening on port 514.
- The source of the input can vary depending on the use case, but to export logs to the OpsRamp portal, the destination must be configured as shown below:
source s_net {tcp(ip(0.0.0.0) port(515)); };
destination d_net { tcp("0.0.0.0" port(514) log_fifo_size(1000)); }; # This line specifies the destination
log { source(s_net); destination(d_net); };
- Execute the following command in the terminal to restart Syslog-NG:
systemctl restart syslog-ng.service
Note
The Syslog-NG configuration shown here is generic. If you need to forward logs for only a specific host, please refer to the official Syslog-NG documentation.Custom Configuration
- Create a custom configuration file at
/opt/opsramp/agent/conf/log.d/log-config.yaml
by copying the content from the sample config file/opt/opsramp/agent/conf/log.d/log-config.yaml.sample
. - Update (add/edit/remove) the configurations as needed for the
syslog_server
source, using the guidelines for field definitions provided below:
type: "" # "syslog" is the type for syslog
source: "" # specify the name of the application(if empty the source name is used)
mode: # The type of socket to use. Valid Sockets are ["tcp","udp","unix"]
address: # specify the "ip:port" on which the syslog server must run (only applicable if mode is set to "tcp" or "udp")
protocol: # The protocol to parse the syslog messages as. Options are rfc3164 and rfc5424
filters: # (optional) This section is used for filtering out logs by attribute_type. Execution happens in the order in which the filters are specified.
- attribute_type: "body" # Possible values for attribute_type [ "body", "attributes", "resource" ] (defaults to "body" if the field is ignored).
key: "" # The tag for which the respective filtering rule must be applied.
include: "" # keeps the records which match the specified pattern.
- key: "" # attribute_type defaults to "body" if the field is ignored.
exclude: "" # removes the records which match the specified pattern
masking: # (optional) This section is used to specify rules to mask any sensitive data in the logs.
- text: "" # The text which needs to be masked.
placeholder: "" # The string to replace the text to be masked.
labels: # (optional) Allows to set up to 5 resource labels. If more than 5 are specified then the first five are considered and others are ignored.
key: "{{value}}"
attributes: [ ] # (optional) Specify parsed fields which need to be set as record attributes.
resource_attributes: [ ] # (optional) Specify parsed fields which need to be set as resource attributes.
- Save the updated configuration file at
/opt/opsramp/agent/conf/log.d/log-config.yaml
. - Restart the agent by executing the following command in the terminal:
systemctl restart opsramp-agent
Supported labels
Following are some of the supported labels:
- source
- resourceUUID
Configuration
To transmit logs from syslog-ng to OpsRamp, set up a new HTTP destination and utilize it to forward logs from the necessary sources in syslog-ng.
Note
The following steps use the default configuration path: /etc/syslog-ng. You can make the necessary changes if your installation uses a different configuration file path.- Create a new configuration file. It is suggested to generate a new configuration file in the conf.d directory within the syslog-ng configuration folder.
touch /etc/syslog-ng/conf.d/10-opsramp.conf
- Add the below content to
/etc/syslog-ng/conf.d/10-opsramp.conf
file created in Step 1.
destination d_opsramp_http {
http(
url("")
method("PUT")
batch-lines(1000)
batch-bytes(1024Kb)
batch-timeout(5000)
user-agent("syslog-ng User Agent")
headers("Content-Type: application/syslog.rfc5424")
body-suffix("\n")
body("<${PRI}>1 ${ISODATE} ${HOST:--} ${PROGRAM:--} ${PID:--} ${MSGID:--} ${SDATA:--} ${MESSAGE}")
);
};
The destination configuration above includes the HTTP endpoint for exporting logs, necessary headers, and default batching options for transmitting logs to OpsRamp.
Note
- For additional configuration details regarding the HTTP destination, refer to the syslog-ng Open Source Edition 3.30 - Administration Guide.
- Set the Content-Type header to application/syslog.rfc5424 for the configuration to work properly.
- OpsRamp only supports request sizes up to 1MB, therefore, batch-bytes must always be configured to a value less than 1MB.
/etc/syslog-ng/conf.d/10-opsramp.conf
file after the destination is configured. The source configuration is named s_src
as shown below:log { source(s_src); destination(d_opsramp_http); };`
- Restart the syslog-ng service using the following command:
sudo service syslog-ng restart
Supported labels
Supported labels refer to the specific identifiers or attributes that can be associated with log messages in syslog-ng. These labels provide additional context or metadata about the log messages, helping to search, sort, and process log messages efficiently.Following are some of the supported labels:
- source
- host
- level
- facility
- severity