An alert escalation policy defines user settings that OpsRamp applies in escalating alerts.
After an alert is correlated, OpsRamp provides options to notify users about the alert and automatically create incident tickets. The goal of alert escalation is to notify users of critical alerts so that a user acknowledges the alert.
Escalation is useful if you follow an on-call process for an alert response. In an on-call process, your IT staff does not watch a console for alerts. Instead, automated notifications are sent to designated staff on pre-defined shifts.
With escalation features, you can notify users using email, text, and voice messages, based on the following criteria:
- User and alert type: Notify specific users based on the type of alert. For example, notify database administrators of alerts from database servers.
- Shift schedule: Notify specific users based on when they are available. For example, notify the IT staff on the day shift, of alerts that arrive between 8:00 AM - 5:00 PM, and notify IT staff on the evening shift, of alerts that arrive between 5:00 PM - 2:00 AM.
- Chain of responsibility: Notify users up a chain of responsibility if alerts remain unacknowledged after notification. For example, notify shift managers of unacknowledged alerts of alerts that remain unacknowledged 30 minutes after the first notification was sent to level 1 staff.
According to functionality, only one incident creation configured client policy will be considered if a client or partner has multiple incident creation configured policies. The client-level policies will be taken into consideration based on precedence order if the partner does not have any incident creation policies specified. But regardless of client or partner level, all notification configured policies will be considered.
View Alert Escalation Policies
The Alert Escalation Policies page lists the escalation policies.
Ensure that you have selected a client from the ALL Clients list.
Go to Setup > Alerts > Alert Escalation.You can find the list of escalation policies applied to the selected client.
Each escalation policy contains the following information:
| Attribute | Description |
|---|---|
| Alert Escalation Policy Name | Name of the alert escalation policy. |
| Applied To | Indicates to which client the policy is applied. |
| Last Updated By | Name of the user who last modified the policy. |
| Last Updated Time | Time the policy was last modified. |
| Precedence | Determine the order in which to handle alerts according to the alert precedence value.Note: This is an optional field and accepts only integer value. |
| No. of notifications | Shows the number of notifications mapped to an escalation policy. You can click the count to view the alerts for which the notifications have been sent.Note: As per the purging policy, deleting alerts will result in a corresponding reduction in the alert count. |
| No. of incidents | Shows the number of incidents mapped to an escalation policy. You can click the count to view the alerts for which the incidents were created.Note: As per the purging policy, deleting alerts will result in a corresponding reduction in the alert count. |
| ML Status | Shows the various stages of machine learning implementation in a policy from analyzing a sequence to correlating alerts. |
| Mode | You can select supported policy modes from the drop-down list. |
Precedence order
The precedence refers to the order or priority assigned to different escalation levels and defines the sequence in which alerts are escalated or forwarded to higher levels.
The alert escalation policies with lower precedence values are usually considered to have a higher priority or are picked first. A lower precedence value typically indicates a higher priority level. Therefore, an escalation policy with a precedence value of 1 would be considered a higher priority than those with higher precedence values.
For example:
- Alert Escalation Policy A: Precedence 1
- Alert Escalation Policy B: Precedence 2
- Alert Escalation Policy C: Precedence 3In this case, Policy A would be considered first, followed by Policy B and Policy C in descending order of precedence.
Note: If a precedence order value is not specified, the system will select according to the default priority concept, where the policy with the highest priority for the new incident or notification will be selected first.
Policy modes
The following policy modes are supported:
| Policy Mode | Description |
|---|---|
| ON | The policy drives automated actions on alerts. |
| OFF | The policy is inactive and does not affect alerts. You can use this mode to review a newly defined policy before choosing one of the other modes. |
| Recommend | The policy creates a recommendation for actions that you should take on the alert. Recommendations are based on OpsRamp learning from historical alerts. The recommendation includes a link to take the action. |
| Observed | This mode permits you to simulate a policy without affecting alerts. The policy creates an observed alert, which simulates the original alert. The observed alert shows the actions that would be taken on the original alert if the policy were in On mode. The observed alert includes a link to the original alert. |
| Recommend and Observed modes apply to incident actions. | |
User scope setting
This setting selects whether alerts are escalated to users within your organization account (partner) or users within a client account.
Resource scope setting
This setting selects resources for which alerts are escalated.
Define alert and resource conditions
You define the alert and resource conditions for the escalation policy.
Action setting
The action setting selects the following:
- Whether the policy takes automated escalation actions or just shows which users should be contacted directly.
- How long to wait before sending a notification or creating an incident.
- Alert state transitions at which notifications are sent.
- What priority is assigned to a notification.
- At what frequency notifications are repeated.
- When notification should stop.
- To which users notifications are sent or to which users incidents are assigned.
- Attributes and content of an auto-created incident. You can specify incident attributes or let OpsRamp automatically set attributes based on learning from historical alerts.
- Whether to update an incident On changes to the alert state.
Alert tokens
An alert token is a placeholder added to an alert that is escalated as an incident so that the incident includes the data the token represents. For example, if $alert.serviceName (Alert metric) is added to an incident after the alert is escalated as an incident, the incident includes the alert metric name.
Tokens are divided into three categories:
- Alert: Alert category is divided into Alert specific tokens and Alert Resource tokens
- Policy: Policy category consists of the Policy name token.
- Functions: Functions category consists of the Substring token.
A substring is a string between a pointer for reference start character and reference end character. A reference pointer is a delimiter. The substring function token allows you to create a dynamic description for an incident. A delimiter is used to specify boundaries for identifying the appropriate string in a data stream. The startDelimiter indicates the beginning element in a character string, and the endDelimiter indicates the end element in a character string.
For example, a user wants to extract the Site Account from the Alert Description and set the account number to the Account Id of the Incident record.
To extract the Account ID from the Alert Description, use the substring ($function.substring(<String>,<startDelimiter>,<endDelimiter>) ) function, and enter the following values:
- String =
$alert.description - startDelimiter =
AT SITE ACCOUNT NO= - endDelimiter does not need to be configured to indicate a natural end delimiter
($function.substring($alert.description,AT SITE ACCOUNT NO=,<endDelimiter>)
Escalation matrix on alert
Based on the current details of an alert, the escalation policy details will be shown on the alert. For instance, if an alert is in a Critical state, the policy will be displayed as Critical. The alert will indicate the OK state condition if the alert’s status is changed to OK.
Details of the time-matched AEP policy applied functionality (such as notification or incident creation) will be shown in the comment section.
Next steps
- See Create Alert Escalation Policy to create an alert escalation policy.
- See Alert and Resource Condition Examples for alert and resource condition examples.