Remote Script Executor (RSE)

Introduction

Using Remote Script Executor (RSE), you can execute scripts on both Linux and Windows workstations with different script types.

• User-defined external arguments support the script through custom attributes.
• Monitor Level script execution – Ability to pull more metrics in a single script execution.
• Ability to execute different types of scripts through custom script options apart from default options.
• Able to use Credentials attached to the device in the script using the dynamic macro support.

Prerequisite

• Gateway version 7.0.0.
• End device’s SSH Credentials must be assigned on End device itself, not on the Gateway device.

Create Metrics

Metrics enter the metadata to correlate the script output and must match the metrics names added to the Monitor.

1. Select a client from the All Clients list.

2. Go to Setup > Monitoring > Metrics.

3. From METRICS, click + Add and enter:

   • Metric Scope: Select Partner Metric or Client Specific Metric.

   • Adaptor Type: Select Application.

   • Application Type: Select Remote Script Executor.

   • Name: Unique Name for the metric. The recommended metric naming convention is <vendorname>_<application name>_<component>_<metric> . For example, apache_tomcat_webapps_count_

   • Tag Name: User-defined tags for better filtering.

   • Display Name: Enter Display Name. For example, System Drive Free Space.

   • Description: Enter the reason for the metric.

   • Data Point type: Value chosen to plot data in the graph.

   • Units: Required values for Units. You can choose Units depending on the chosen Data Point Type.

   • Unit Multiplication Factor: Value chosen to multiply a value based on the chosen unit.

   • Data Point Conversion: One of the following options:

     • Value: To populate data using values.
     • Enumerated Map: To populate alert and graph details after adding the states such as Up or Down. The screen displays an more section where you can add the state descriptions and select Alerts and Graphs to populate data.

   • Metric Processing: One of the following values for processing the Metric:

     • Graph
     • Notification
     • Graph and Notification
     • None

   1. Graph
      Choose this option when user only needs a Graph for the metric without an alert.

   2. Notification
      Choose this option when user only needs an Alert for the metric without a graph.
      When selecting this option, a dynamic UI will appear to configure alert thresholds, as shown in the screenshot below:

      

      Configuring Alert Thresholds:
      

      When setting up notifications, the following options are available for configuring alert thresholds:
      

      Case 1: Enumerated Map

• If "Datapoint Value Conversion" is set to "Enumerated Map," users need to specify the oState column values for Warning and Critical thresholds.
• Warning Threshold: State [Specify State column Value]
• Critical Threshold: State [Specify State column Value]

Setting Alert thresholds for Enumerated Map conversion from strings to integers:

Setting Alert thresholds for Enumerated Map conversion from integers to strings:

Case 2: Value

• If "Datapoint Value Conversion" is set to "Value" (the default setting), users need to specify the metric values for Warning and Critical thresholds.
• Warning Threshold: Value [Specify Warning-level Threshold Value for the metric]
• Critical Threshold: Value [Specify Critical-level Threshold Value for the metric]

Additional Fields:

• Subject: The metric subject is populated by default but can be customized.
• Description: The metric description is populated by default but can be customized.

3. Graph and Notification:

Choose this option when user needs both Alert and Graph for the metric.

When selecting this option, a dynamic UI will appear to configure alert thresholds, as shown in the screenshot below:

Configuring Alert Thresholds:
The alert threshold configuration options are the same as the Notification option.

4. None: Select this option when both alert and graph are not needed for the metric.

1. Click Save.

Create monitor

Custom Remote Script Executor Monitor is a collection of Remote Script Executor metrics. You can create a template using the Remote Script Executor monitor.

Prerequisite: Add the required custom parameters.

1. From All Clients, select a client.
2. Go to Setup > Monitoring > Monitors.
3. From Monitors, click + Add.
4. From CREATE A MONITOR, enter the details for the following parameters and click Save.
   • Monitor Scope: Select Partner Monitor or Client Specific Monitor.
   • Type: Select Application.
   • Application Type: Select Remote Script Executor.
   • Name: Enter a unique name for the monitor.
   • Description: Enter the reason for creating the monitor.
   • Script: Custom script in the Script text box. You can refer to Script and JSON output formats-example to view the example script.
   • Metrics: To add Metrics:
     1. From Metrics, click Add Metrics.
     2. Search the metric and click Add Metrics. The Metrics section displays the selected metric.
     3. After adding the metrics, navigate to Configuration Parameters.
   • Configuration Parameters: To add configuration Parameters:
   1. Enter Default Value for all the configuration parameters.
   2. Select the Configurable checkbox.
   3. Click +Add.
   4. Enter Configuration Parameters depending on the selected script type.
      You can view the default value for the respective Name selected.
      If you select the Custom value in Configuration Parameters, add the configuration parameter: custom.script.execution.path. The custom.script.execution.path option allows users to execute various types of scripts beyond the default options (Bash/PowerShell/Python). For instance, if a user is using Perl for monitoring, they can select the Custom in the remote.server.scripttype configuration parameter.
   5. Click Add in Configuration Parameters.
   6. Select custom.script.execution.path, and provide the path to the Perl executable.
      Note: By default, VmWare appliance do not have bash prompt, in such cases you can provide custom path for script execution by using this custom option.
      

Example: If you are using Python3 for monitoring, you can follow the same process and provide the path to the Python3 executable, such as /usr/bin/python3. Using the executable path ensures that the correct interpreter is used to run the script.

MONITORS screen displays the new monitor.

Macros in Remote Script Executor

Macros are implemented to address the passing of dynamic arguments to scripts. Use the following macros to make available native attributes and custom attribute defined on the resource, in the script:

Static macros

Use static macros to override the resource values. See the Static macro reference for a list of static macros.

Dynamic macros

[{
    "Process_Regex_Expression": "opsramp.*",
    "Process_Friendly_Name": "OpsRamp_Processes"
}, {
    "Process_Regex_Expression": "svchost.*NetworkService",
    "Process_Friendly_Name": "svchost_NetworkService"
}, {
    "Process_Regex_Expression": "trayicon.exe",
    "Process_Friendly_Name": "OpsRampTrayIcon"
}]

Credential Types

Below are the supported credential types:

[{
"uuid":"xB35mBgN354UGM3zQFZBQSaJ",
"type":"VMWARE",
"name":"sample-vmware",
"timeoutMs":10000,
"transportType":"HTTPS",
"appName":null,
"domain":null,
"username":"testuser",
"password":"xxxxxx",
"port":443
}]

[{
"uuid": "NwKGzg6qqSF29Sdy5hRJrAyH",
"type": "SSH",
"name": "sample-ssh",
"timeoutMs": 10000,
"transportType": "HTTP",
"appName": null,
"userName": "testuser",
"password": "xxxxxx",
"port": 22,
"privkey": null,
"passPhrasePasswd": null,
"privKeyFileName": null,
"sshAuthType": "PASSWORD"
    }]

The following are the remaining credential types:
SNMP, XEN, WINDOWS, JMX, HTTP, Database, CIM, NETAPP, NETAPPCLUSTER, HYPERFLEX, PURESTORAGE, FTP, CISCOUCS, EMCCLARIION, EMCVNX, EMCVNXE, EMCVMAX, IBM, HPEVA, REMOTE_CLI, TELNET, XTREMIO, VIPTELA, EMCVPLEX, EMCRPA, NUTANIX, HITACHIVSP, AZURESTACK, APPLICATION, and CITRIX_CVDA.

Script execution path for configuration parameters in Linux and Windows

If your runtime is not set as an environment variable, mention the absolute path of your runtime. For example, if Python is not set as an environment variable, enter the absolute path of Python such as /usr/lib/python.

Create template

A template is an instance of a monitor and is applied to devices.

1. From All Clients, select a client.

2. Go to Setup > Monitoring >Templates.

3. Click + Add.

4. From MONITOR TEMPLATE, enter the details for the following parameters and click Save:

   • Select Template Scope: Template type.

   • Collector Type: Select Gateway.

   • Monitor Type: Select Monitors for G2 Templates.

   • Applicable for: Select Device.

   • Template Name: Name of the template.

   • Description: Summary of the template.

   • Generation: Generation that the template belongs to.

   • Tags: User-defined tags for better filtering.

   • Prerequisites: Essential things that you must consider while monitoring using the template. For example, check the SQL services while monitoring the SQL Parameters using the Windows templates.

   • Status: Active or End-of-life templates.

   • Notes: Information that you want to add to the template.

   • Template Family Name: Category that applies to the application. For example, Windows Server, Storage Server, and Network Server.

   • Deployment Type: One of the following methods to apply the template to the resources:

     • Custom
     • Optional
     • Standard

5. After providing the template details, MONITOR TEMPLATE displays the Monitors section.
   The user interface for the Monitors section varies with the option selected in the Collector Type.

   • Monitor: To add details in Monitors:

   1. From the Monitors section, click Add.

   2. Enter details for the following parameters:

      • Frequency: Refer to the frequency to execute the template.
      • Monitor Definition: Monitor type.
      • When to Alert: Select Breach of a Threshold.

   3. Select Availability and Alert checkbox to initiate monitoring. The TEMPLATES section displays the configured details.

Assign template to a device

You need to create credentials to assign a template to a device.

Create credentials

If the configuration parameter remote.server.operating system is:

• Windows, you must consider Windows type credentials.
• LINUX or UNIX, you must consider SSH type credentials.

1. Select a client from the All Clients list.

2. Go to Infrastructure > Resources.

3. From the list of devices, select a device. Navigates to the device information page.

4. From the center pane, click Credentials.

5. Click +Add.

6. From Create new credential, provide details for the following parameters and click Save:

   • Name: Name of the credential.
   • Description: Description of the new credential.
   • Type: Application type. Select Windows.
   • Username: Username for the credential set.
   • Password: Password that you set for the credential set.
   • Confirm Password: Enter the same password.
   • Port: Port used to connect to the application.
   • Connection Time-out (ms): Time taken for the resource to connect to the application. The Credentials screen displays the new credentials.

Assign template

1. From the left pane, click Infrastructure. The Infrastructure screen of the selected client is displayed.
2. From the Resources tab, select a required resource from the list of resources. Or, use the search option to find the resource.
3. Click the resource name to view details.
4. From the left pane, click Monitors. Templates tab is displayed.
5. From Templates tab, click +Assign Templates.
6. From Apply Templates, select the templates. The selected Templates section displays the chosen templates.
7. Click Assign. The template gets assigned to the selected device.

After assigning the template to a resource for monitoring, click Get Latest Metric Values to view the latest metric information.

Use the following global templates to monitor your Linux resources:

• Gateway - Linux OS Performance Remote Monitoring
• Gateway - Linux OS Performance Remote Monitoring - V2
• Gateway - Linux Network Remote Monitoring

View graphs

The gateway monitors the remote system using the assigned templates and displays the results in a graphical format.

1. From the left pane, click Infrastructure. The Infrastructure screen of the selected client is displayed.
2. From the Resources tab, select a required resource from the list of resources. Or, use the search option to find the resource.
3. Click the resource name to view details.
4. From the left pane, click Metrics. The Metrics page displays graphs generated by all monitoring templates assigned to a device.
5. Search with the template name to filter the graphs.

Troubleshooting

• The device is accessible to the gateway.
• The appropriate port is available from the remote server.

Remote script executor example

#!/bin/bash

CPU=$(top -bn1 | grep load | awk ‘{printf "%.2f\t\t\n", $(NF-2)}’)
MEMORY=$(free -m | awk ‘NR==2{printf "%.2f\t\t", $3*100/$2 }’)
DISK=`df -h | awk ‘$NF=="/"{ print $5}’ | sed ‘s/%//g’`

printf "{\"disk.utilization\" : %s , \"memory.utilization\" : %s , \"cpu.usage\" : %s}" "$DISK" "$MEMORY" "$CPU"

Output :

{"disk.utilization" : 23 , "memory.utilization" : 24.28 , "cpu.usage" : 0.64 }

Standard JSON Output formats: Remote script executor script

Format 1:

Description:

This format is used when the metric does not have any dynamic components or instances. In such cases, construct the JSON by using the metric name as the key and mapping that key to the appropriate value.

Format for the JSON output when there are no exceptions:

{
  "Metric1": 98,
  "Metric2": 70,
  "Metric3": 80
}

Or

Format for the JSON output when exceptions occur:

{

  "Metric1": 98,
  "Metric2": 70,
  "Metric3": 80,
  "scriptExceptions": {
    "subject": "No monitoring data / Unable to fetch monitoring data / Incomplete script execution",
    "description": "Failed to collect data for following metrics. \n metricName: <Metric Name 1>, FailureReason: Failure Reason 1 \n metricName: <Metric Name 2>, FailureReason: Failure Reason 2",
    "raiseAlert": true,
    "logRequired": true,
    "alertState": "warning"
  }
}

Example:

{
  "system_windows_uptime_inMinutes": 120,
  "system_windows_overallCPU_utilization": 50,
}

Format 2:

Description: This format is used, when one metric returns a string and another metric returns a number. In such cases, a JSON output is prepared by mapping those strings or metrics to their appropriate values.

{
"Metric1": 98,
"Metric2": "STATE",
"Metric3": 80
}

Example:

{

   "system_windows_memory_usage": 35.75,
   "system_windows_winrm_status": "running"
}

Note: We recommend using numerical representation for String outputs by utilising Enumerated Map option of RSE.

Format 3:

Description: This format may be used, when the metric contains multiple components. In the below example, the data is fetched for multiple components of disks.

Format for the JSON output when there are no exceptions:

{
  "MetricName1": {
    "components": {
      "component1": 70,
      "component2": 98
    }
  },
  "MetricName2": {
    "components": {
      "component1": 77,
      "component2": 98
    }
  }
}

Or

Format for the JSON output when exceptions occur:

{
  "MetricName1": {
    "components": {
      "component1": 70,
      "component2": 98
    }
  },
  "MetricName2": {
    "components": {
      "component1": 77,
      "component2": 98
    }
  },

"scriptExceptions": {
    "subject": "No monitoring data / Unable to fetch monitoring data / Incomplete script execution",
    "description": "Failed to collect data for following metrics. \n metricName: <Metric Name 1>, FailureReason: Failure Reason 1 \n metricName: <Metric Name 2>, FailureReason: Failure Reason 2",
    "raiseAlert": true,
    "logRequired": true,
    "alertState": "warning"
  }
}

Example:

{
  "System_Windows_PhysicalDisk_WriteBytes_PerSec": {
    "components": {
      "physicaldisk_1": 0,
      "physicaldisk_2": 0
    }
  },
  "System_Windows_PhysicalDisk_AvgDisk_SecPerWrite": {
    "components": {
      "physicaldisk_1": 0,
      "physicaldisk_2": 0
    }
  }
}

Explanation:

In this example, System_Windows_PhysicalDisk_WriteBytes_PerSec is a metric name that has two components, physicaldisk_1 and physicaldisk_2, with their values mapped inside the components key.

Similarly, System_Windows_PhysicalDisk_AvgDisk_SecPerWrite is another metric name organized in the same way.

Format 4:

Description: This format is used when user needs metric-level alertTokens for all metrics of the script. Alert tokens are used to specify more information about the metrics in alert subject or in alert description or in both.

Format for the JSON output when there are no exceptions:

{
  "MetricName1": {
    "components": {
      "component1": "STATE",
      "component2": 98
    },
    "alertTokens": {
      "token1": "value",
      "token2": "value2"
    }
  },
  "MetricName2": {
    "components": {
      "component1": 77,
      "component2": 98
    },
    "alertTokens": {
      "token1": "value",
      "token2": "value2"
    }
  }
}

Or

Format for the JSON output when exceptions occur:

{
  "MetricName1": {
    "components": {
      "component1": "STATE",
      "component2": 98
    },
    "alertTokens": {
      "token1": "value",
      "token2": "value2"
    }
  },
  "MetricName2": {
    "components": {
      "component1": 77,
      "component2": 98
    },
    "alertTokens": {
      "token1": "value",
      "token2": "value2"
    }
  },
  "scriptExceptions": {
    "subject": "No monitoring data / Unable to fetch monitoring data / Incomplete script execution",
    "description": "Failed to collect data for following metrics. \n metricName: <Metric Name 1>, FailureReason: Failure Reason 1 \n metricName: <Metric Name 2>, FailureReason: Failure Reason 2",
    "raiseAlert": true,
    "logRequired": true,
    "alertState": "warning"
  }
}

Example:

{
  "system_linux_memory_utilization": {
    "components": {
      "real_memory_utilization": 50
    },
    "alertTokens": {
      "memory.usage": "Total memory: 16 GB, Used memory: 8 GB"
    }
  }
}

Explanation:

In this example, the system_linux_memory_utilization metric includes a component real_memory_utilization with a value of 50. The alert tokens offer a summary at the metric level, indicating “Total memory: 16 GB, Used memory: 8 GB”. This information is included in the alert description, the alert subject, or both for the system_linux_memory_utilization metric.

Format 5:

Description: This format is used, when user needs alert tokens at metric-level, for only few of the metrics of the script.

Format:

{
    "MetricName1": 254,
    "MetricName2": {
        "components": {
            "comp1": 90,
            "comp2": 60
        }
    },
    "MetricName3": {
        "components": {
            "component1": "STATE",
            "component2": 98
        },
        "alertTokens": {
            "token1": "value",
            "token2": "value2"
        }
    }
}

Example:

{
  "system_linux_services_status": {
    "components": {
      "iptables": "inactive",
      "irqbalance": "active",
      "kdump": "active",
      "ypxfrd": "inactive"
    },
    "alertTokens": {
      "services.active.counts": " 2 services are active",

      "services.inactive.counts": "2 services are inactive"
    }
  },
  "system_linux_totalServices_count": "4"
}

Explanation:

The system_linux_services_status metric includes two alert tokens at the metric-level.

• Token 1, denoted as services.active.counts, provides the count of active services.
• Token 2, identified as services.inactive.counts, offers the count of inactive services.

These tokens provide valuable insights into the status of the services monitored under this metric.

On the other hand, the metric system_linux_services_count does not have any associated alert tokens. So, alerts for this metric will not include any additional descriptive information.

Format 6:

Description: This format is used, when user needs Component Level Alert Tokens. Alert token value for each metric component are specified separately, as mentioned below:

Format for the JSON output when there are no exceptions:

{
  "MetricName1": 254,
  "MetricName2": {
    "components": {
      "comp1": 90,
      "comp2": 60
    }
  },
  "MetricName3": {
    "components": {
      "component1": "STATE",
      "component2": 98
    },
    "alertTokens": {
      "token1": "value",
      "token2": "value2"
    }
  },
  "MetricName4": {
    "components": {
      "component1": 10,
      "component2": 20
    },
    "alertTokens": {
      "token1": {
        "component1": "token 1 value of component 1",
        "component2": "token 1 value of component 2"
      },
      "token2": {
        "component1": "token 2 value of component 1",
        "component2": "token 2 value of component 2"
      }
    }
  }
}

Or

Format for the JSON output when exceptions occur:

{
  "MetricName1": 254,
  "MetricName2": {
    "components": {
      "comp1": 90,
      "comp2": 60
    }
  },
  "MetricName3": {
    "components": {
      "component1": "STATE",
      "component2": 98
    },
    "alertTokens": {
      "token1": "value",
      "token2": "value2"
    }
  },
  "MetricName4": {
    "components": {
      "component1": 10,
      "component2": 20
    },
    "alertTokens": {
      "token1": {
        "component1": "token 1 value of component 1",
        "component2": "token 1 value of component 2"
      },
      "token2": {
        "component1": "token 2 value of component 1",
        "component2": "token 2 value of component 2"
      }
    }
  },
  "scriptExceptions": {
    "subject": "No monitoring data / Unable to fetch monitoring data / Incomplete script execution",
    "description": "Failed to collect data for following metrics. \n metricName: <Metric Name 1>, FailureReason: Failure Reason 1 \n metricName: <Metric Name 2>, FailureReason: Failure Reason 2",
    "raiseAlert": true,
    "logRequired": true,
    "alertState": "warning"
  }
}

Example:

{
  "system_linux_interfaces_count": {
    "components": {
      "system_linux_interfaces_count": 8
    },
    "alertTokens": {
      "interfaces_names": "interfaces names are cni0, ens160, flannel.1, veth05f2cc15,  veth10f0079d,  veth42a5dd4f,  vetha7efbaa5,  vethdc3c2d78"
    }
  },
  "system_linux_network_interface_trafficIn": {
    "components": {
      "cni0": 14329842632,
      "ens160": 13491465976
    },
    "alertTokens": {
      "mac.address": "cni0:42-a5-a4-fd-86-eb,ens160:00-0c-29-b7-be-c3"
    }
  },
  "system_linux_network_interface_trafficOut": {
    "components": {
      "cni0": "9148429744",
      "ens160": "12488937472"
    }
  },

"scriptExceptions": {
    "subject": "An exception has occurred. Unable to fetch the monitoring data",
    "alertState": "critical",
    "description": "Failed to collect data for following metrics. \n metricName: system_linux_network_interface_errorsIn, FailureReason: rx_errors attribute not available for interface cni0, metricName: system_linux_network_interface_errorsOut, FailureReason: tx_errors attribute not available for interface cni0",
    "raiseAlert": true,
    "logRequired": true
  }
}

Explanation:

In this example, the JSON structure demonstrates the use of scriptExceptions along with component-level and normal metric-level alert token.

The system_linux_interfaces_count metric includes a component that counts the number of interfaces and a metric-level alert token listing the names of these interfaces.

For system_linux_network_interface_trafficIn, the components show the inbound traffic for specific interfaces, while the component-level alert token provides the MAC addresses of these interfaces, offering more details on the interface.

system_linux_network_interface_trafficOut presents outbound traffic data for the interfaces but does not include additional alert tokens.

Multiple Custom Script Arguments in Macro

For any RSE script, we can pass upto 5 different types of custom arguments and for each argument we can pass a maximum of 20 values.

Each Custom argument contains 3 fields:

• Name
• Description
• Default Value

Here, Name and Description are the mandatory fields for a custom script argument.
The Default value is an optional field, which we can specify only if we want to give any default value to the custom script argument.

Marco Name: ${custom.script.multi.arguments}

Prerequisites

• Multiple custom script arguments are supported with Agent and Gateway version 14.0 and above.
• Macro custom.script.multi.arguments will support Agent and Gateway collector types.

Windows Process Monitoring - Use Case

This is a real time use case for multiple custom scrpit aurgument usages.

Step 1- Create Metric: Refer to the Document to know how to create metric.

Step 2- Create a Monitor: Refer to the Document to know how to create a monitor.
While creating a monitor, make the following changes:

• Under the Script section, use the below sample script.

$customArgs = ${custom.script.multi.arguments}
 
If (($([Int]$customArgs.Length) -eq 0) -or ($([String]$customArgs) -eq "Custom Script Arguments") -or ($($([String]$customArgs).Trim()) -eq ""))
{
    write-host "If a user does not provide any custom arguments, This script will exit automatically..."
    Exit 
}
 
$argSets = $([String]$customArgs) | convertFrom-Json
$rawDataColItems = Get-WMIObject Win32_PerfRawData_PerfProc_Process -namespace root\cimv2 -ErrorAction Stop | select Name, HandleCount, ThreadCount, IDProcess
    $processDataColItems = Get-WMIObject Win32_Process -namespace root\cimv2 -ErrorAction Stop | select Name, CommandLine, ProcessId
    $win32ProcessNameArgsDict = @{}
    $win32ProcessNamesDict = @{}
    ForEach($processDataColItem in $processDataColItems)
    {
        $win32ProcessNameArgsDict.$([String]$processDataColItem.ProcessId) = $([String]$processDataColItem.CommandLine)
        $win32ProcessNamesDict.$([String]$processDataColItem.ProcessId) = $([String]$processDataColItem.Name).replace(".exe","")
    }
 
    $handleCountStr = $ThreadCountStr  = ""
     
    ForEach($argSet in $argSets)
    {
        $reqProcessesCount = 0
        [UInt32]$handleCount = 0
        [UInt32]$ThreadCount = 0
        $componentName = ""
        $searchString = ([String]$argSet.Process_Regex_Expression).Trim()
        $componentName = ([String]$argSet.Process_Friendly_Name).Trim()
        if($searchString -eq "")
        {
            continue
        }
         
        ForEach($rawDataColItem in $rawDataColItems)
        {
            $procId = $([String]$rawDataColItem.IDProcess)
            $input = $([String]$win32ProcessNameArgsDict.$procId)
             
            If($input -eq "")
            {
                $input = $([String]$win32ProcessNamesDict.$procId)
                $searchString = $($searchString.Tolower()).replace(".exe","")
            }
            If($input -match $searchString)
            {
                If ($([String]$componentName) -eq "")
                {
                    $componentName = $($([String]$win32ProcessNamesDict.$procId).Trim())
                }
                $handleCount = $handleCount + $([UInt32]$rawDataColItem.HandleCount)
                $ThreadCount = $ThreadCount + $([UInt32]$rawDataColItem.ThreadCount)    
            }
        }
        If ($([String]$componentName) -eq "")
        {
            $componentName = $searchString
        }
        $handleCountStr += """$componentName""" + ":" + "$handleCount" + ","
        $ThreadCountStr += """$componentName""" + ":" + "$ThreadCount" + ","
    }
    $handleCountStr = $handleCountStr.Substring(0, $handleCountStr.Length - 1)
    $ThreadCountStr = $ThreadCountStr.Substring(0, $ThreadCountStr.Length - 1)
 
    Write-Host  "{""system.windows.process.handlecount"":{ ""components"": { $handleCountStr }}, ""system.windows.process.threadcount"":{ ""components"": { $ThreadCountStr }}}"

   

• Under the Metric section, click +Add and use the metrics created in Step 1.
• Under the Configuration Parameters section, click + Add and select custom.script.multi.arguments as shown below.

  

Step 3- Pass multiple custom script arguments: In the Monitor Creation Page, we must pass the following multiple custom script arguments.

• Argument 1:
  • Name: Process_Regex_Expression
  • Description: Provide regex expression to match process name and arguments.(Ex: opsramp.*)
• Argument 2:
  • Name: Process_Friendly_Name
  • Description: Provide process friendly name to be used as metric component name.(Ex: Opsramp_Processes).

Finally, click Save to save the Monitor page.

Step 4- Create a Template: Refer the Document to know how to create a template.
Create the template with above created metrics and monitor.

Step 5- Assign a template: Refer the Document to know how to assign template.

Below is the final script output

{
    "system.windows.process.handlecount": {
        "components": {
            "OpsRamp_Processes": 0,
            "svchost_NetworkService": 539,
            "OpsRampTrayIcon": 0
        }
    },
    "system.windows.process.threadcount": {
        "components": {
            "OpsRamp_Processes": 0,
            "svchost_NetworkService": 19,
            "OpsRampTrayIcon": 0
        }
    }
}

Generate Alert Tokens in RSE

The Alert Tokens feature is applicable for RSE G2 based templates that utilize the Agent or Gateway collector types. This feature will be useful when you wish to include additional details about an alert in the alert Subject or Description.

To utilize the alertTokens feature for any RSE metric, the components key is essential. Below are examples that illustrate how to set up alert tokens for both a single metric and multiple metrics:

Example: Alert Tokens for a Single Metric

{
  "windows_certStore_certificatesExpiryInDays": {
    "components": {
      "Entrust Root Certification Authority - G2_4A538C28": 2327,
      "Microsoft Root Certificate Authority 2011_3F8BC8B5FC9FB29643B569D66C42E144": 4259
    },
    "alertTokens": {
      "cert.info": "Entrust Root Certification Authority - G2_4A538C28: Issued By - Entrust Root Certification Authority - G2; Issued To - Entrust Root Certification Authority - G2; Serial Number - 4A538C28, Microsoft Root Certificate Authority 2011_3F8BC8B5FC9FB29643B569D66C42E144: Issued By - Microsoft Root Certificate Authority 2011; Issued To - Microsoft Root Certificate Authority 2011; Serial Number - 3F8BC8B5FC9FB29643B569D66C42E144"
    }
  }
}

Example: Alert Tokens for Multiple Metrics
If you have multiple metrics that require alert tokens, the same structure applies for each metric. Here’s how you need to structure it: (Here Mac Address will be shown in alert description of respective Network Interface.)

{
 "system_linux_network_interface_trafficIn": {
 "components": {
 "veth559f1827": "9112175520.00",
 "flannel.1": "949487296.00",
 "cni0": "96658072008.00"
 },
 "alertTokens": {
 "mac.address": "cni0:96-ca-36-db-b2-14,ens160:00-0c-29-b7-be-c3,flannel.1:2e-a5-e4-6a-5d-08,veth061406eb:6a-0d-89-90-8c-05,veth559f1827:e6-ac-8e-fe-94-d7"
 }
 },
 "system_linux_network_interface_trafficOut": {
 "components": {
 "veth559f1827": "8375015576.00",
 "flannel.1": "4254213376.00",
 "cni0": "90963679760.00"
 },
 "alertTokens": {
 "mac.address": "cni0:96-ca-36-db-b2-14,ens160:00-0c-29-b7-be-c3,flannel.1:2e-a5-e4-6a-5d-08,veth061406eb:6a-0d-89-90-8c-05,veth559f1827:e6-ac-8e-fe-94-d7"
 }
 }
}

Windows Certificates Store Monitoring - Use Case

This is a real time use case to generate alert tokens in RSE.

Step 1- Create a Metric: Refer to the Document to know how to create a metric.

While creating a metric, make the following changes:

• Name: In this case, we named the metric as windows_certStore_certificates_expiry_inDays_v2.
• Metric Processing: Choose Graph and Notification.
• Description: In the description section, add a string as cert.info:$cert.info to display more information on the alert.

Now, we have created a metric successfully.

Step 2- Create a Monitor: Refer to the Document to know how to create a monitor.

While creating a monitor, make the following changes:

• Under the Metric section, click +Add and use the metrics created in Step 1.
• Under Script section, use the below sample RSE script.

  $custom_args = ${custom.script.arguments}
  $global:certs = @{}
  $global:cert_attributes = @{}
  #####################################################
  Function get_attributes
  {
  param($input_certs)
           
          $now = Get-Date
   
          Foreach ($value in $input_certs)
          {
             $CertIssuer = $value.Issuer
             #$CertIssuer
             $CertIssuer = $CertIssuer.split(",")
             $cert_name = $CertIssuer[0]
             $cert_name = $cert_name -replace "CN=|OU=",""
             $CertSubject = $value.Subject
             #$CertSubject
             $CertSubject = $CertSubject.split(",")
             $cert_subject = $CertSubject[0]
             $cert_subject = $cert_subject -replace "CN=|OU=",""
             $cert_Serialnumber = $value.Serialnumber
             $count = (New-TimeSpan -Start $now -End $value.NotAfter).Days
             $certs[$cert_name] = [int]$count
             $attributes = "Issuer - $cert_name; Subject - $cert_subject; Serial Number - $cert_Serialnumber"
             #$attributes
             $cert_attributes[$cert_name] = $attributes
          }
  }
  #####################################################
  if((([String]$custom_args).Trim()) -eq "" -or ([int](($custom_args).length)) -eq 0)
  {
      write-host "Empty custom script arguments, hence exiting the script"
      Exit
  }
  else{
      Try
      {
          if((([String]$custom_args).Trim()) -eq "all")
          {
              $CertsAll = Get-ChildItem -path cert: -Recurse -ErrorAction Stop| where-object {$_.NotAfter -gt (Get-Date)} 
              get_attributes $CertsAll
          }
          else
          {
              $StrDecodedArgs = [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($custom_args))
              #Write-Host $StrDecodedArgs
              $arrParams = $StrDecodedArgs -Split(';')
              $StrCertPaths = $arrParams[0]
              $StrExclusionList = $arrParams[1]
   
              $arrCertPaths = $StrCertPaths -Split(',')
              $ExclusionItems = $StrExclusionList -Split(',')
   
               
              If ($([int]$arrCertPaths.count)    -ge 1)
              {
                Foreach ($Param In $arrCertPaths)
                {
                  $CertificateStore = $Param.Trim()
                  If (Test-Path $CertificateStore) 
                  {
                      #write-host $CertificateStore
                      $CertsAll = Get-ChildItem -path $CertificateStore -ErrorAction Stop | where-object {$_.NotAfter -gt (Get-Date)} | where-object{$_.Thumbprint -notin $ExclusionItems}
                      #Loop through the params (cert store) to verify the certificates expiring in each store
                      If ($([int]$CertsAll.count) -ge 1)  
                      {
                          get_attributes $CertsAll
                      }
                  }
                }
              }
              else
              {
                  $finalJsonOutput = "{`"windows_certStore_certificates_expiry_inDays_v2`":{`"components`":{} }}"
                  write-host $finalJsonOutput
                  Exit
              }
          }
          $cert_details = $certs | ConvertTo-Json -compress 
          #-------------------------------------------------------------------------------
          $cert_info = ""
          foreach ($certName in $cert_attributes.Keys) { 
          $cert_info += $certName + ":"+ $($cert_attributes[$certName]) + ","
          } 
          $cert_info = $cert_info.Substring(0, $cert_info.Length - 1)
           
          $finalJsonOutput = "{`"windows_certStore_certificates_expiry_inDays_v2`":{`"components`":$cert_details ,`"alertTokens`":{`"cert.info`":""$cert_info""}}}"
          write-host $finalJsonOutput
      }
   
  Catch
  {
      write-host "Exception - Windows_Certificates_Store_Expiry_InDays_v2: " $_.Exception.Message
      Exit
  }
  }

Now, we have created a monitor successfully.

Step 3- Create a Template: Refer the Document to know how to create a template.

Create the template using above created metric and monitor.

Step 5- Assign a template: Refer the Document to know how to assign a template.

Assign the template to a device and provide any custom script arguments, if applicable. You can refer to the screenshot below for reference.

Below is the final script output

{
  "windows_certStore_certificates_expiry_inDays_v2": {
    "components": {
      "Baltimore CyberTrust Root": 716,
      "ISRG Root X1": 4391,
      "Microsoft Root Certificate Authority 2010": 4410,
      "VeriSign Class 3 Public Primary Certification Authority - G5": 4799,
      "GlobalSign Root CA": 1707,

      "CLIUSR":30
    },
    "alertTokens": {
      "cert.info": "Baltimore CyberTrust Root:Issuer - Baltimore CyberTrust Root; Subject - Baltimore CyberTrust Root; Serial Number - 020000B9,ISRG Root X1:Issuer - ISRG Root X1; Subject - ISRG Root X1; Serial Number - 008210CFB0D240E3594463E0BB63828B00,Microsoft Root Certificate Authority 2010:Issuer - Microsoft Root Certificate Authority 2010; Subject - Microsoft Root Certificate Authority 2010; Serial Number - 28CC3A25BFBA44AC449A9B586B4339AA,VeriSign Class 3 Public Primary Certification Authority - G5:Issuer - VeriSign Class 3 Public Primary Certification Authority - G5; Subject - VeriSign Class 3 Public Primary Certification Authority - G5; Serial Number - 18DAD19E267DE8BB4A2158CDCC6B3B4A,GlobalSign Root CA:Issuer - GlobalSign Root CA; Subject - GlobalSign Root CA; Serial Number - 040000000001154B5AC394,CLIUSR: Issuer - CLIUSR; Subject - CLIUSR; Serial Number - 67983B6CBC96A6AE424D50017BF6ECE3"
    }
  }
}

View the Alert Tokens

To view the Alert Tokens, follow these steps:

1. Navigate to the Infrastructure > Resources section in the user interface.
2. Select the desired resource for which you want to view the details.
3. Navigate to the Details tab.
4. You should be able to see the alert token information as shown in the figure.

Exception Handling In RSE

To capture additional logs and generate alerts for script failure reasons in custom scripts, users can utilise the scriptExceptions option of RSE. This option allows for detailed logging and alert generation, helping to identify and address issues efficiently at monitor level.

The scriptExceptions option provides flexibility for users to specify whether the alert state should be critical or warning. Users can customise the alert subject and description messages, and also decide if logging or alerting (or both) are necessary.

Script Exceptions are supported only from Agent & Gateway version 14.0.0 and above. If this option is used with versions below Agent & Gateway version 14.0.0, the template will not retrieve any data from the device.

Script exceptions should adhere to the following format. Note that all keys are case-sensitive.

Format for scriptExceptions when there are exceptions in script:

{
  "scriptExceptions": {
    "subject": "No monitoring data / Unable to fetch monitoring data / Incomplete script execution",
    "description": "Failed to collect data for following metrics. \n metricName: cpu.util, FailureReason: Invalid WMI class \n metricName: memory.util, FailureReason: unable to read /proc/mem",
    "raiseAlert": true,
    "logRequired": true,
    "alertState": "warning"
  }
}

Possible values of each field:

1. subject: The user can provide any meaningful alert subject here.
2. description: This field should contain detailed information about the exception. It is recommended to include the metric name and the reason for the data collection failure.
3. raiseAlert: Set this to true if an alert for the exception is needed; otherwise, set it to false. Alert will be generated on metric rse_metric_collection_failures, if it is set true.
4. logRequired: Set this to true if a log entry for the exception is needed; otherwise, set it to false.
5. alertState: Set this to critical for a critical alert, or warning if a warning alert is sufficient.

Including the metric name in the description is recommended to identify which metric the exception belongs to.

If there are multiple exceptions in the script, consolidate all of them into a single alert description, as shown above. The scriptExceptions payload should be returned only once per script execution and only if exceptions are encountered. If there are no exceptions, return only the metric data without including the scriptExceptions payload.

Few scenarios for Using Exception Handling:

1. When the user doesn’t provide inputs as recommended.
2. When the commands or queries used in script are not available or supported on the target device.
3. When the user wants to add extra information to the logs.
4. To handle any dynamic exceptions that may occur in the script.

Linux Network Interfaces Monitoring Use-Case:

This real-time use case demonstrates the use of scriptExceptions and alertTokens options in RSE.

Step 1:

First, create metrics needed for the RSE script, here we created four metrics named as below:

• system_linux_network_interface_errorsIn
• system_linux_network_interface_errorsOut
• system_linux_network_interface_discardsIn

Step 2:

1. In the MONITOR page.
2. Under Metrics section, add the above created metrics.
3. Under script section, paste the below RSE Script.

Sample Script:

#!/bin/bash
 
scriptExceptions=""
json_output="{"
 
count=1
str_append_errors_in=""
str_append_errors_out=""
str_append_discards_in=""
 
address_json=""
exceptionMessages=""
 
# Function to handle exceptions and append messages to the variable
handle_exception() {
    local metricName="$1"
    local interface="$2"
    local attribute="$3"
     
    local exceptionMsg="$attribute attribute not available for interface $interface"
     
    # Check if the metric name is the same as the previous one
    if [ "$metricName" != "$previousMetricName" ]; then
        if [ -n "$exceptionMessages" ]; then
            exceptionMessages+=", "
        fi
        exceptionMessages+="metricName: $metricName, FailureReason: $exceptionMsg"
    else
        # Append the failure reason to the existing metric entry
        exceptionMessages+="\n $exceptionMsg"
    fi
     
    # Store the current metric name for comparison in the next iteration
    previousMetricName="$metricName"
}
 
 
# Main function
main() {
    local components_received="{\"components\":{"
    local components_transmitted="{\"components\":{"
    local components_total="{\"components\":{"
 
    interface_dirs=$(ls -I "lo" /sys/class/net/)
    # Iterate over interfaces
    for interface in $interface_dirs; do
        attributes=("address" "rx_errors" "tx_errors" "rx_dropped" "tx_dropped")
        address=$(cat "/sys/class/net/$interface/address" 2>/dev/null | tr ':' '-')
        rx_errors=$(cat "/sys/class/net/$interface/statistics/rx_errors" 2>/dev/null)
        tx_errors=$(cat "/sys/class/net/$interface/statistics/tx_errors" 2>/dev/null)
        rx_dropped=$(cat "/sys/class/net/$interface/statistics/rx_dropped" 2>/dev/null)
         
        # Iterate through the attributes and check if any are blank
        for attribute in "${attributes[@]}"; do
            if [ -z "${!attribute}" ]; then
                case "$attribute" in
                    rx_errors) handle_exception "system_linux_network_interface_errorsIn" "$interface" "$attribute" ;;
                    tx_errors) handle_exception "system_linux_network_interface_errorsOut" "$interface" "$attribute" ;;
                    rx_dropped) handle_exception "system_linux_network_interface_discardsIn" "$interface" "$attribute" ;;
                esac
            fi
        done
 
        # Append to JSON strings
        for metric in "${attributes[@]}"; do
            value="${!metric}"  # Get the value of the current metric using indirect variable expansion
 
            # Check if the value is not blank
            if [ -n "$value" ]; then
                # Determine the appropriate JSON string based on the metric
                case "$metric" in
                    rx_errors) json_string="$str_append_errors_in" ;;
                    tx_errors) json_string="$str_append_errors_out" ;;
                    rx_dropped) json_string="$str_append_discards_in" ;;
                    address) json_string="$address_json" ;;
                esac
 
                # Append the metric to the appropriate JSON string
                if [ "$metric" == "address" ]; then
                    json_string+="$interface:$value,"
                else
                    json_string+="\"$interface\":\"$value\","
                fi
                 case "$metric" in
                    rx_errors) str_append_errors_in="$json_string" ;;
                    tx_errors) str_append_errors_out="$json_string" ;;
                    rx_dropped) str_append_discards_in="$json_string" ;;
                    address) address_json="$json_string" ;;
                esac
            fi
        done
    done
     
    str_append_errors_in="${str_append_errors_in%,}"
    str_append_errors_out="${str_append_errors_out%,}"
    str_append_discards_in="${str_append_discards_in%,}"
    components_received="${components_received%,}}"
    components_transmitted="${components_transmitted%,}}"
    components_total="${components_total%,}}"
 
    # Construct the JSON output
    if [ -n "$exceptionMessages" ]; then
        json_output+="\"scriptExceptions\": {"
        json_output+="\"subject\": \"An exception has occurred. Unable to fetch the monitoring data.\","
        json_output+="\"alertState\": \"critical\","
        json_output+="\"description\": \"Failed to collect data for following metrics. \n $exceptionMessages\","
        json_output+="\"logRequired\": true,"
        json_output+="\"raiseAlert\": true"
        json_output+="},"   
    fi
 
    address_json=$(echo $address_json | sed 's/.$//')      
    if [ -n "$str_append_errors_in" ]; then
        json_output+="\"system_linux_network_interface_errorsIn\":{\"components\":{$str_append_errors_in},\"alertTokens\":{\"mac.address\":\"$address_json\"}},"
    fi
 
    if [ -n "$str_append_errors_out" ]; then
        json_output+="\"system_linux_network_interface_errorsOut\":{\"components\":{$str_append_errors_out},\"alertTokens\":{\"mac.address\":\"$address_json\"}},"
    fi
 
    if [ -n "$str_append_discards_in" ]; then
        json_output+="\"system_linux_network_interface_discardsIn\":{\"components\":{$str_append_discards_in},\"alertTokens\":{\"mac.address\":\"$address_json\"}}}"
    fi   
  
    echo "$json_output"
}
 
# Run main function
main

Step 3:

Create the template with above created metrics and monitor.

Step 4:

Assign the template to any device.

Below is the JSON Output for above Script when there are no exceptions:

{
"system_linux_network_interface_errorsIn": {
    "components": {
      "cni0": "0",
       "ens160": "0",
      "flannel.1": "0"
    },
    "alertTokens": {
      "mac.address": "cni0:42-a5-a4-fd-86-eb,ens160:00-0c-29-b7-be-c3,flannel.1:1a-3a-7b-0b-fd-e4"
    }
  },
  "system_linux_network_interface_errorsOut": {
    "components": {
      "cni0": "0",
      "ens160": "0",
      "flannel.1": "0"
    },
    "alertTokens": {
      "mac.address": "cni0:42-a5-a4-fd-86-eb,ens160:00-0c-29-b7-be-c3,flannel.1:1a-3a-7b-0b-fd-e4"
    }
  },
  "system_linux_network_interface_discardsIn": {
    "components": {
      "cni0": "0",
      "ens160": "120",
      "flannel.1": "0"
    },
    "alertTokens": {
      "mac.address": "cni0:42-a5-a4-fd-86-eb,ens160:00-0c-29-b7-be-c3,flannel.1:1a-3a-7b-0b-fd-e4"
    }
  }
}

Below is the JSON Output for above Script when exceptions occur:

{
  "scriptExceptions": {
    "subject": "An exception has occurred. Unable to fetch the monitoring data.",
    "alertState": "critical",
    "description": "Failed to collect data for following metrics. \n metricName: system_linux_network_interface_errorsIn, FailureReason: rx_errors attribute not available for interface cni0\n
     rx_errors attribute not available for interface ens160\n rx_errors attribute not available for interface flannel.1",
    "logRequired": true,
    "raiseAlert": true
  },
  "system_linux_network_interface_errorsOut": {
    "components": {
      "cni0": "0",
      "ens160": "0",
      "flannel.1": "0"
    },
    "alertTokens": {
      "mac.address": "cni0:42-a5-a4-fd-86-eb,ens160:00-0c-29-b7-be-c3,flannel.1:1a-3a-7b-0b-fd-e4"
    }
  },
  "system_linux_network_interface_discardsIn": {
    "components": {
      "cni0": "0",
      "ens160": "120",
      "flannel.1": "0"
    },
    "alertTokens": {
      "mac.address": "cni0:42-a5-a4-fd-86-eb,ens160:00-0c-29-b7-be-c3,flannel.1:1a-3a-7b-0b-fd-e4"
    }
  }
}

In the provided JSON, the scriptExceptions block indicates that an exception occurred while collecting monitoring data, for the metric system_linux_network_interface_errorsIn, the failure reason was that the rx_errors attribute was not available for the interfaces ens160 and flannel.1. This block provides details on the exception, including the subject, alert state, and a description of the failure reasons.

For the system_linux_network_interface_errorsIn metric, data for the cni0 interface was successfully collected since there was no exception for this specific interface. Since the other two metrics, system_linux_network_interface_errorsOut, system_linux_network_interface_discardsIn, did not encounter any exceptions, their data was successfully collected and printed without issues.

Below is the alert shown for device:

In Debug level logs we can see scriptExceptions as below:

Below is the screenshot of the heal alert after resolving the script exceptions:

Enumerated Mapping In RSE

This feature is particularly beneficial for the following scenarios:

1. Graph Plotting for String Metrics:

   • When a metric returns a string value, it is not possible to plot a graph using it. In such cases, Enumerated Mapping allows for the conversion of these string values into integers, enabling graph plotting and visualization.

2. State Representation Metrics:

   • For metrics that return integer values representing different states (such as online/offline), Enumerated Mapping helps map these integers to their corresponding states, making the data more readable and meaningful.

Using Enumerated Mapping, users can ensure that the metric data is more informative and easier to interpret. This feature provides clear insights into the metric states through both numerical and textual representations while fetching the latest metrics data, and in alerts and graphs.

To use the Enumerated Mapping option, users must select “Datapoint Value Conversion” as “Enumerated Map” from the dropdown menu when creating a metric.

There are two types of Enumerated Mapping supported.

Type 1: String to Integer Mapping.

Type 2: Integer to String Mapping.

Type 1: String to Integer Mapping

For state/status-related metrics, it is common to return string values. However, monitoring through graphs only supports integer values. In such cases, String to Integer Mapping is necessary.

When creating a metric, users are required to provide mappings for all possible string values of that metric.

These mappings should include the string values in the “State” column along with any corresponding unique integer representations in the “Description” column, as shown below:

These mapped values, i.e., the string value and its respective integer, can be shown in both Alerts and Graphs, or only in alerts or graphs, by selecting the appropriate “Alerts” and “Graph” checkboxes, as shown in above screenshot.

Additionally, these mappings will appear in the “Get Latest Metric Value” page against the metric value.

Type 2: Integer to String Mapping

When a metric returns direct integer values representing specific states or status, Integer to String Mapping can be used to display the exact metric state referred to by the integer.

Users need to provide mappings for all possible integer values of that metric in State column and their corresponding string representations in Description column, while creating the metric, as shown below:

These mapped values, i.e., the integer value and its respective string, can be shown in both Alerts and Graphs, or only in alerts or graphs, by selecting the appropriate “Alerts” and “Graph” checkboxes, as shown in above screenshot.

Additionally, these mappings will appear in the “Get Latest Metric Value” page against the metric value.

Note: Ensure to configure alert thresholds based on the State column values only, if alert is required.

Windows Services Monitoring Use-Case:

This real-time use-case demonstrates the use of Enumerated Map option in RSE.

Step 1:

First, create the necessary metrics for the RSE script. In this use-case, we will create a metric named System_Windows_Service_Status_Ext.

Here, since the metric returns only string values representing the service’s state, we have defined an Enumerated Map. For each possible string value, we provided the corresponding unique integer representation while creating the metric as follows:

We configured a Critical alert for the metric, using State column value, as shown below:

Step 2:

1. On the MONITOR page,
2. Under Metrics section, add the above created metrics.
3. Under script section, paste the below RSE Script. Sample Script:

$customArgs = ${custom.script.arguments}
$servicesHash = @{}
$metricName = "System_Windows_Service_Status_Ext"
$finalJsonOutput = @{$metricName = @{}}
$global:scriptExceptions = @{}
 
Function Normalize()
{
  param([String]$str)
  $str = $str.Trim()
  $str = $str.Replace('"', '\"')
  $str = $str.Replace("\", "\\")
  Return $str
}
 
If (($([Int]$customArgs.Length) -eq 0) -or ($([String]$customArgs) -eq "Custom Script Arguments") -or ($($([String]$customArgs).Trim()) -eq ""))
{
  $exceptionMsg = @{
    "subject" = "Unable to fetch the monitoring data."
    "alertState" = "critical"
    "description" = "Failed to collect data for following metrics. `n metricName: $metricName, FailureReason: empty or invalid arguments"
    "logRequired" = $TRUE
    "raiseAlert" = $TRUE
  }
  $finalJsonOutput["scriptExceptions"] = $exceptionMsg
  $exceptionJson = @{ "scriptExceptions" = $finalJsonOutput["scriptExceptions"] } | ConvertTo-Json -Compress
  Write-Host $exceptionJson
  exit
}
 
 
Try
{
  If ($finalJsonOutput["scriptExceptions"]) {
    # If script exceptions are already set, skip the main logic
    throw "Script exceptions encountered, skipping main logic."
  }
 
  If($([String]$customArgs).contains(","))
  {
    $servicenamePattern= $([String]$customArgs) -split ',' | ForEach-Object { $_.Trim() }
  }
  Else
  {
    $servicenamePattern = @($([String]$customArgs))
  }
 
  ForEach($pattern in $servicenamePattern)
  {
    If($([String]$pattern).Trim() -eq "")
    {
      continue
    }
    $serviceColItems = Get-WmiObject win32_service -namespace root\cimv2 -ErrorAction Stop | where-object{$_.Name -Match $pattern} | select Name, DisplayName, State
   
    ForEach($serviceColItem in $serviceColItems)
    {
      $componentName = Normalize "$([String]$serviceColItem.DisplayName)"
      $metricValue = $([String]$serviceColItem.State)
       
      $servicesHash[$componentName] = $metricValue  
    }
  } 
   $finalJsonOutput[$metricName]["components"] = $servicesHash
}
catch
{
  $exceptionMsg = @{
    "subject" = "Unable to fetch the monitoring data."
    "alertState" = "critical"
    "description" = "Failed to collect data for following metrics. `n metricName: $metricName, FailureReason: $($_.Exception.Message)"
    "logRequired" = $TRUE
    "raiseAlert" = $TRUE
  }
  $global:scriptExceptions += $exceptionMsg
}
 
# Check for script exceptions after the last try-catch block
if ($global:scriptExceptions.Count -gt 0) {
    $finalJsonOutput["scriptExceptions"] = $global:scriptExceptions
    $exceptionJson = $finalJsonOutput["scriptExceptions"] | ConvertTo-Json -Compress
}
 
$services_details = $finalJsonOutput | ConvertTo-Json -Compress
write-host $services_details

Step 3:

Create the template with above created metrics and monitor.

Step 4:

Assign the template to any device.

Below is the JSON output for the script without any exceptions:

{
  "System_Windows_Service_Status_Ext": {
    "components": {
      "Secondary Logon": "Stopped",
      "DNS Client": "Running",
      "Netlogon": "Running",
      "Power": "Running",
      "opsramp-agent": "Running"
    }
  }
}

Below is the JSON output for the script with exceptions:

{
  "scriptExceptions": {
    "subject": "Unable to fetch the monitoring data.",
    "alertState": "critical",
    "description": "Failed to collect data for following metrics. \n metricName: System_Windows_Service_Status_Ext, FailureReason: empty or invalid arguments",
    "logRequired": true,
    "raiseAlert": true
  }
}

Below is the screenshot demonstrating how both the string value and its corresponding integer are displayed in Metric value column, when fetching the latest metric values.

Below are screenshots of how the Enumerated Mappings are displayed in graphs and alerts:

In Graph:

In the screenshot above, we can observe that as defined in the Enumerated Map, the value 0 corresponds to the “Stopped” state of a service, and the value 1 corresponds to the “Running” state.

In Alert:

In the screenshot above, we can observe that the metric value displayed as Stopped (0), and the threshold configured as Running for Critical alert. According to the configured thresholds, if the metric value does not equal Running, a critical alert will be triggered, as shown in the screenshot above.

Because we defined 0 for the Stopped state and selected both the alerts and graph checkboxes while defining the Enumerated Map, we can see both the string value and its corresponding integer in the alert as well as in the graph.