The Kubernetes controller manager is a daemon that embeds the core control loops shipped with Kubernetes. This controller Pod exposes its metrics to analyze. A controller is a control loop that watches the shared state of the cluster through the Api Server and makes changes attempting to move the current state towards the state. Examples of controllers that ship with Kubernetes today are the replication controller, endpoints controller, namespace controller, and serviceaccounts controller.
Collector Type: Agent
Category: Application Monitors
Application Name: k8s Controller
Global Template Name: Kubernetes Controller
Validate
Go to Resources under the Infrastructure tab to check if your resources are on boarded and the metrics are collected.
Collected Metrics
| Metric Name | Display Name | Description |
|---|---|---|
| controller.workqueue.work_duration.sum | Kube Controller Workqueue Work Duration Seconds Sum | How long in seconds processing an item from workqueue takes. |
| controller.workqueue.work_duration.count | Kube Controller Workqueue Work Duration Seconds Count | Total time in seconds processing an item from workqueue takes. |
| controller.workqueue.work_unfinished_duration | Kube Controller Workqueue Unfinished Work Seconds | How many seconds of work has done that is in progress and hasn't been observed by work_duration. Large values indicate stuck threads. |
| controller.workqueue.work_longest_duration | Kube Controller Workqueue Longest Running Processor Seconds | How many seconds has the longest running processor for workqueue been running. |
| controller.workqueue.queue_duration.sum | Kube Controller Workqueue Queue Duration Seconds Sum | How long in seconds an item stays in workqueue before being requested. |
| controller.workqueue.queue_duration.count | Kube Controller Workqueue Queue Duration Seconds Count | Total how long in seconds an item stays in workqueue before being requested. |
| controller.workqueue.nodes.count | Kube Controller Registered Nodes | Number of registered Nodes per zone. |
| controller.workqueue.nodes.unhealthy | Kube Controller Node Collector Unhealthy Nodes in Zone | Number of not Ready Nodes per zone. |
| controller.workqueue.nodes.evictions | Kube Controller Node Collector Evictions Number | Number of Node evictions that happened since the current instance of NodeController started. |
| controller.workqueue.depth | Kube Controller Workqueue Depth | Current depth of workqueue. |
| controller.workqueue.adds | Kube Controller Workqueue Adds Total | Total number of adds handled by workqueue. |
| controller.workqueue.retries | Kube Controller Workqueue Retries Total | Total number of retries handled by workqueue. |
| controller.rate_limiter.use | Kube Controller Node Lifecycle Controller Rate Limiter Use | A metric measuring the saturation of the rate limiter for node_lifecycle_controller. |
| controller.go.goroutines | Kube Controller Go Goroutines | Number of goroutines that currently exist. |
| controller.threads | Kube Controller Os Threads | Number of OS threads created. |
| controller.process.max_fds | Kube Controller Process Max Fds | Maximum number of open file descriptors. |
| controller.process.open_fds | Kube Controller Process Open Fds | Number of open file descriptors. |
Note
If the Kubernetes Controller Manager template does not work, check if the manager service is running on all interfaces or only on the loopback interface. If it is running on the loopback interface, change it to bind to all interfaces by editing the/etc/kubernetes/manifests/kube-controller-manager.yaml file on the master node and ensure that the –bind-address parameter is set to 0.0.0.0.