This guide provides a step-by-step approach to resolving issues where K3S, Gateway, Longhorn, and related pods fail to start due to a missing sandbox image (rancher/mirrored-pause:3.6). The guide covers checking the availability of image tar files, restarting the K3S service, and verifying the images imported into the containerd runtime.

Problem

K3S/Gateway/Longhorn pods are failing to start, showing an error like:

failed to get sandbox image "rancher/mirrored-pause:3.6"

Possible Causes

This error occurs when the K3S base images are either missing or removed from the K3S cluster node and the system is unable to pull them from the internet.

Troubleshooting Steps

Ensure the user is logged in with root or sudo privileges.

Step 1: Check for Image Tar File

If the customer is using the OpsRamp provided ISO, check if the image tar file is available in the following path:

/var/lib/rancher/k3s/agent/images/

Example Output:

ls /var/lib/rancher/k3s/agent/images/
k3s-airgap-images-amd64-1.29.0.tar

Step 2: Restart K3S Service

If the image tar file is present, restart the K3S service using the following command:

systemctl restart k3s
If the image tar file is missing, please contact OpsRamp Support team.

Step 3: Verify Imported Images

After K3S restarts, it will automatically import images to the containerd.
To check the imported images, run the following command:

k3s ctr i ls -q

Example Output:

docker.io/rancher/klipper-helm:v0.8.2-build20230815
docker.io/rancher/klipper-lb:v0.4.4
docker.io/rancher/local-path-provisioner:v0.0.24
docker.io/rancher/mirrored-coredns-coredns:1.10.1
docker.io/rancher/mirrored-library-busybox:1.36.1
docker.io/rancher/mirrored-library-traefik:2.10.5
docker.io/rancher/mirrored-metrics-server:v0.6.3
docker.io/rancher/mirrored-pause:3.6