Delete K8S Namespace Stuck At Terminating State

17 Jun 2019 . category: devops . Comments
#devops #kubernetes #terminating #namespace

Problem

Sometimes after deleting a namespace (for example kiali-operator), it sticks on “Terminating” state, and when you enter:

kubectl delete namespace kiali-operator

You’ll get

warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
Error from server (Conflict): Operation cannot be fulfilled on namespaces "kiali-operator": The system is ensuring all content is removed from this namespace.  Upon completion, this namespace will automatically be purged by the system.

Solution

Let’s get rid of kiali-operator namespace!

kubectl get ns kiali-operator -o json > kiali-operator.json

Edit kiali-operator.json and remove kubernetes from finalizers.

{
    "apiVersion": "v1",
    "kind": "Namespace",
    "metadata": {
        "annotations": {
            "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"kiali-operator\",\"version\":\"v0.21.0\"},\"name\":\"kiali-operator\"}}\n"
        },
        "creationTimestamp": "2019-06-17T05:03:48Z",
        "deletionTimestamp": "2019-06-17T05:11:37Z",
        "labels": {
            "app": "kiali-operator",
            "version": "v0.21.0"
        },
        "name": "kiali-operator",
        "resourceVersion": "34541023",
        "selfLink": "/api/v1/namespaces/kiali-operator",
        "uid": "4a4997ec-90bd-11e9-a3be-0619e18c0adc"
    },
    "spec": {
        "finalizers": [
            "kubernetes" <-- remove it
        ]
    },
    "status": {
        "phase": "Terminating"
    }
}

Using kubernetes proxy

kubernetes proxy

Then run this command

curl -k -H "Content-Type: application/json" -X PUT --data-binary @kiali-operator.json http://127.0.0.1:8001/api/v1/namespaces/kiali-operator/finalize

Now, check your namespaces

kubernetes get namespaces

Namespace kiali-operator has been gone!

---