kubernetesでnamespaceが消えない場合の対処法

kubernetesを触っていると稀にnamespaceが削除されずに残り続けてしまう場面に遭遇します。具体的には以下のような状態がずっと続きます。

$ kubectl get namespaces
NAME              STATUS        AGE
argocd            Terminating   81m
default           Active        84m
kube-node-lease   Active        84m
kube-public       Active        84m
kube-system       Active        84m

この時はargocdがどれだけ待ってもTerminatingでした。namespaceはコントロールプレーンで管理している情報なのでワーカーノードの再起動などでは解決せず困っていたのですが、以下英語の記事に解決法が記載されていたので備忘録として残しておきます。

1. namespaceの情報をjsonファイルに出力

$ kubectl get ns argocd -o json > argocd.json

出力したファイルの内容は以下のようになっています。

$ cat argocd.json
{
    "apiVersion": "v1",
    "kind": "Namespace",
    "metadata": {
        "creationTimestamp": "2021-04-08T15:09:11Z",
        "deletionTimestamp": "2021-04-08T16:14:47Z",
        "managedFields": [
            {
                "apiVersion": "v1",
                "fieldsType": "FieldsV1",
                "fieldsV1": {
                    "f:status": {
                        "f:phase": {}
                    }
                },
                "manager": "kubectl",
                "operation": "Update",
                "time": "2021-04-08T15:09:11Z"
            },
            {
                "apiVersion": "v1",
                "fieldsType": "FieldsV1",
                "fieldsV1": {
                    "f:status": {
                        "f:conditions": {
                            ".": {},
                            "k:{\"type\":\"NamespaceContentRemaining\"}": {
                                ".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}
                            },
                            "k:{\"type\":\"NamespaceDeletionContentFailure\"}": {
                                ".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}
                            },
                            "k:{\"type\":\"NamespaceDeletionDiscoveryFailure\"}": {
                                ".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}
                            },
                            "k:{\"type\":\"NamespaceDeletionGroupVersionParsingFailure\"}": {
                                ".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}
                            },
                            "k:{\"type\":\"NamespaceFinalizersRemaining\"}": {
                                ".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}
                            }
                        }
                    }
                },
                "manager": "kube-controller-manager",
                "operation": "Update",
                "time": "2021-04-08T16:14:52Z"
            }
        ],
        "name": "argocd",
        "resourceVersion": "25625",
        "selfLink": "/api/v1/namespaces/argocd",
        "uid": "af8b4e6d-c38e-438f-b342-0d2cc1cfc70d"
    },
    "spec": {
        "finalizers": [
            "kubernetes"
        ]
    },
    "status": {
        "conditions": [
            {
                "lastTransitionTime": "2021-04-08T16:14:52Z",
                "message": "All resources successfully discovered",
                "reason": "ResourcesDiscovered",
                "status": "False",
                "type": "NamespaceDeletionDiscoveryFailure"
            },
            {
                "lastTransitionTime": "2021-04-08T16:14:52Z",
                "message": "All legacy kube types successfully parsed",
                "reason": "ParsedGroupVersions",
                "status": "False",
                "type": "NamespaceDeletionGroupVersionParsingFailure"
            },
            {
                "lastTransitionTime": "2021-04-08T16:14:52Z",
                "message": "All content successfully deleted, may be waiting on finalization",
                "reason": "ContentDeleted",
                "status": "False",
                "type": "NamespaceDeletionContentFailure"
            },
            {
                "lastTransitionTime": "2021-04-08T16:14:52Z",
                "message": "Some resources are remaining: applications.argoproj.io has 2 resource instances",
                "reason": "SomeResourcesRemain",
                "status": "True",
                "type": "NamespaceContentRemaining"
            },
            {
                "lastTransitionTime": "2021-04-08T16:14:52Z",
                "message": "Some content in the namespace has finalizers remaining: resources-finalizer.argocd.argoproj.io in 2 resource instances",
                "reason": "SomeFinalizersRemain",
                "status": "True",
                "type": "NamespaceFinalizersRemaining"
            }
        ],
        "phase": "Terminating"
        ]
    }
}

2. jsonファイルの中身を変更

以下のようにspec.finalizersの配列の中身を空に変更してください。

{
    ...
    "spec": {
        "finalizers": []
    },
    ...
}

3. kubectlで変更を反映

後は変更したjsonファイルを以下のコマンドで反映させるとnamespaceが削除されます。

$ kubectl replace --raw "/api/v1/namespaces/argocd/finalize" -f ./argocd.json

$ kubectl get namespaces
NAME              STATUS   AGE
default           Active   117m
kube-node-lease   Active   117m
kube-public       Active   117m
kube-system       Active   117m