環境

Ubuntu 22.04
kubernetes v1.30.0

$ kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"30", GitVersion:"v1.30.0", GitCommit:"7c48c2bd72b9bf5c44d21d7338cc7bea77d0ad2a", GitTreeState:"clean", BuildDate:"2024-04-17T17:34:08Z", GoVersion:"go1.22.2", Compiler:"gc", Platform:"linux/amd64"}

本題

flannel をインストールした際に POD が正常に起動しなかった。

$ kubectl get po -A
NAMESPACE      NAME                           READY   STATUS              RESTARTS      AGE
kube-flannel   kube-flannel-ds-njbp7          0/1     CrashLoopBackOff    3 (16s ago)   65s
kube-system    coredns-7db6d8ff4d-5xtdq       0/1     ContainerCreating   0             16m
kube-system    coredns-7db6d8ff4d-8xqhj       0/1     ContainerCreating   0             16m
kube-system    etcd-lime                      1/1     Running             1             16m
kube-system    kube-apiserver-lime            1/1     Running             1             16m
kube-system    kube-controller-manager-lime   1/1     Running             1             16m
kube-system    kube-proxy-fgskc               1/1     Running             0             16m
kube-system    kube-scheduler-lime            1/1     Running             1             16m

ログを見てみるとこんなエラーが

Error registering network: failed to acquire lease: subnet "10.244.0.0/16" specified in the flannel net config doesn't contain "192.168.0.0/24" PodCIDR of the "lime" node

確かに node の PodCIDR が192.168.0.0./24になっていた。

$ kubectl describe node lime
Name:               lime
Roles:              control-plane
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=lime
                    kubernetes.io/os=linux
                    node-role.kubernetes.io/control-plane=
                    node.kubernetes.io/exclude-from-external-load-balancers=
Annotations:        flannel.alpha.coreos.com/backend-data: {"VNI":1,"VtepMAC":"8a:db:a8:98:d0:22"}
                    flannel.alpha.coreos.com/backend-type: vxlan
                    flannel.alpha.coreos.com/kube-subnet-manager: true
                    flannel.alpha.coreos.com/public-ip: 133.18.202.53
                    kubeadm.alpha.kubernetes.io/cri-socket: unix:///run/containerd/containerd.sock
                    node.alpha.kubernetes.io/ttl: 0
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Wed, 24 Apr 2024 21:23:26 +0900
Taints:             <none>
Unschedulable:      false
Lease:
  HolderIdentity:  lime
  AcquireTime:     <unset>
  RenewTime:       Wed, 24 Apr 2024 21:45:27 +0900
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Wed, 24 Apr 2024 21:41:41 +0900   Wed, 24 Apr 2024 21:23:24 +0900   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Wed, 24 Apr 2024 21:41:41 +0900   Wed, 24 Apr 2024 21:23:24 +0900   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Wed, 24 Apr 2024 21:41:41 +0900   Wed, 24 Apr 2024 21:23:24 +0900   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Wed, 24 Apr 2024 21:41:41 +0900   Wed, 24 Apr 2024 21:36:15 +0900   KubeletReady                 kubelet is posting ready status
Addresses:
  InternalIP:  10.150.11.1
  Hostname:    lime
Capacity:
  cpu:                2
  ephemeral-storage:  25625852Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             2005932Ki
  pods:               110
Allocatable:
  cpu:                2
  ephemeral-storage:  23616785165
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             1903532Ki
  pods:               110
System Info:
  Machine ID:                 55bf6f37f4724e4aa9d8c1e2e0dfdc7e
  System UUID:                55bf6f37-f472-4e4a-a9d8-c1e2e0dfdc7e
  Boot ID:                    1197fc44-f017-4a7e-b35e-719a87801b27
  Kernel Version:             6.5.0-28-generic
  OS Image:                   Ubuntu 22.04.4 LTS
  Operating System:           linux
  Architecture:               amd64
  Container Runtime Version:  containerd://1.6.31
  Kubelet Version:            v1.30.0
  Kube-Proxy Version:         v1.30.0
PodCIDR:                      192.168.0.0/24
PodCIDRs:                     192.168.0.0/24
Non-terminated Pods:          (9 in total)
  Namespace                   Name                            CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
  ---------                   ----                            ------------  ----------  ---------------  -------------  ---
  kube-flannel                kube-flannel-ds-njbp7           100m (5%)     0 (0%)      50Mi (2%)        0 (0%)         6m55s
  kube-system                 coredns-7db6d8ff4d-5xtdq        100m (5%)     0 (0%)      70Mi (3%)        170Mi (9%)     21m
  kube-system                 coredns-7db6d8ff4d-8xqhj        100m (5%)     0 (0%)      70Mi (3%)        170Mi (9%)     21m
  kube-system                 etcd-lime                       100m (5%)     0 (0%)      100Mi (5%)       0 (0%)         22m
  kube-system                 kube-apiserver-lime             250m (12%)    0 (0%)      0 (0%)           0 (0%)         22m
  kube-system                 kube-controller-manager-lime    200m (10%)    0 (0%)      0 (0%)           0 (0%)         22m
  kube-system                 kube-proxy-fgskc                0 (0%)        0 (0%)      0 (0%)           0 (0%)         21m
  kube-system                 kube-scheduler-lime             100m (5%)     0 (0%)      0 (0%)           0 (0%)         22m
  velero                      velero-upgrade-crds-b4kg7       0 (0%)        0 (0%)      0 (0%)           0 (0%)         16m
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests     Limits
  --------           --------     ------
  cpu                950m (47%)   0 (0%)
  memory             290Mi (15%)  340Mi (18%)
  ephemeral-storage  0 (0%)       0 (0%)
  hugepages-1Gi      0 (0%)       0 (0%)
  hugepages-2Mi      0 (0%)       0 (0%)
Events:
  Type     Reason                   Age    From             Message
  ----     ------                   ----   ----             -------
  Normal   Starting                 21m    kube-proxy
  Normal   Starting                 22m    kubelet          Starting kubelet.
  Warning  InvalidDiskCapacity      22m    kubelet          invalid capacity 0 on image filesystem
  Normal   NodeHasSufficientMemory  22m    kubelet          Node lime status is now: NodeHasSufficientMemory
  Normal   NodeHasNoDiskPressure    22m    kubelet          Node lime status is now: NodeHasNoDiskPressure
  Normal   NodeHasSufficientPID     22m    kubelet          Node lime status is now: NodeHasSufficientPID
  Normal   NodeAllocatableEnforced  22m    kubelet          Updated Node Allocatable limit across pods
  Normal   RegisteredNode           21m    node-controller  Node lime event: Registered Node lime in Controller
  Normal   NodeReady                9m18s  kubelet          Node lime status is now: NodeReady

こうなる心当たりはあって、kubeadm-config.yamlのpodSubnetを192.168.0.0/16にしていた。

apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
nodeRegistration:
  criSocket: unix:///run/containerd/containerd.sock
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: "ipvs"
ipvs:
  strictARP: true # MetalLBに必要
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
networking:
  podSubnet: "192.168.0.0/16"

そこで、 podSubnet を10.244.0.0/16にしたら無事起動しました！