Raspberry Pi 4にKubernetesを入れる

こちらの記事にしたがってK8sをRaspberry Piに入れようとしていたのですが、一部情報が古くなっているのでメモを残しておきます。

とくにDockerは今は非推奨となっており、cri runtimeとしてcrioを使うのが普通だそうです。こちらが非常に参考になりました、ありがとうございます。

前提

• Raspberry Pi ImagerでOSが書き込めたと仮定して進めます。
• M1 Macbook Airで動作確認しております。

以下はMaster nodeだけをとりあえず対象にして行っています。

IPの固定

sudo raspi-config nonint do_hostname raspi001
sudo raspi-config nonint do_change_timezone Asia/Tokyo
sudo raspi-config  --expand-rootfs

sudo apt-get update && sudo apt-get dist-upgrade -y
sudo apt-get update && sudo apt-get upgrade -y

cat <<EOF >> /etc/dhcpcd.conf
static ip_address=192.168.13.101/24
static routers=192.168.13.1
static domain_name_servers=192.168.13.1 8.8.8.8
EOF


sudo reboot

cgroups周りの設定

sudo sed -i 's/$/ cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory/g' /boot/cmdline.txt
sudo reboot

スワップの無効化

sudo swapoff --all
sudo systemctl stop dphys-swapfile
sudo systemctl disable dphys-swapfile
systemctl status dphys-swapfile

iptables周りの設定

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sudo apt-get install -y iptables arptables ebtables
sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
sudo update-alternatives --set arptables /usr/sbin/arptables-legacy
sudo update-alternatives --set ebtables /usr/sbin/ebtables-legacy

sudo sysctl --system

crioのインストール

まずはcrioの設定です。

cat <<EOF | sudo tee /etc/modules-load.d/crio.conf
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
sudo sysctl --system

インストールは以下のようになります。

OS=Raspbian_10
VERSION=1.20
cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /
EOF
cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.list
deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$VERSION/$OS/ /
EOF
curl -L https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/Release.key | sudo apt-key --keyring /etc/apt/trusted.gpg.d/libcontainers.gpg add -
curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | sudo apt-key --keyring /etc/apt/trusted.gpg.d/libcontainers.gpg add -
sudo apt update
sudo apt install -y cri-o cri-o-runc
sudo systemctl daemon-reload
sudo systemctl enable crio
sudo systemctl start crio

k8s install

sudo apt-get update && sudo apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
sudo apt update
sudo apt install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
cat <<EOF | sudo tee /etc/default/kubelet
KUBELET_EXTRA_ARGS=--container-runtime-endpoint='unix:///var/run/crio/crio.sock'
EOF

reloadする

systemctl daemon-reload
systemctl restart kubelet

kubeadmでクラスター初期化

sudo kubeadm init --pod-network-cidr=10.244.0.0/16
...
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.13.101:6443 --token 9zztqc.nlw3opu0n21vx31j \
	--discovery-token-ca-cert-hash sha256:45da3c0977e3108401591ff2ed23a89180a485f1ad14b7e7fd5ac8bafd82b945

コマンドで確認してみましょう。

pi@raspi001:~ $ mkdir -p $HOME/.kube
pi@raspi001:~ $ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
pi@raspi001:~ $ sudo chown $(id -u):$(id -g) $HOME/.kube/config
pi@raspi001:~ $ kubectl get node
NAME       STATUS   ROLES           AGE   VERSION
raspi001   Ready    control-plane   18m   v1.24.1

最後に

Dockerが非推奨になっていたのを知らず、無駄な時間を溶かしてしまいました。
ここらへんは移り変わりが速いので、2年後にはまた別のランタイムやらを使っている気がします、、、

おまけ：worker nodeの参加とnginxの起動

最初の名前設定とipアドレスだけ変えたコマンドが以下です。それ以外はクラスターにjoinするところまでいっしょです。

sudo raspi-config nonint do_hostname k8s-node1
sudo raspi-config nonint do_change_timezone Asia/Tokyo
sudo raspi-config  --expand-rootfs

sudo apt-get update && sudo apt-get dist-upgrade -y
sudo apt-get update && sudo apt-get upgrade -y

cat <<EOF >> /etc/dhcpcd.conf
static ip_address=192.168.13.102/24
static routers=192.168.13.1
static domain_name_servers=192.168.13.1 8.8.8.8
EOF


sudo reboot

joinするときは先ほどちらっと出てきたコマンドを叩きます。

kubeadm join 192.168.13.101:6443 --token 9zztqc.nlw3opu0n21vx31j \
	--discovery-token-ca-cert-hash sha256:45da3c0977e3108401591ff2ed23a89180a485f1ad14b7e7fd5ac8bafd82b945
[preflight] Running pre-flight checks
	[WARNING SystemVerification]: missing optional cgroups: hugetlb blkio
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

マスターノードで走っているのが確認できます。

$ kubectl get node
NAME        STATUS   ROLES           AGE     VERSION
k8s-node1   Ready    <none>          7m19s   v1.24.1
raspi001    Ready    control-plane   44m     v1.24.1

flannelもインストールしておきます。

kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp.flannel.unprivileged configured
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel unchanged
configmap/kube-flannel-cfg unchanged
daemonset.apps/kube-flannel-ds configured

以下のpodを作成してみました。

sample-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: static-web
  labels:
    role: myrole
spec:
  containers:
    - name: web
      image: nginx
      ports:
        - name: web
          containerPort: 80
          protocol: TCP

リソースを作成します。

$ kubectl create -f sample-pod.yaml

以下のように作成されました。

$ kubectl get pods
NAME         READY   STATUS    RESTARTS   AGE
static-web   1/1     Running   0          50s