はじめに
kubernetesの実験環境の構築に何度も苦しめられましたが、k0sにより自由に構築と破棄ができる体制が整ったので備忘録として記載します。
環境
- buildserver
- Hardware : Raspberry pi 3B+
- OS : Raspbian GNU/Linux 11 (bullseye)
- Memory : 1 GB
- SD card : 16 GB
- masternode
- Hardware : ESPRIMO Q7010/E
- OS : Ubuntu 22.04
- Memory : 16 GB
- SSD : 256 GB
- workernode01
- Hardware : ThinkCentre m720q
- OS : Ubuntu 22.04
- Memory : 8 GB
- SSD : 128 GB
環境構成図は下記の通り
masternodeとworkernode01はWindows11とUbuntu 22.04のデュアルブートで構築しているため、実際にk8s関連で利用できるディスク領域は少ない状態で実施した。
構築手順
構築利用パッケージの導入/必要設定の投入
buildserver, masternode, workernode01に下記の設定やパッケージ等が入っていることを確認すること。
vimの導入は自由に。
# apt install vim
# apt install ssh
# passwd root
設定ファイル変更
下記buildserverの設定を変更。
# vim /etc/hosts
127.0.1.1 buildserver
192.168.0.11 masternode
192.168.0.12 workernode01
SSH設定&鍵生成
buildserverでrootログインをyesに変更。公開鍵の転送に必要
# vim /etc/ssh/sshd_config
PermitRootLogin yes
次にbuildserverで鍵を生成する。
鍵生成の途中の入力はすべて空エンターにしている。
# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa
Your public key has been saved in /root/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX root@buildserver
The key's randomart image is:
+---[RSA 3072]----+
（省略）
+----[SHA256]-----+
buidserverで下記手順を実施し、k8sを展開するノードたち（MasternodeやWorkernode）に生成した鍵を転送する。
転送した後、鍵でちゃんとSSH接続できるかを確認（パスワード入力なくてもログインできるようになっていること）
# ssh-copy-id root@masternode
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@masternode's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'root@masternode'"
and check to make sure that only the key(s) you wanted were added.
# ssh-copy-id root@workernode01
（省略）
# ssh root@masternode
Welcome to Ubuntu 22.04.3 LTS (GNU/Linux 6.2.0-39-generic x86_64)
（省略）
root@masternode:~#
root@masternode:~# exit
この段階でおそらくsshdのPermitRootLoginをprohibit-passwordに変えても問題なさそうだが未検証
k0sの導入
下記の手順通りに導入
# VER=$(curl -s https://api.github.com/repos/k0sproject/k0sctl/releases/latest|grep tag_name | cut -d '"' -f 4)
# echo $VER
v0.16.0
# wget https://github.com/k0sproject/k0sctl/releases/download/${VER}/k0sctl-linux-arm -O k0sctl
（省略）
k0sctl 100%[===================>] 15.81M 6.82MB/s in 2.3s
2023-12-11 20:20:20 (6.82 MB/s) - ‘k0sctl’ saved [16580608/16580608]
# chmod +x k0sctl
# cp -r k0sctl /usr/local/bin/ /bin
# k0sctl version
version: v0.16.0
commit: 7e8c272
次にks0がk8s環境を構築する際に参照する設定ファイルを生成。
# cat k0sctl.yaml
apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
name: k0s-cluster
spec:
hosts:
- ssh:
address: masternode
user: root
port: 22
keyPath: ~/.ssh/id_rsa
role: controller+worker
- ssh:
address: workernode01
user: root
port: 22
keyPath: ~/.ssh/id_rsa
role: worker
上記設定ファイルをもとに構築開始。
1分くらいで完了していたので、シンプルな構成であればかなり早く構築できる。
# k0sctl apply --config k0sctl.yaml
?????????????????????????????????????????????? ????????? ???
???????????????????????????????????????? ??? ???
???????????????????????????????????????? ??? ???
???????????????????????????????????????? ??? ???
?????????????????????????????????????????????? ??? ??????????
k0sctl v0.16.0 Copyright 2023, k0sctl authors.
Anonymized telemetry of usage will be sent to the authors.
By continuing to use k0sctl you agree to these terms:
https://k0sproject.io/licenses/eula
INFO ==> Running phase: Connect to hosts
INFO [ssh] workernode01:22: connected
INFO [ssh] masternode:22: connected
INFO ==> Running phase: Detect host operating systems
INFO [ssh] workernode01:22: is running Ubuntu 22.04.3 LTS
INFO [ssh] masternode:22: is running Ubuntu 22.04.3 LTS
INFO ==> Running phase: Acquire exclusive host lock
INFO ==> Running phase: Prepare hosts
INFO ==> Running phase: Gather host facts
INFO [ssh] masternode:22: using masternode as hostname
INFO [ssh] workernode01:22: using workernode01 as hostname
INFO [ssh] masternode:22: discovered enx04ab182bc1b9 as private interface
INFO [ssh] workernode01:22: discovered eno1 as private interface
INFO [ssh] masternode:22: discovered 192.168.0.11 as private address
INFO [ssh] workernode01:22: discovered 192.168.0.12 as private address
INFO ==> Running phase: Validate hosts
INFO ==> Running phase: Gather k0s facts
INFO ==> Running phase: Validate facts
INFO ==> Running phase: Configure k0s
WARN [ssh] masternode:22: generating default configuration
INFO [ssh] masternode:22: validating configuration
INFO [ssh] masternode:22: configuration was changed, installing new configuration
INFO ==> Running phase: Initialize the k0s cluster
INFO [ssh] masternode:22: installing k0s controller
INFO [ssh] masternode:22: waiting for the k0s service to start
INFO [ssh] masternode:22: waiting for kubernetes api to respond
INFO ==> Running phase: Install workers
INFO [ssh] workernode01:22: validating api connection to https://192.168.0.11:6443
INFO [ssh] masternode:22: generating token
INFO [ssh] workernode01:22: writing join token
INFO [ssh] workernode01:22: installing k0s worker
INFO [ssh] workernode01:22: starting service
INFO [ssh] workernode01:22: waiting for node to become ready
INFO ==> Running phase: Release exclusive host lock
INFO ==> Running phase: Disconnect from hosts
INFO ==> Finished in 52s
INFO k0s cluster version v1.28.4+k0s.0 is now installed
masternodeに接続し、下記コマンドを入力し動作確認。
kubectlコマンドを利用する際はk0s kubectlをつけて実行する。
# k0s kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-85df575cdb-ctz26 1/1 Running 0 1h
coredns-85df575cdb-k8k2c 1/1 Running 0 1h
konnectivity-agent-2f4ts 1/1 Running 0 1h
konnectivity-agent-zvm97 1/1 Running 0 1h
kube-proxy-czvwf 1/1 Running 0 1h
kube-proxy-j9b8v 1/1 Running 0 1h
kube-router-2fd5g 1/1 Running 0 1h
kube-router-scmfv 1/1 Running 0 1h
metrics-server-7556957bb7-lclh8 1/1 Running 0 1h
環境の破棄
下記コマンドで環境破棄できる。
なお、今回の環境では15秒程度でリセットできた。
# k0sctl reset
k0sctl v0.16.0 Copyright 2023, k0sctl authors.
Anonymized telemetry of usage will be sent to the authors.
By continuing to use k0sctl you agree to these terms:
https://k0sproject.io/licenses/eula
? Going to reset all of the hosts, which will destroy all configuration and data, Are you sure? Yes
INFO ==> Running phase: Connect to hosts
INFO [ssh] masternode:22: connected
INFO [ssh] workernode01:22: connected
（省略）
INFO ==> Running phase: Reset workers
INFO [ssh] workernode01:22: reset
INFO ==> Running phase: Reset controllers
INFO [ssh] masternode:22: reset
INFO ==> Running phase: Reset leader
INFO [ssh] masternode:22: reset
INFO ==> Running phase: Release exclusive host lock
INFO ==> Running phase: Disconnect from hosts
INFO ==> Finished in 15s
設定ファイルの変更箇所にもよると思うが、reset前にk0sctl.yamlの内容を変更すると、
FATA reset failed - log file saved to /root/.cache/k0sctl/k0sctl.log: failed on 1 hosts:
- [ssh] masternode:22: [ssh] masternode:22: is configured as k0s controller+worker but is already running as controller - role change is not supported
というエラーを吐き出したので、きちんとresetした後に変更すると安全。
最後に
短時間で構築と破棄ができそうなので、
k0sctl.yamlを色々変更してmasterとworkerの構成を変えたり、追加機能の導入（calicoなど）が気軽に試せると思う。
本当はworkernode02にRaspberry piを指定して構築したかったがエラー吐き出して失敗した。
Raspberry piだとパッケージの追加や何かしら設定をいじる必要があるような記述があったので（ここやここにそれらしき記載あり）、それを試してみたい。
エラー対処
k0s doesn't appear to be running but has been installed as a service
applyの途中で失敗した後、resetを実行した際にbuildserverにて下記エラーが発生した。
FATA reset failed - log file saved to /root/.cache/k0sctl/k0sctl.log: failed on 1 hosts:
- [ssh] workernode01:22: k0s doesn't appear to be running but has been installed as a service at /etc/systemd/system/k0sworker.service - please remove it or start the service
エラー文の通り、対象ノードworkernode01に入り、下記コマンドを実行しresetを再度実行、解消された。
# systemctl stop k0sworker
# systemctl disable k0sworker
なお、masternodeで発生したケースでは下記のようなエラー文が出力される。
その際はsystemctl stop k0scontrollerとsystemctl disable k0scontrollerで解消される。
FATA apply failed - log file saved to /root/.cache/k0sctl/k0sctl.log: failed on 1 hosts:
- [ssh] masternode:22: k0s doesn't appear to be running but has been installed as a service at /etc/systemd/system/k0scontroller.service - please remove it or start the service