前提
EC2でAmazonLinuxインスタンスを作成してあること（CPUとメモリは最低でも2以上を要求されるのでt2.microではダメ）
背景
当たり前のようにkubernete環境構築で詰まったので手順をメモ
環境
2020/3月時点でのバージョン
[root@kube-master work]# kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T21:03:42Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T20:55:23Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
事前準備
- host名の設定/hostsファイル編集
- Dockerのインストール
- swapの無効化
- SELinuxを止める（本番環境ではやらないほうがいいらしい）
- iptables編集
- Dockerのkubelet向けcgroup設定
host名の設定
# いちおうrootユーザーになっとく
[ec2-user@ip-10-0-10-176 ~]$ sudo su -
[root@kube-master ~]# hostnamectl set-hostname kube-master
[root@kube-master ~]# echo "[your aws ec2 public ipaddress] kube-master" >> /etc/hosts
[root@kube-master ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost6 localhost6.localdomain6
3.112.45.181 kube-master
Dockerインストール
[root@kube-master ~]# yum update -y
[root@kube-master ~]# yum install -y docker
[root@kube-master ~]# systemctl enable docker && systemctl start docker
インストールの確認
[root@kube-master ~]# docker info | grep -i version
Server Version: 18.09.9-ce
containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Kernel Version: 4.14.171-136.231.amzn2.x86_64
[root@kube-master ~]# docker info | grep -i driver
Storage Driver: overlay2
Logging Driver: json-file
Cgroup Driver: cgroupfs # あとでここ変える
swapの無効化
swap無効化し、swap領域が使われていないことを確認する。
公式サイトにswap無効化してくださいと書かれている（Swap disabled. You MUST disable swap in order for the kubelet to work properly.）
[root@kube-master ~]# swapoff -a
[ec2-user@ip-10-0-10-176 ~]$ free
total used free shared buff/cache available
Mem: 8166360 224408 7181288 484 760664 7695804
Swap: 0 0 0
SELinuxを止める
以下の理由により
Setting SELinux in permissive mode by running setenforce 0 and sed ... effectively disables it. This is required to allow containers to access the host filesystem, which is needed by pod networks for example. You have to do this until SELinux support is improved in the kubelet.
[ec2-user@ip-10-0-10-176 ~]$ setenforce 0
setenforce: SELinux is disabled
[root@kube-master ~]# sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
sysctlでネットワークをブリッジできるようする
Some users on RHEL/CentOS 7 have reported issues with traffic being routed incorrectly due to iptables being bypassed. You should ensure net.bridge.bridge-nf-call-iptables is set to 1 in your sysctl config, e.g.
[root@kube-master ~]# cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
Dockerのkubelet向けcgroup設定
DockerのcgroupDriverをsystemdに設定する
[root@kube-master ~]# cat > /etc/docker/daemon.json <<EOF
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
]
}
EOF
[root@kube-master ~]# mkdir -p /etc/systemd/system/docker.service.d
[root@kube-master ~]# systemctl daemon-reload
[root@kube-master ~]# systemctl restart docker
[root@kube-master ~]# docker info | grep -i driver
Storage Driver: overlay2
Logging Driver: json-file
Cgroup Driver: systemd
kubernetesインストール
- Kubernetesのリポジトリ登録
- kubelet/kubeadm/kubectlインストール
- kubeadm init
- CNI構築
- all-in-one化
Kubernetesのリポジトリ登録
公式のものを参考に
[root@kube-master ~]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
kubelet/kubeadm/kubectlインストール
[root@kube-master ~]# yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
[root@kube-master ~]# systemctl enable kubelet && systemctl start kubelet
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
kubeadm実行
公式サイトを参考に実行していく
CNIにはCalicoを選択
[root@kube-master ~]# kubeadm init --pod-network-cidr=192.168.0.0/16
initが成功したらkubeadmの指示に従う
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
[root@kube-master ~]# mkdir -p $HOME/.kube
[root@kube-master ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@kube-master ~]# chown $(id -u):$(id -g) $HOME/.kube/config
※ kubeadm join --token ~~はall-in-one構成では必要ないらしいのでやらない
calicoネットワーク構築
[root@kube-master ~]# kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml
kubernetesのall-in-one構築
kubectl taint nodes --all node-role.kubernetes.io/master-
動作確認
[root@kube-master ~]# kubectl get all --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/calico-kube-controllers-5c45f5bd9f-74f4h 1/1 Running 0 4m32s
kube-system pod/calico-node-mmvqh 1/1 Running 0 4m33s
kube-system pod/coredns-6955765f44-45mkb 1/1 Running 0 16m
kube-system pod/coredns-6955765f44-xntv5 1/1 Running 0 16m
kube-system pod/etcd-kube-master 1/1 Running 0 16m
kube-system pod/kube-apiserver-kube-master 1/1 Running 0 16m
kube-system pod/kube-controller-manager-kube-master 1/1 Running 0 16m
kube-system pod/kube-proxy-d2qmc 1/1 Running 0 16m
kube-system pod/kube-scheduler-kube-master 1/1 Running 0 16m
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 16m
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 16m
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system daemonset.apps/calico-node 1 1 1 1 1 beta.kubernetes.io/os=linux 4m34s
kube-system daemonset.apps/kube-proxy 1 1 1 1 1 beta.kubernetes.io/os=linux 16m
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system deployment.apps/calico-kube-controllers 1/1 1 1 4m34s
kube-system deployment.apps/coredns 2/2 2 2 16m
NAMESPACE NAME DESIRED CURRENT READY AGE
kube-system replicaset.apps/calico-kube-controllers-5c45f5bd9f 1 1 1 4m34s
kube-system replicaset.apps/coredns-6955765f44 2 2 2 16m