LoginSignup
1
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

@ishida0503

kubernetesのWORKERノードをWindows Serverで構築

Last updated at Posted at 2020-03-30

何となく、構築してみたかっただけです。
適当です。

2020/03/30 00:50現在
なんかnssmのサイト落ちてる？

0 構築環境

・ノード構成

ノード種類 ホスト名 OS IPアドレス サブネットマスク 備考
MASTER VM200001 CentOS 7.7 192.168.1.221 24
WORKER VM200002 Windows Server 2019 Standard 192.168.1.222 24 ServerCore、英語版、評価版

・kubernetesネットワーク

ネットワーク種類 ネットワーク サブネットマスク
POD CIDER 10.244.0.0 16
SERVICE CIDER 10.0.0.0 16

・hosts
/etc/hosts,C:\Windows\System32\drivers\etc\hostsに以下を登録

192.168.1.221 VM200001
192.168.1.222 VM200002

1 MASTERノードの構築

1.1 SELinux無効化

# sed -i -e 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
# setenforce 0

1.2 SWAP無効化

# sed -i -e '/swap/d' /etc/fstab
# swapoff -a

1.3 カーネルパラメタ設定

# cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

# sysctl --system

1.4 FireWall無効化

# systemctl disable firewalld
# systemctl stop firewalld

1.5 yumリポジトリ登録

$ cat > /etc/yum.repos.d/add.repo << EOF
[docker-ce-stable]
name=Docker CE Stable
baseurl=https://download.docker.com/linux/centos/7/x86_64/stable
enabled=1
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg

[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

1.6 Docker、kubernetesのインストール

# yum install yum-utils device-mapper-persistent-data lvm2 nfs-utils rpcbind wget -y
# yum install docker-ce kubelet kubeadm kubectl -y

1.7 DockerのCgroup設定

# mkdir /etc/docker/
# echo {\"exec-opts\":[\"native.cgroupdriver=systemd\"]} > /etc/docker/daemon.json

1.8 各種サービス起動設定

# systemctl enable docker
# systemctl restart docker

# systemctl enable kubelet
# systemctl restart kubelet

1.9 初期化

# kubeadm init --pod-network-cidr=10.244.0.0/16 --service-cidr=10.0.0.0/16

1.10 MASTERノードでもPODが動くようにする。

※今回はLinuxのWORKERノードは作らない

# kubectl taint nodes --all node-role.kubernetes.io/master-

1.11 CNI(flannel)のマニュフェストの修正

# wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
# vi kube-flannel.yml

・修正箇所1

  cni-conf.json: |
    {
      "name": "cbr0",
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "flannel",
          "delegate": {
            "hairpinMode": true,
            "isDefaultGateway": true
          }
        },
        {
          "type": "portmap",
          "capabilities": {
            "portMappings": true
          }
        }
      ]
    }

      	↓

  cni-conf.json: |
    {
      "name": "vxlan0", ★変更箇所
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "flannel",
          "delegate": {
            "hairpinMode": true,
            "isDefaultGateway": true
          }
        },
        {
          "type": "portmap",
          "capabilities": {
            "portMappings": true
          }
        }
      ]
    }

・修正箇所2

  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan"
      }
    }

      	↓

  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan", ★変更箇所
        "VNI" : 4096,    ★変更箇所
        "Port": 4789     ★変更箇所
      }
    }

・修正箇所3

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-flannel-ds-amd64
  namespace: kube-system
  labels:
    tier: node
    app: flannel
spec:
  selector:
    matchLabels:
      app: flannel
  template:
    metadata:
      labels:
        tier: node
        app: flannel
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:


      	↓

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-flannel-ds-amd64
  namespace: kube-system
  labels:
    tier: node
    app: flannel
spec:
  selector:
    matchLabels:
      app: flannel
  template:
    metadata:
      labels:
        tier: node
        app: flannel
    spec:
      nodeSelector:                 ★追加箇所
        kubernetes.io/os: linux     ★追加箇所
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:

1.12 CNI(flannel)のインストール

# kubectl apply -f kube-flannel.yml

1.13 CNI(flannel)の不要なDaemonSetを削除

# kubectl -n kube-system delete ds \
kube-flannel-ds-arm \
kube-flannel-ds-arm64 \
kube-flannel-ds-ppc64le \
kube-flannel-ds-s390x

2 Windows WORKERノードの構築

※PowerShellで実施する。

2.1 インストールディレクトリ作成

mkdir c:\k

2.2 バイナリプログラムおよびインストールスクリプトを設置

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
wget https://storage.googleapis.com/kubernetes-release/release/v1.18.0/bin/windows/amd64/kubectl.exe -o c:\k\kubectl.exe
wget https://storage.googleapis.com/kubernetes-release/release/v1.18.0/bin/windows/amd64/kubelet.exe -o c:\k\kubelet.exe
wget https://storage.googleapis.com/kubernetes-release/release/v1.18.0/bin/windows/amd64/kube-proxy.exe -o c:\k\kube-proxy.exe
wget https://github.com/coreos/flannel/releases/download/v0.12.0/flanneld.exe -o c:\k\flanneld.exe
wget https://github.com/Microsoft/SDN/raw/master/Kubernetes/flannel/start.ps1 -o c:\k\start.ps1
wget https://raw.githubusercontent.com/microsoft/SDN/master/Kubernetes/flannel/register-svc.ps1 -o c:\k\register-svc.ps1
wget https://nssm.cc/release/nssm-2.24.zip -o c:\k\nssm-2.24.zip
Expand-Archive -Path c:\k\nssm-2.24.zip -DestinationPath c:\k
copy c:\k\nssm-2.24\win64\nssm.exe c:\k

2.3 configの設置

MASTERノードの/etc/kubernetes/admin.confをc:\k\の下にconfigという名前でコピーする。

2.4 環境変数(PATHとKUBECONFIG)の設定

[Environment]::SetEnvironmentVariable("Path", $env:Path + ";C:\k", [EnvironmentVariableTarget]::Machine)
[Environment]::SetEnvironmentVariable("KUBECONFIG","C:\k\config", [EnvironmentVariableTarget]::User)

2.5 Dockerのインストール

Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
Install-Package -Name Docker -ProviderName DockerMsftProvider

2.6 再起動

Restart-Computer -Force

2.7 WORKERとして参加

cd c:\k
.\start.ps1 -ManagementIP <WindowsノードIP>  -NetworkMode overlay -ClusterCIDR <POD CIDR> -ServiceCIDR <Service Cider> -KubeDnsServiceIP <kube-DNSのサービスIP> -LogDir c:\k

・例

cd c:\k
.\start.ps1 -ManagementIP 192.168.1.222  -NetworkMode overlay -ClusterCIDR 10.244.0.0/16 -ServiceCIDR 10.0.0.0/16 -KubeDnsServiceIP 10.0.0.10 -LogDir c:\k

2.8 サービス自動起動設定

C:\k\register-svc.ps1 -NetworkMode overlay -ManagementIP <WindowsノードIP> -ClusterCIDR <POD CIDR> -KubeDnsServiceIP <kube-DNSのサービスIP> -LogDir c:\k

・例

C:\k\register-svc.ps1 -NetworkMode overlay -ManagementIP 192.168.1.222 -ClusterCIDR 10.244.0.0/16 -KubeDnsServiceIP 10.0.0.10 -LogDir c:\k

おまけ

[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]# kubectl  get node -o wide
NAME       STATUS   ROLES    AGE     VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                                  KERNEL-VERSION                CONTAINER-RUNTIME
vm200001   Ready    master   19m     v1.18.0   192.168.1.221   <none>        CentOS Linux 7 (Core)                     3.10.0-1062.18.1.el7.x86_64   docker://19.3.8
vm200002   Ready    <none>   2m49s   v1.18.0   192.168.1.222   <none>        Windows Server 2019 Standard Evaluation   10.0.17763.1098               docker://19.3.5
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]# kubectl  get node --show-labels
NAME       STATUS   ROLES    AGE     VERSION   LABELS
vm200001   Ready    master   20m     v1.18.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=vm200001,kubernetes.io/os=linux,node-role.kubernetes.io/master=
vm200002   Ready    <none>   2m56s   v1.18.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=windows,kubernetes.io/arch=amd64,kubernetes.io/hostname=vm200002,kubernetes.io/os=windows,node.kubernetes.io/windows-build=10.0.17763
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]# kubectl  get all -o wide -A
NAMESPACE     NAME                                   READY   STATUS    RESTARTS   AGE   IP              NODE       NOMINATED NODE   READINESS GATES
kube-system   pod/coredns-66bff467f8-svj5g           1/1     Running   0          19m   10.244.0.3      vm200001   <none>           <none>
kube-system   pod/coredns-66bff467f8-vznrl           1/1     Running   0          19m   10.244.0.2      vm200001   <none>           <none>
kube-system   pod/etcd-vm200001                      1/1     Running   0          20m   192.168.1.221   vm200001   <none>           <none>
kube-system   pod/kube-apiserver-vm200001            1/1     Running   0          20m   192.168.1.221   vm200001   <none>           <none>
kube-system   pod/kube-controller-manager-vm200001   1/1     Running   0          20m   192.168.1.221   vm200001   <none>           <none>
kube-system   pod/kube-flannel-ds-amd64-cd7rq        1/1     Running   0          17m   192.168.1.221   vm200001   <none>           <none>
kube-system   pod/kube-proxy-m9qn2                   1/1     Running   0          19m   192.168.1.221   vm200001   <none>           <none>
kube-system   pod/kube-scheduler-vm200001            1/1     Running   0          20m   192.168.1.221   vm200001   <none>           <none>

NAMESPACE     NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE   SELECTOR
default       service/kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP                  20m   <none>
kube-system   service/kube-dns     ClusterIP   10.0.0.10    <none>        53/UDP,53/TCP,9153/TCP   20m   k8s-app=kube-dns

NAMESPACE     NAME                                   DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE   CONTAINERS     IMAGES                                 SELECTOR
kube-system   daemonset.apps/kube-flannel-ds-amd64   1         1         1       1            1           kubernetes.io/os=linux   17m   kube-flannel   quay.io/coreos/flannel:v0.12.0-amd64   app=flannel
kube-system   daemonset.apps/kube-proxy              1         1         1       1            1           kubernetes.io/os=linux   20m   kube-proxy     k8s.gcr.io/kube-proxy:v1.18.0          k8s-app=kube-proxy

NAMESPACE     NAME                      READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES                     SELECTOR
kube-system   deployment.apps/coredns   2/2     2            2           20m   coredns      k8s.gcr.io/coredns:1.6.7   k8s-app=kube-dns

NAMESPACE     NAME                                 DESIRED   CURRENT   READY   AGE   CONTAINERS   IMAGES                     SELECTOR
kube-system   replicaset.apps/coredns-66bff467f8   2         2         2       19m   coredns      k8s.gcr.io/coredns:1.6.7   k8s-app=kube-dns,pod-template-hash=66bff467f8
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#

[root@VM200001 ~]#
[root@VM200001 ~]# cat iis.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: iis
  labels:
    app: iis
spec:
  replicas: 2
  selector:
    matchLabels:
      app: iis
  template:
    metadata:
      labels:
        app: iis
    spec:
      nodeSelector:
        kubernetes.io/os: windows
      containers:
      - name: iis
        image: mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2019
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: iis
  labels:
    app: iis
spec:
  ports:
  - port: 80
    targetPort: 80
    nodePort: 30080
  selector:
    app: iis
  type: NodePort
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]# kubectl apply -f iis.yaml
deployment.apps/iis created
service/iis created
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]# kubectl  get deploy,pod,svc -o wide
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES                                                                SELECTOR
deployment.apps/iis   2/2     2            2           56s   iis          mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2019   app=iis

NAME                       READY   STATUS    RESTARTS   AGE   IP           NODE       NOMINATED NODE   READINESS GATES
pod/iis-6975cd4f89-28v94   1/1     Running   0          56s   10.244.1.3   vm200002   <none>           <none>
pod/iis-6975cd4f89-dq6jc   1/1     Running   0          56s   10.244.1.4   vm200002   <none>           <none>

NAME                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE   SELECTOR
service/iis          NodePort    10.0.119.23   <none>        80:30080/TCP   56s   app=iis
service/kubernetes   ClusterIP   10.0.0.1      <none>        443/TCP        37m   <none>
[root@VM200001 ~]#
[root@VM200001 ~]#
[root@VM200001 ~]#

image.png

image.png

参考

Windows 上のコンテナーに関するドキュメント
https://docs.microsoft.com/ja-jp/virtualization/windowscontainers/

1
2
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?