概要
- k8sテスト環境構築
Descheduler 設定
構築目次
環境
- Rancher: v2.5.3
- kubernetes(Client): v1.19.4
- kubernetes(Server): v1.19.4
- Descheduler: v0.20.0
Descheduler
Deschedulerは設定したPolicyに該当するPodをNodeから削除する
→ 再スケジューリングはデフォルトのschedulerが行う
-
Descheduler Github Page
https://github.com/kubernetes-sigs/descheduler
https://github.com/kubernetes-sigs/descheduler/releases
動作確認
- ポリシーの中で「RemoveDuplicates」の動作を確認する
→ Deployment/ReplicaSetなどのPodが1Node上で複数稼働中の場合はPodを削除する - deschedulerの実行には
Jobリソースを利用する
----------
- テスト用Deployment作成
test-nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
labels:
app: nginx
spec:
selector:
matchLabels:
app: nginx
replicas: 3
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.19.6
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
- manifest適用
## Manifest 適用
$ kubectl apply -f test-nginx.yaml
## 1Pod/1Nodeを確認
$ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-76ccf9dd9d-mw58c 1/1 Running 0 15s 10.42.1.186 worker01 <none> <none>
nginx-76ccf9dd9d-plnq8 1/1 Running 0 15s 10.42.3.110 worker03 <none> <none>
nginx-76ccf9dd9d-s9sdx 1/1 Running 0 15s 10.42.2.113 worker02 <none> <none>
- ポリシー違反状態を作る
## worker03ノードからPodを退去させる
$ kubectl drain worker03 --ignore-daemonsets --delete-local-data --force
## ノードステータス確認
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
master01 Ready controlplane,etcd 12d v1.19.7
worker01 Ready worker 12d v1.19.7
worker02 Ready worker 12d v1.19.7
worker03 Ready,SchedulingDisabled worker 12d v1.19.7
## Pod確認
→ 同一deploymentのPodがworker02で複数稼働中
$ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-76ccf9dd9d-mw58c 1/1 Running 0 5m1s 10.42.1.186 worker01 <none> <none>
nginx-76ccf9dd9d-s9sdx 1/1 Running 0 5m1s 10.42.2.113 worker02 <none> <none>
nginx-76ccf9dd9d-zvxdx 1/1 Running 0 113s 10.42.2.116 worker02 <none> <none>
## worker03ノードをスケジューリング可能に変更
$ kubectl uncordon worker03
## ノードステータス確認
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
master01 Ready controlplane,etcd 12d v1.19.7
worker01 Ready worker 12d v1.19.7
worker02 Ready worker 12d v1.19.7
worker03 Ready worker 12d v1.19.7
- descheduler job作成
マニフェストは以下参照
https://github.com/kubernetes-sigs/descheduler/tree/master/kubernetes
descheduler.yaml
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: descheduler-cluster-role
rules:
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "watch", "list"]
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list", "delete"]
- apiGroups: [""]
resources: ["pods/eviction"]
verbs: ["create"]
- apiGroups: ["scheduling.k8s.io"]
resources: ["priorityclasses"]
verbs: ["get", "watch", "list"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: descheduler-sa
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: descheduler-cluster-role-binding
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: descheduler-cluster-role
subjects:
- name: descheduler-sa
kind: ServiceAccount
namespace: default
---
apiVersion: v1
kind: ConfigMap
metadata:
name: descheduler-policy-configmap
namespace: default
data:
policy.yaml: |
apiVersion: "descheduler/v1alpha1"
kind: "DeschedulerPolicy"
strategies:
# 「RemoveDuplicates」を有効にする
"RemoveDuplicates":
enabled: true
---
apiVersion: batch/v1
kind: Job
metadata:
name: descheduler-job
namespace: default
spec:
parallelism: 1
completions: 1
template:
metadata:
name: descheduler-pod
spec:
priorityClassName: system-cluster-critical
containers:
- name: descheduler
image: k8s.gcr.io/descheduler/descheduler:v0.20.0
volumeMounts:
- mountPath: /policy-dir
name: policy-volume
command:
- "/bin/descheduler"
args:
- "--policy-config-file"
- "/policy-dir/policy.yaml"
- "--v"
- "3"
resources:
requests:
cpu: "500m"
memory: "256Mi"
securityContext:
runAsUser: 1001 # 実行ユーザエラーが出て設定追加
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
privileged: false
readOnlyRootFilesystem: true
runAsNonRoot: true
restartPolicy: "Never"
serviceAccountName: descheduler-sa
volumes:
- name: policy-volume
configMap:
name: descheduler-policy-configmap
- Job実行
## Job実行
$ kubectl apply -f descheduler.yaml
## 完了確認
$ kubectl get job
NAME COMPLETIONS DURATION AGE
descheduler-job 1/1 1s 18s
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
descheduler-job-qnp5r 0/1 Completed 0 59s
..........
- Pod確認
1Pod/1Nodeに戻っていることを確認
$ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-76ccf9dd9d-kz8x2 1/1 Running 0 73s 10.42.3.113 worker03 <none> <none>
nginx-76ccf9dd9d-mw58c 1/1 Running 0 7m39s 10.42.1.186 worker01 <none> <none>
nginx-76ccf9dd9d-s9sdx 1/1 Running 0 7m39s 10.42.2.113 worker02 <none> <none>