LoginSignup
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

@WataMayuko

sample-controllerを動かしてOperatorについて学ぶ

Posted at

はじめに

Operatorの説明を読んでもOperatorとHelmの違いがよく分からん、ということでサンプルを動かしながら学んでみました。

参考

sample-controllerのソースコード：https://github.com/kubernetes/sample-controller
Dockerfile：https://github.com/uzimihsr/sample-controller/blob/in-cluster-config/Dockerfile
手順参考：https://blog.uzimihsr.com/post/2021-07-13-kubernetes-run-sample-controller-as-deployment/

用語

用語 意味
CRD Kubernetesの拡張機能であるCRを利用するための定義
CR CRDで定義した後、通常のリソースと同じように作成できるようになる。specにどのようなフィールドを設定するかは作成者が決められる
Custom Contoroller CRの管理を行うController.Reconciliation Loopを実行するコンポーネント
Operotor CRDとCustom Controllerのセット。特定のソフトウェアの管理を自動化するためのソフトウェア。

Sample Controllerについて

client-goとcode-generatorを使ったKubernetes Wayで実装されたサンプルのcontroller。

Deploymentの上位リソースであるFooを管理するController。
ReplicasetオブジェクトをDeploymentが管理するように、DeploymentオブジェクトをFooが管理できるようにする。
Foo ControllerはNginxイメージを使ったDeploymentを管理し、Fooが管理しているDeploymentオブジェクトを追加・削除されると即時にDeploymentを望ましい状態に調整する。

OpenShift上で動かしてみる

1.まずはソースコードを手元にclone

$ git clone https://github.com/kubernetes/sample-controller
Cloning into 'sample-controller'...
remote: Enumerating objects: 27266, done.
remote: Counting objects: 100% (6353/6353), done.
remote: Compressing objects: 100% (896/896), done.
remote: Total 27266 (delta 5618), reused 5589 (delta 5439), pack-reused 20913
Receiving objects: 100% (27266/27266), 11.84 MiB | 3.29 MiB/s, done.
Resolving deltas: 100% (19121/19121), done.

$ cd sample-controller/

$ ls
artifacts/          CONTRIBUTING.md  controller_test.go  go.mod  hack/    main.go  pkg/       SECURITY_CONTACTS
code-of-conduct.md  controller.go    docs/               go.sum  LICENSE  OWNERS   README.md

$ vi Dockerfile

$ ls
CONTRIBUTING.md  LICENSE  README.md          artifacts           controller.go       docs    go.sum  main.go
Dockerfile       OWNERS   SECURITY_CONTACTS  code-of-conduct.md  controller_test.go  go.mod  hack    pkg

2.openshiftにログイン後、イメージをビルドする

$ oc new-project sample-controller
Now using project "sample-controller" on server "https://XXX.com:6443".

You can add applications to this project with the 'new-app' command. For example, try:

    oc new-app rails-postgresql-example

to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application:

    kubectl create deployment hello-node --image=registry.k8s.io/e2e-test-images/agnhost:2.43 -- /agnhost serve-hostname

$ oc new-build --binary --strategy=docker --name sample
-controller
    * A Docker build using binary input will be created
      * The resulting image will be pushed to image stream tag "sample-controller:latest"
      * A binary build was created, use 'oc start-build --from-dir' to trigger a new build

--> Creating resources with label build=sample-controller ...
    imagestream.image.openshift.io "sample-controller" created
    buildconfig.build.openshift.io "sample-controller" created
--> Success
$ oc start-build sample-controller --from-dir .
Uploading directory "." as binary input for the build ...
..
Uploading finished
build.build.openshift.io/sample-controller-1 started

3.CRDを作成する

$ cd artifacts/examples/
$ ls
crd-status-subresource.yaml  crd.yaml  example-foo.yaml
$ oc apply -f crd.yaml
customresourcedefinition.apiextensions.k8s.io/foos.samplecontroller.k8s.io created

$ oc get crd | grep samplecontroller
foos.samplecontroller.k8s.io

$ oc api-resources | grep foo
foos
 samplecontroller.k8s.io/v1alpha1              true         Foo

4.CRを作成する

$ oc apply -f example-foo.yaml
foo.samplecontroller.k8s.io/example-foo created
$ oc get foo
NAME          AGE
example-foo   60s
$ oc get foo -o yaml
apiVersion: v1
items:
- apiVersion: samplecontroller.k8s.io/v1alpha1
  kind: Foo
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"samplecontroller.k8s.io/v1alpha1","kind":"Foo","metadata":{"annotations":{},"name":"example-foo","namespace":"sample-controller"},"spec":{"deploymentName":"example-foo","replicas":1}}
    creationTimestamp: "2024-08-01T07:21:53Z"
    generation: 1
    name: example-foo
    namespace: sample-controller
    resourceVersion: "2618365"
    uid: 02da87b8-b7e2-4aed-bf30-59127c7ecb82
  spec:
    deploymentName: example-foo
    replicas: 1
kind: List
metadata:
  resourceVersion: ""

CRDとCRはcontrollerがなくても作成することが可能。
しかしCRDとCRというリソースが作成されただけで、何の効果もないオブジェクトが存在するだけの状態。

4.controllerをdeploymentとしてデプロイする

$ oc create deploy sample-controller
 --image=image-registry.openshift-image-registry.svc:5000/sample-controller/sample-controller
deployment.apps/sample-controller created
$ oc get pod
NAME                                 READY   STATUS      RESTARTS   AGE
sample-controller-1-build            0/1     Completed   0          16m
sample-controller-5cb7844bc9-6dk8f   1/1     Running     0          12s

5.deploymentとfooを参照する権限がないとエラーが出てくる

$ oc logs sample-controller-5cb7844b
c9-6dk8f
・
・
・
E0801 07:28:26.879984       1 reflector.go:158] "Unhandled Error" err="pkg/mod/k8s.io/client-go@v0.0.0-20240727175048-71959c526d54/tools/cache/reflector.go:243: Failed to watch *v1.Deployment: failed to list *v1.Deployment: deployments.apps is forbidden: User \"system:serviceaccount:sample-controller:default\" cannot list resource \"deployments\" in API group \"apps\" at the cluster scope" logger="UnhandledError"
W0801 07:28:34.472872       1 reflector.go:561] pkg/mod/k8s.io/client-go@v0.0.0-20240727175048-71959c526d54/tools/cache/reflector.go:243: failed to list *v1alpha1.Foo: foos.samplecontroller.k8s.io is forbidden: User "system:serviceaccount:sample-controller:default" cannot list resource "foos" in API group "samplecontroller.k8s.io" at the cluster scope
E0801 07:28:34.472919       1 reflector.go:158] "Unhandled Error" err="pkg/mod/k8s.io/client-go@v0.0.0-20240727175048-71959c526d54/tools/cache/reflector.go:243: Failed to watch *v1alpha1.Foo: failed to list *v1alpha1.Foo: foos.samplecontroller.k8s.io is forbidden: User \"system:serviceaccount:sample-controller:default\" cannot list resource \"foos\" in API group \"samplecontroller.k8s.io\" at the cluster scope" logger="UnhandledError"

6.権限をつける

$ oc create clusterrole foo-control --verb="*" --resource=foo,deployment
clusterrole.rbac.authorization.k8s.io/foo-control created

foo-control   2024-08-01T07:32:45Z
$ oc get clusterrole foo-control -o
yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: "2024-08-01T07:32:45Z"
  name: foo-control
  resourceVersion: "2624839"
  uid: 18688cce-1c13-4576-bc9e-f381eeb643b5
rules:
- apiGroups:
  - apps
  resources:
  - deployments
  verbs:
  - '*'
- apiGroups:
  - samplecontroller.k8s.io
  resources:
  - foos
  verbs:
  - '*'

$ oc create clusterrolebinding foo-control-binding --clusterrole=foo-control --serviceaccount=sample-controller:default
clusterrolebinding.rbac.authorization.k8s.io/foo-control-binding created:q:

しかしエラー

oc logs sample-controller-5cb7844bc9-n5vqj
・
・
・
E0801 07:57:44.677381       1 controller.go:228] "Error syncing; requeuing for later retry" err="deployments.apps \"example-foo\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>" logger="UnhandledError" objectReference="sample-controller/example-foo"

7.取り敢えず動かしたいだけなので、権限が強すぎるが以下のclusterroleに変更
（この辺りを調べる学ぶ必要がありそうhttps://kubernetes.io/ja/docs/concepts/overview/working-with-objects/owners-dependents/ ）

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: "2024-08-01T07:32:45Z"
  name: foo-control
  resourceVersion: "2639689"
  uid: 18688cce-1c13-4576-bc9e-f381eeb643b5
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - '*'

fooで管理されるdeploymentができたことを確認

$ oc get deploy
NAME                READY   UP-TO-DATE   AVAILABLE   AGE
example-foo         1/1     1            1           8m10s
sample-controller   1/1     1            1           12m

8.fooのreplica数を1から5に変更

apiVersion: samplecontroller.k8s.io/v1alpha1
kind: Foo
metadata:
  name: example-foo
  namespace: sample-controller
spec:
  deploymentName: example-foo
- replicas: 1
+ replicas: 5

deploymentが5つに増えていることを確認

$ oc get deploy
NAME                READY   UP-TO-DATE   AVAILABLE   AGE
example-foo         5/5     5            5           10m
sample-controller   1/1     1            1           15m
0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?