Go to Qiita Advent Calendar 2024 Top
LoginSignup
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

@rjegg(rjtamago)

AWS上に導入したOpenShiftのworkerノード拡張

Last updated at Posted at 2019-12-29

#はじめに
AWS上にOpenShiftを簡単に導入でき、その結果できあがるAWSリソースの一覧は”OpenShiftをAWSに導入するための見積もりに必要な情報”で示した通りです。
その後もう少しいじってみたい！ということで、今回はworkerノードを追加した結果を記載します。

#環境情報

OpenShiftインストーラー： openshift-client-linux-4.2.12.tar.gz
作業端末：Windows10 Subsystem for Linux（Ubuntu：VERSION=16.04.3 LTS）にAWSCLI導入済
AWS：バージニア北部リージョン（us-east-1）
OpenShift環境：こちらでインストールした環境

#ノード追加

##初期構成

  • Node
$ ./oc get nodes
NAME                           STATUS   ROLES    AGE   VERSION
ip-10-0-131-148.ec2.internal   Ready    worker   15h   v1.14.6+cebabbf4a
ip-10-0-135-200.ec2.internal   Ready    master   15h   v1.14.6+cebabbf4a
ip-10-0-144-36.ec2.internal    Ready    master   15h   v1.14.6+cebabbf4a
ip-10-0-155-190.ec2.internal   Ready    worker   15h   v1.14.6+cebabbf4a
ip-10-0-162-16.ec2.internal    Ready    worker   15h   v1.14.6+cebabbf4a
ip-10-0-173-76.ec2.internal    Ready    master   15h   v1.14.6+cebabbf4a
  • Machine

ノードのホストを記述する基本的なユニットです。

$ ./oc get machine -n openshift-machine-api
NAME                                     STATE     TYPE        REGION      ZONE         AGE
ocp4-aws-qt9tx-master-0                  running   m4.xlarge   us-east-1   us-east-1a   15h
ocp4-aws-qt9tx-master-1                  running   m4.xlarge   us-east-1   us-east-1b   15h
ocp4-aws-qt9tx-master-2                  running   m4.xlarge   us-east-1   us-east-1c   15h
ocp4-aws-qt9tx-worker-us-east-1a-vmjz4   running   m4.large    us-east-1   us-east-1a   15h
ocp4-aws-qt9tx-worker-us-east-1b-5txwz   running   m4.large    us-east-1   us-east-1b   15h
ocp4-aws-qt9tx-worker-us-east-1c-4h8zq   running   m4.large    us-east-1   us-east-1c   15h
  • Machineset

マシンのグループです。MachineSet とマシンの関係は、ReplicaSet と Pod の関係と同様です。

$ ./oc get machinesets -n openshift-machine-api
NAME                               DESIRED   CURRENT   READY   AVAILABLE   AGE
ocp4-aws-qt9tx-worker-us-east-1a   1         1         1       1           15h
ocp4-aws-qt9tx-worker-us-east-1b   1         1         1       1           15h
ocp4-aws-qt9tx-worker-us-east-1c   1         1         1       1           15h
ocp4-aws-qt9tx-worker-us-east-1d   0         0                             15h
ocp4-aws-qt9tx-worker-us-east-1e   0         0                             15h
ocp4-aws-qt9tx-worker-us-east-1f   0         0                             15h
  • workerノードの追加対象Machineset（ocp4-aws-qt9tx-worker-us-east-1d）詳細

マシンを追加する必要がある場合や、マシンの数を縮小したりする必要がある場合、コンピューティングのニーズに応じて MachineSet の replicas フィールドを変更します。
今回はまだworkerノードが設定・起動していないMachineset ocp4-aws-qt9tx-worker-us-east-1d からworkerノードを追加することにします。

ocp4-aws-qt9tx-worker-us-east-1d
$ ./oc get machinesets -n openshift-machine-api ocp4-aws-qt9tx-worker-us-east-1d -o yaml
apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
metadata:
  creationTimestamp: "2019-12-26T14:27:57Z"
  generation: 1
  labels:
    machine.openshift.io/cluster-api-cluster: ocp4-aws-qt9tx
  name: ocp4-aws-qt9tx-worker-us-east-1d
  namespace: openshift-machine-api
  resourceVersion: "3314"
  selfLink: /apis/machine.openshift.io/v1beta1/namespaces/openshift-machine-api/machinesets/ocp4-aws-qt9tx-worker-us-east-1d
  uid: e9998df3-27eb-11ea-a1e1-1285990f1d61
spec:
  replicas: 0
  selector:
    matchLabels:
      machine.openshift.io/cluster-api-cluster: ocp4-aws-qt9tx
      machine.openshift.io/cluster-api-machineset: ocp4-aws-qt9tx-worker-us-east-1d
  template:
    metadata:
      creationTimestamp: null
      labels:
        machine.openshift.io/cluster-api-cluster: ocp4-aws-qt9tx
        machine.openshift.io/cluster-api-machine-role: worker
        machine.openshift.io/cluster-api-machine-type: worker
        machine.openshift.io/cluster-api-machineset: ocp4-aws-qt9tx-worker-us-east-1d
    spec:
      metadata:
        creationTimestamp: null
      providerSpec:
        value:
          ami:
            id: ami-01e7fdcb66157b224
          apiVersion: awsproviderconfig.openshift.io/v1beta1
          blockDevices:
          - ebs:
              iops: 0
              volumeSize: 120
              volumeType: gp2
          credentialsSecret:
            name: aws-cloud-credentials
          deviceIndex: 0
          iamInstanceProfile:
            id: ocp4-aws-qt9tx-worker-profile
          instanceType: m4.large
          kind: AWSMachineProviderConfig
          metadata:
            creationTimestamp: null
          placement:
            availabilityZone: us-east-1d
            region: us-east-1
          publicIp: null
          securityGroups:
          - filters:
            - name: tag:Name
              values:
              - ocp4-aws-qt9tx-worker-sg
          subnet:
            filters:
            - name: tag:Name
              values:
              - ocp4-aws-qt9tx-private-us-east-1d
          tags:
          - name: kubernetes.io/cluster/ocp4-aws-qt9tx
            value: owned
          userDataSecret:
            name: worker-user-data
status:
  observedGeneration: 1
  replicas: 0

上記replicas: 0の数値をコマンドもしくは直接編集することでworkerノードが追加されます。

##設定変更
oc scaleコマンドを使ってworkerノードを追加（スケール）します。
参考：2.1. MachineSet の手動によるスケーリング

$ ./oc scale machinesets ocp4-aws-qt9tx-worker-us-east-1d --replicas=1 -n openshift-machine-api
machineset.machine.openshift.io/ocp4-aws-qt9tx-worker-us-east-1d scaled

##設定変更後

  • イベント確認
$ ./oc get event -n openshift-machine-api
LAST SEEN   TYPE     REASON    OBJECT                                           MESSAGE
9m29s       Normal   Updated   machine/ocp4-aws-qt9tx-master-0                  Updated machine ocp4-aws-qt9tx-master-0
9m32s       Normal   Updated   machine/ocp4-aws-qt9tx-master-1                  Updated machine ocp4-aws-qt9tx-master-1
9m31s       Normal   Updated   machine/ocp4-aws-qt9tx-master-2                  Updated machine ocp4-aws-qt9tx-master-2
9m30s       Normal   Updated   machine/ocp4-aws-qt9tx-worker-us-east-1a-vmjz4   Updated machine ocp4-aws-qt9tx-worker-us-east-1a-vmjz4
9m30s       Normal   Updated   machine/ocp4-aws-qt9tx-worker-us-east-1b-5txwz   Updated machine ocp4-aws-qt9tx-worker-us-east-1b-5txwz
9m30s       Normal   Updated   machine/ocp4-aws-qt9tx-worker-us-east-1c-4h8zq   Updated machine ocp4-aws-qt9tx-worker-us-east-1c-4h8zq
5m42s       Normal   Created   machine/ocp4-aws-qt9tx-worker-us-east-1d-64l7m   Created Machine ocp4-aws-qt9tx-worker-us-east-1d-64l7m
81s         Normal   Updated   machine/ocp4-aws-qt9tx-worker-us-east-1d-64l7m   Updated machine ocp4-aws-qt9tx-worker-us-east-1d-64l7m
  • Machine確認
$ ./oc get machine -n openshift-machine-api
NAME                                     STATE     TYPE        REGION      ZONE         AGE
ocp4-aws-qt9tx-master-0                  running   m4.xlarge   us-east-1   us-east-1a   15h
ocp4-aws-qt9tx-master-1                  running   m4.xlarge   us-east-1   us-east-1b   15h
ocp4-aws-qt9tx-master-2                  running   m4.xlarge   us-east-1   us-east-1c   15h
ocp4-aws-qt9tx-worker-us-east-1a-vmjz4   running   m4.large    us-east-1   us-east-1a   15h
ocp4-aws-qt9tx-worker-us-east-1b-5txwz   running   m4.large    us-east-1   us-east-1b   15h
ocp4-aws-qt9tx-worker-us-east-1c-4h8zq   running   m4.large    us-east-1   us-east-1c   15h
ocp4-aws-qt9tx-worker-us-east-1d-64l7m   running   m4.large    us-east-1   us-east-1d   6m14s
  • Machineset確認
$ ./oc get machinesets -n openshift-machine-api
NAME                               DESIRED   CURRENT   READY   AVAILABLE   AGE
ocp4-aws-qt9tx-worker-us-east-1a   1         1         1       1           15h
ocp4-aws-qt9tx-worker-us-east-1b   1         1         1       1           15h
ocp4-aws-qt9tx-worker-us-east-1c   1         1         1       1           15h
ocp4-aws-qt9tx-worker-us-east-1d   1         1         1       1           15h
ocp4-aws-qt9tx-worker-us-east-1e   0         0                             15h
ocp4-aws-qt9tx-worker-us-east-1f   0         0                             15h
  • Node確認
$ ./oc get nodes
NAME                           STATUS   ROLES    AGE    VERSION
ip-10-0-131-148.ec2.internal   Ready    worker   15h    v1.14.6+cebabbf4a
ip-10-0-135-200.ec2.internal   Ready    master   15h    v1.14.6+cebabbf4a
ip-10-0-144-36.ec2.internal    Ready    master   15h    v1.14.6+cebabbf4a
ip-10-0-155-190.ec2.internal   Ready    worker   15h    v1.14.6+cebabbf4a
ip-10-0-162-16.ec2.internal    Ready    worker   15h    v1.14.6+cebabbf4a
ip-10-0-173-76.ec2.internal    Ready    master   15h    v1.14.6+cebabbf4a
ip-10-0-191-196.ec2.internal   Ready    worker   2m7s   v1.14.6+cebabbf4a
  • AWSのリソース

image.png

一番下のocp4-aws-qt9x-worker-us-east-1d-64l7mがoc scaleコマンドにより自動で追加されたEC2。

##詳細情報

  • workerノードを追加したMachineset（ocp4-aws-qt9tx-worker-us-east-1d）詳細
ocp4-aws-qt9tx-worker-us-east-1d
$ ./oc get machinesets -n openshift-machine-api ocp4-aws-qt9tx-worker-us-east-1d -o yaml
apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
metadata:
  creationTimestamp: "2019-12-26T14:27:57Z"
  generation: 2
  labels:
    machine.openshift.io/cluster-api-cluster: ocp4-aws-qt9tx
  name: ocp4-aws-qt9tx-worker-us-east-1d
  namespace: openshift-machine-api
  resourceVersion: "296242"
  selfLink: /apis/machine.openshift.io/v1beta1/namespaces/openshift-machine-api/machinesets/ocp4-aws-qt9tx-worker-us-east-1d
  uid: e9998df3-27eb-11ea-a1e1-1285990f1d61
spec:
  replicas: 1
  selector:
    matchLabels:
      machine.openshift.io/cluster-api-cluster: ocp4-aws-qt9tx
      machine.openshift.io/cluster-api-machineset: ocp4-aws-qt9tx-worker-us-east-1d
  template:
    metadata:
      creationTimestamp: null
      labels:
        machine.openshift.io/cluster-api-cluster: ocp4-aws-qt9tx
        machine.openshift.io/cluster-api-machine-role: worker
        machine.openshift.io/cluster-api-machine-type: worker
        machine.openshift.io/cluster-api-machineset: ocp4-aws-qt9tx-worker-us-east-1d
    spec:
      metadata:
        creationTimestamp: null
      providerSpec:
        value:
          ami:
            id: ami-01e7fdcb66157b224
          apiVersion: awsproviderconfig.openshift.io/v1beta1
          blockDevices:
          - ebs:
              iops: 0
              volumeSize: 120
              volumeType: gp2
          credentialsSecret:
            name: aws-cloud-credentials
          deviceIndex: 0
          iamInstanceProfile:
            id: ocp4-aws-qt9tx-worker-profile
          instanceType: m4.large
          kind: AWSMachineProviderConfig
          metadata:
            creationTimestamp: null
          placement:
            availabilityZone: us-east-1d
            region: us-east-1
          publicIp: null
          securityGroups:
          - filters:
            - name: tag:Name
              values:
              - ocp4-aws-qt9tx-worker-sg
          subnet:
            filters:
            - name: tag:Name
              values:
              - ocp4-aws-qt9tx-private-us-east-1d
          tags:
          - name: kubernetes.io/cluster/ocp4-aws-qt9tx
            value: owned
          userDataSecret:
            name: worker-user-data
status:
  availableReplicas: 1
  fullyLabeledReplicas: 1
  observedGeneration: 2
  readyReplicas: 1
  replicas: 1

replicas: 1となっているのが確認できます。

  • 追加されたMachineの詳細情報
ocp4-aws-qt9tx-worker-us-east-1d-64l7m
$ ./oc get machine -n openshift-machine-api ocp4-aws-qt9tx-worker-us-east-1d-64l7m -o yaml
apiVersion: machine.openshift.io/v1beta1
kind: Machine
metadata:
  annotations:
    machine.openshift.io/instance-state: running
  creationTimestamp: "2019-12-27T06:19:22Z"
  finalizers:
  - machine.machine.openshift.io
  generateName: ocp4-aws-qt9tx-worker-us-east-1d-
  generation: 2
  labels:
    machine.openshift.io/cluster-api-cluster: ocp4-aws-qt9tx
    machine.openshift.io/cluster-api-machine-role: worker
    machine.openshift.io/cluster-api-machine-type: worker
    machine.openshift.io/cluster-api-machineset: ocp4-aws-qt9tx-worker-us-east-1d
    machine.openshift.io/instance-type: m4.large
    machine.openshift.io/region: us-east-1
    machine.openshift.io/zone: us-east-1d
  name: ocp4-aws-qt9tx-worker-us-east-1d-64l7m
  namespace: openshift-machine-api
  ownerReferences:
  - apiVersion: machine.openshift.io/v1beta1
    blockOwnerDeletion: true
    controller: true
    kind: MachineSet
    name: ocp4-aws-qt9tx-worker-us-east-1d
    uid: e9998df3-27eb-11ea-a1e1-1285990f1d61
  resourceVersion: "296241"
  selfLink: /apis/machine.openshift.io/v1beta1/namespaces/openshift-machine-api/machines/ocp4-aws-qt9tx-worker-us-east-1d-64l7m
  uid: d2cb0d58-2870-11ea-acf1-1259770d53d1
spec:
  metadata:
    creationTimestamp: null
  providerID: aws:///us-east-1d/i-06040dc85dd9592fc
  providerSpec:
    value:
      ami:
        id: ami-01e7fdcb66157b224
      apiVersion: awsproviderconfig.openshift.io/v1beta1
      blockDevices:
      - ebs:
          iops: 0
          volumeSize: 120
          volumeType: gp2
      credentialsSecret:
        name: aws-cloud-credentials
      deviceIndex: 0
      iamInstanceProfile:
        id: ocp4-aws-qt9tx-worker-profile
      instanceType: m4.large
      kind: AWSMachineProviderConfig
      metadata:
        creationTimestamp: null
      placement:
        availabilityZone: us-east-1d
        region: us-east-1
      publicIp: null
      securityGroups:
      - filters:
        - name: tag:Name
          values:
          - ocp4-aws-qt9tx-worker-sg
      subnet:
        filters:
        - name: tag:Name
          values:
          - ocp4-aws-qt9tx-private-us-east-1d
      tags:
      - name: kubernetes.io/cluster/ocp4-aws-qt9tx
        value: owned
      userDataSecret:
        name: worker-user-data
status:
  addresses:
  - address: 10.0.191.196
    type: InternalIP
  - address: ""
    type: ExternalDNS
  - address: ip-10-0-191-196.ec2.internal
    type: InternalDNS
  lastUpdated: "2019-12-27T06:23:45Z"
  nodeRef:
    kind: Node
    name: ip-10-0-191-196.ec2.internal
    uid: 50608a5c-2871-11ea-a4d3-0aa8664615c9
  providerStatus:
    apiVersion: awsproviderconfig.openshift.io/v1beta1
    conditions:
    - lastProbeTime: "2019-12-27T06:19:24Z"
      lastTransitionTime: "2019-12-27T06:19:24Z"
      message: machine successfully created
      reason: MachineCreationSucceeded
      status: "True"
      type: MachineCreation
    instanceId: i-06040dc85dd9592fc
    instanceState: running
    kind: AWSMachineProviderStatus
  • ノード詳細
ip-10-0-191-196.ec2.internal
$ ./oc get nodes ip-10-0-191-196.ec2.internal -o yaml
apiVersion: v1
kind: Node
metadata:
  annotations:
    machine.openshift.io/machine: openshift-machine-api/ocp4-aws-qt9tx-worker-us-east-1d-64l7m
    machineconfiguration.openshift.io/currentConfig: rendered-worker-2167ebc4b5181931d0ce49a36db005e9
    machineconfiguration.openshift.io/desiredConfig: rendered-worker-2167ebc4b5181931d0ce49a36db005e9
    machineconfiguration.openshift.io/state: Done
    volumes.kubernetes.io/controller-managed-attach-detach: "true"
  creationTimestamp: "2019-12-27T06:22:53Z"
  labels:
    beta.kubernetes.io/arch: amd64
    beta.kubernetes.io/instance-type: m4.large
    beta.kubernetes.io/os: linux
    failure-domain.beta.kubernetes.io/region: us-east-1
    failure-domain.beta.kubernetes.io/zone: us-east-1d
    kubernetes.io/arch: amd64
    kubernetes.io/hostname: ip-10-0-191-196
    kubernetes.io/os: linux
    node-role.kubernetes.io/worker: ""
    node.openshift.io/os_id: rhcos
  name: ip-10-0-191-196.ec2.internal
  resourceVersion: "305698"
  selfLink: /api/v1/nodes/ip-10-0-191-196.ec2.internal
  uid: 50608a5c-2871-11ea-a4d3-0aa8664615c9
spec:
  providerID: aws:///us-east-1d/i-06040dc85dd9592fc
status:
  addresses:
  - address: 10.0.191.196
    type: InternalIP
  - address: ip-10-0-191-196.ec2.internal
    type: Hostname
  - address: ip-10-0-191-196.ec2.internal
    type: InternalDNS
  allocatable:
    attachable-volumes-aws-ebs: "39"
    cpu: 1500m
    hugepages-1Gi: "0"
    hugepages-2Mi: "0"
    memory: 7547440Ki
    pods: "250"
  capacity:
    attachable-volumes-aws-ebs: "39"
    cpu: "2"
    hugepages-1Gi: "0"
    hugepages-2Mi: "0"
    memory: 8161840Ki
    pods: "250"
  conditions:
  - lastHeartbeatTime: "2019-12-27T06:48:46Z"
    lastTransitionTime: "2019-12-27T06:22:53Z"
    message: kubelet has sufficient memory available
    reason: KubeletHasSufficientMemory
    status: "False"
    type: MemoryPressure
  - lastHeartbeatTime: "2019-12-27T06:48:46Z"
    lastTransitionTime: "2019-12-27T06:22:53Z"
    message: kubelet has no disk pressure
    reason: KubeletHasNoDiskPressure
    status: "False"
    type: DiskPressure
  - lastHeartbeatTime: "2019-12-27T06:48:46Z"
    lastTransitionTime: "2019-12-27T06:22:53Z"
    message: kubelet has sufficient PID available
    reason: KubeletHasSufficientPID
    status: "False"
    type: PIDPressure
  - lastHeartbeatTime: "2019-12-27T06:48:46Z"
    lastTransitionTime: "2019-12-27T06:23:45Z"
    message: kubelet is posting ready status
    reason: KubeletReady
    status: "True"
    type: Ready
  daemonEndpoints:
    kubeletEndpoint:
      Port: 10250
  images:
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d5de6bd0ca757e07c58651234ceac857b24422fba98ed0b2912e51ba5fa2d4ee
    - <none>:<none>
    sizeBytes: 1194804151
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:aa6b4ea74e537e53aa7b72f2d479eafbad60c78e9ea49e3bd5b59720cd46c332
    - <none>:<none>
    sizeBytes: 405473489
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0048e6f8730510893370553c1386e34827abcdb4dffaa2d392f1614c63a1c9c7
    - <none>:<none>
    sizeBytes: 388527143
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1d7fdf7cacbd78bef24f9eb9d355161529ab2d561896de62572d81e55b61d370
    - <none>:<none>
    sizeBytes: 323890713
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b10e3e94d34509b1bc5d826b0ea33c352eefd37c157b386580f2b87f7d35bf40
    - <none>:<none>
    sizeBytes: 318297754
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e451a219626274f7556b0ad21ff7bb2ca31be2ddd93f4c7533da49b78c82678d
    - <none>:<none>
    sizeBytes: 306490354
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c919d6f77371e7d4a5f2d30cdc5d5bf3942c001c1ca82640fe5d1c15f7377cbd
    - <none>:<none>
    sizeBytes: 305375776
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a8129d076e19fea59b3f9e8a04fbc0e0cdf53321ef3ab460bdbd1ffbd2230438
    - <none>:<none>
    sizeBytes: 285110290
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4debe16999ebb9ca954822a6c75ee4461186be6eeb13313bf4ede367264ba5e3
    - <none>:<none>
    sizeBytes: 282984020
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9be3b6804fbb09d58c352a2110ad42d79c158dc258d7b02244bf209e453a7148
    - <none>:<none>
    sizeBytes: 255927512
  - names:
    - quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a81e1a02bb25f93e6f3d3b9d5ddddb07a0c8025b4e5bd717cc6e4a6f3b8bdea2
    - <none>:<none>
    sizeBytes: 250722882
  nodeInfo:
    architecture: amd64
    bootID: a4b39f9a-09f4-4889-8fcb-e73105aeb683
    containerRuntimeVersion: cri-o://1.14.11-0.24.dev.rhaos4.2.gitc41de67.el8
    kernelVersion: 4.18.0-147.0.3.el8_1.x86_64
    kubeProxyVersion: v1.14.6+cebabbf4a
    kubeletVersion: v1.14.6+cebabbf4a
    machineID: 0c252dda944644e1af2d6f5cdec3c6c6
    operatingSystem: linux
    osImage: Red Hat Enterprise Linux CoreOS 42.81.20191210.1 (Ootpa)
    systemUUID: ec22f9cf-8e3c-84a9-5739-1c33d4822876

#まとめ
oc scaleコマンドを使って簡単にworkerノードを追加することができました。

この後は、作成したOpenShiftクラスタを削除した話を別の記事に書きます。
→こちら（AWS上に導入したOpenShiftを削除する）

#参考
マシン管理

1
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?