Go to Qiita Advent Calendar 2024 Top
LoginSignup
2
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

@prodigy413(Obi)

EFK インストール

Last updated at Posted at 2020-05-05

概要

  • k8sテスト環境構築

    EFK インストール

構築目次

環境

  • Rancher: v2.4.8
  • kubernetes(Client): v1.19.1
  • kubernetes(Server): v1.18.8
  • ECK(Elastic Cloud on Kubernetes): v1.2.1
  • Elasticsearch: v7.9.2
  • Kibana: v7.9.2
  • Fluentd: v1.9.3

ECK インストール

ECK operator インストール

※namespaceは自動作成

$ kubectl apply -f https://download.elastic.co/downloads/eck/1.2.1/all-in-one.yaml
customresourcedefinition.apiextensions.k8s.io/apmservers.apm.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/beats.beat.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/elasticsearches.elasticsearch.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/enterprisesearches.enterprisesearch.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/kibanas.kibana.k8s.elastic.co created
namespace/elastic-system created
serviceaccount/elastic-operator created
secret/elastic-webhook-server-cert created
clusterrole.rbac.authorization.k8s.io/elastic-operator created
clusterrole.rbac.authorization.k8s.io/elastic-operator-view created
clusterrole.rbac.authorization.k8s.io/elastic-operator-edit created
clusterrolebinding.rbac.authorization.k8s.io/elastic-operator created
rolebinding.rbac.authorization.k8s.io/elastic-operator created
service/elastic-webhook-server created
statefulset.apps/elastic-operator created
validatingwebhookconfiguration.admissionregistration.k8s.io/elastic-webhook.k8s.elastic.co created

## 確認 ##
$ kubectl get crd | grep -i elastic
apmservers.apm.k8s.elastic.co                        2020-09-27T15:32:37Z
beats.beat.k8s.elastic.co                            2020-09-27T15:32:37Z
elasticsearches.elasticsearch.k8s.elastic.co         2020-09-27T15:32:37Z
enterprisesearches.enterprisesearch.k8s.elastic.co   2020-09-27T15:32:37Z
kibanas.kibana.k8s.elastic.co                        2020-09-27T15:32:37Z

$ kubectl get statefulset -n elastic-system
NAME               READY   AGE
elastic-operator   1/1     62s

$ kubectl get pod -n elastic-system
NAME                 READY   STATUS    RESTARTS   AGE
elastic-operator-0   1/1     Running   0          61s

$ kubectl get svc -n elastic-system
NAME                     TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
elastic-webhook-server   ClusterIP   10.43.201.47   <none>        443/TCP   62s

Elasticsearch インストール

  • Manifest作成
elasticsearch.yaml
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: test-elastic
  namespace: elastic-system
spec:
  version: 7.9.2
  nodeSets:
  - name: default
    count: 1
    config:
      node.master: true
      node.data: true
      node.ingest: true
      node.store.allow_mmap: false
    podTemplate:
      spec:
        volumes:
        - name: elasticsearch-data
          emptyDir: {}
  • Deploy
$ kubectl apply -f elasticsearch.yaml
elasticsearch.elasticsearch.k8s.elastic.co/test-elastic created

## 確認 ##
$ kubectl get elasticsearch -n elastic-system
NAME           HEALTH   NODES   VERSION   PHASE   AGE
test-elastic   green    1       7.9.2     Ready   3m5s

$ kubectl get statefulset -n elastic-system
NAME                      READY   AGE
..........
test-elastic-es-default   1/1     13m
..........

$ kubectl get pod -n elastic-system
NAME                        READY   STATUS    RESTARTS   AGE
..........
test-elastic-es-default-0   1/1     Running   0          2m5s
..........

$ kubectl get svc -n elastic-system
NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
..........
test-elastic-es-default     ClusterIP   None           <none>        9200/TCP   5m22s
test-elastic-es-http        ClusterIP   10.43.136.65   <none>        9200/TCP   5m23s
test-elastic-es-transport   ClusterIP   None           <none>        9300/TCP   5m23s
..........

Kibana インストール

  • Manifest作成
kibana.yaml
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: test-kibana
  namespace: elastic-system
spec:
  version: 7.9.2
  count: 1
  elasticsearchRef:
    name: test-elastic
  • Deploy
$ kubectl apply -f kibana.yaml
kibana.kibana.k8s.elastic.co/test-kibana created

## 確認 ##
$ kubectl get kibana -n elastic-system
NAME          HEALTH   NODES   VERSION   AGE
test-kibana   green    1       7.9.2     2m56s

$ kubectl get deploy -n elastic-system
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
test-kibana-kb   1/1     1            1           2m25s

$ kubectl get pod -n elastic-system
NAME                             READY   STATUS    RESTARTS   AGE
..........
test-kibana-kb-c478fcc7c-rgm6z   1/1     Running   0          10m
..........

$ kubectl get svc -n elastic-system
NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
..........
test-kibana-kb-http         ClusterIP   10.43.222.22   <none>        5601/TCP   2m40s
..........

ネットワーク設定

  • Nginx Ingress 設定
  • KibanaにHTTPSでアクセスするため、Nginx Ingressのpassthrough設定が必要
  • Ingressをpassthroughなしでインストールした場合は以下実行
$ helm upgrade nginx-ingress ingress-nginx/ingress-nginx -n ingress-system --set "controller.extraArgs.enable-ssl-passthrough="
  • Kibanaアクセス用Ingress追加
ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: kibana-ingress
  namespace: elastic-system
  annotations:
    # passthrough設定追加
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
  rules:
  - host: kibana.test.local
    http:
      paths:
      - path: /
        backend:
          serviceName: test-kibana-kb-http
          servicePort: 5601
  • Deploy
$ kubectl apply -f ingress.yaml
ingress.networking.k8s.io/kibana-ingress created

## 確認 ##
$ kubectl get ingress -n elastic-system
NAME             CLASS    HOSTS               ADDRESS           PORTS   AGE
kibana-ingress   <none>   kibana.test.local   192.168.245.111   80      105s
  • hosts 設定追加
    Ingress用IPに「kibana.test.local」に追加
$ cat /etc/hosts
........
192.168.245.111 kibana.test.local
........

Kibana アクセス

  • パスワード確認
$ kubectl get secret test-elastic-es-elastic-user -o=jsonpath='{.data.elastic}' -n elastic-system | base64 --decode; echo
35XJBcQ7i1dov0H3Q727u1aU

Fluentd 設定

Fluentd インストール

  • インストール用manifest作成
fluentd-daemonset-elasticsearch.yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: fluentd
  namespace: elastic-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: fluentd
  namespace: elastic-system
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - namespaces
  verbs:
  - get
  - list
  - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: fluentd
roleRef:
  kind: ClusterRole
  name: fluentd
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: fluentd
  namespace: elastic-system
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: elastic-system
  labels:
    k8s-app: fluentd-logging
    version: v1
spec:
  selector:
    matchLabels:
      k8s-app: fluentd-logging
      version: v1
  template:
    metadata:
      labels:
        k8s-app: fluentd-logging
        version: v1
    spec:
      serviceAccount: fluentd
      serviceAccountName: fluentd
      tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
      containers:
      - name: fluentd
        image: fluent/fluentd-kubernetes-daemonset:v1-debian-elasticsearch
        env:
          - name: FLUENT_UID
            value: "0"
          - name: FLUENTD_SYSTEMD_CONF
            value: "disable"
          - name:  FLUENT_ELASTICSEARCH_HOST
            value: "test-elastic-es-http"
          - name:  FLUENT_ELASTICSEARCH_PORT
            value: "9200"
          - name: FLUENT_ELASTICSEARCH_SCHEME
            value: "https"
          - name: FLUENT_ELASTICSEARCH_SSL_VERIFY
            value: "false"
          - name: FLUENT_ELASTICSEARCH_SSL_VERSION
            value: "TLSv1_2"
          - name: FLUENT_ELASTICSEARCH_USER
            value: "elastic"
          - name: FLUENT_ELASTICSEARCH_PASSWORD
            valueFrom:
              secretKeyRef:
                name: test-elastic-es-elastic-user
                key: elastic
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
  • Deploy
$ kubectl apply -f fluentd-daemonset-elasticsearch.yaml

## 確認 ##
$ kubectl get pod -n elastic-system
NAME                              READY   STATUS    RESTARTS   AGE
......
fluentd-22crn                     1/1     Running   0          4m24s
fluentd-pcxqt                     1/1     Running   0          4m24s
fluentd-w9fvg                     1/1     Running   0          4m24s
......
2
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
2
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?