Red Hat OpenShift Container Platform (OCP) 4.10 では、Logging 5.4 Technology Previews の位置付けながら log collector に Vector を使用することができます。
Release notes for Logging
Logging 5.4 Technology Previews
https://docs.openshift.com/container-platform/4.10/logging/cluster-logging-release-notes.html#cluster-logging-technology-previews-5.4
About Vector
Vector is a log collector offered as a tech-preview alternative to the current default collector for the logging subsystem.The following outputs are supported:
- elasticsearch. An external Elasticsearch instance. The elasticsearch output can use a TLS connection.
- kafka. A Kafka broker. The kafka output can use an unsecured or TLS connection.
- loki. Loki, a horizontally scalable, highly available, multi-tenant log aggregation system.
ここでは、上記の記述に従って、Cluster Logging 5.4 の log collector を fluentd から vector に変更してみます。
Cluster Logging Version の確認
Cluster Logging 5.4 が導入されていることを確認します。
$ oc version
Client Version: 4.10.13
Server Version: 4.10.13
Kubernetes Version: v1.23.5+b463d71
$ oc get csv -n openshift-logging
NAME DISPLAY VERSION REPLACES PHASE
cluster-logging.5.4.1-24 Red Hat OpenShift Logging 5.4.1-24 Succeeded
elasticsearch-operator.5.4.1-24 OpenShift Elasticsearch Operator 5.4.1-24 Succeeded
ClusterLogging custom resource (CR) の変更
以下のコマンドで、ClusterLogging CR を変更します。
$ oc -n openshift-logging edit ClusterLogging instance
通常は、.spec.collection に fluentd が設定されています。
spec:
collection:
logs:
type: "fluentd"
fluentd:
resources:
limits:
memory: 736Mi
requests:
cpu: 200m
memory: 736Mi
これを、以下のように vector に変更します。
spec:
collection:
logs:
type: "vector"
vector: {}
log collector Pod の確認
現在の実装では、log collector を vector に変更しても、Pod Name、Container Name は変更されません。
そこで、log collector Pod が再起動された後に、vector で稼働していることを oc logs コマンドで確認してみます。
$ oc get pod -l component=collector -n openshift-logging
NAME READY STATUS RESTARTS AGE
collector-7fq8w 2/2 Running 0 2m
collector-cm5n6 2/2 Running 0 2m
collector-gbwzv 2/2 Running 0 2m
collector-grkg4 2/2 Running 0 2m
collector-h4c8x 2/2 Running 0 2m
collector-rzqsm 2/2 Running 0 3m
collector-sgfhs 2/2 Running 0 2m
collector-zpg25 2/2 Running 0 2m
$ oc get pod -l component=collector -n openshift-logging -o jsonpath='{range .items[*]}{"POD : "}{.metadata.name}{" CONTAINER : "}{.spec.containers[*].name}{"\n"}{end}'
POD : collector-7fq8w CONTAINER : collector logfilesmetricexporter
POD : collector-cm5n6 CONTAINER : collector logfilesmetricexporter
POD : collector-gbwzv CONTAINER : collector logfilesmetricexporter
POD : collector-grkg4 CONTAINER : collector logfilesmetricexporter
POD : collector-h4c8x CONTAINER : collector logfilesmetricexporter
POD : collector-rzqsm CONTAINER : collector logfilesmetricexporter
POD : collector-sgfhs CONTAINER : collector logfilesmetricexporter
POD : collector-zpg25 CONTAINER : collector logfilesmetricexporter
$ oc logs collector-7fq8w -c collector | head -n 31
Jun 13 06:04:16.693 INFO vector::app: Log level is enabled. level="info"
Jun 13 06:04:16.694 INFO vector::app: Loading configs. path=[("/etc/vector/vector.toml", Some(Toml))]
Jun 13 06:04:16.698 INFO vector::sources::kubernetes_logs: Obtained Kubernetes Node name to collect logs for (self). self_node_name="infra01"
Jun 13 06:04:16.725 INFO vector::topology: Running healthchecks.
Jun 13 06:04:16.725 INFO vector::topology::builder: Healthcheck: Passed.
Jun 13 06:04:16.726 INFO vector::topology: Starting source. name="raw_journal_logs"
Jun 13 06:04:16.726 INFO vector::topology: Starting source. name="k8s_audit_logs"
Jun 13 06:04:16.726 INFO source{component_kind="source" component_name=k8s_audit_logs component_type=file}: vector::sources::file: Starting file server. include=["/var/log/kube-apiserver/audit.log"] exclude=[]
Jun 13 06:04:16.726 INFO vector::topology: Starting source. name="openshift_audit_logs"
Jun 13 06:04:16.726 INFO vector::topology: Starting source. name="host_audit_logs"
Jun 13 06:04:16.726 INFO vector::topology: Starting source. name="internal_metrics"
Jun 13 06:04:16.726 INFO vector::topology: Starting source. name="raw_container_logs"
Jun 13 06:04:16.726 INFO source{component_kind="source" component_name=openshift_audit_logs component_type=file}: vector::sources::file: Starting file server. include=["/var/log/oauth-apiserver.audit.log"] exclude=[]
Jun 13 06:04:16.726 INFO source{component_kind="source" component_name=k8s_audit_logs component_type=file}:file_server: file_source::checkpointer: Loaded checkpoint data.
Jun 13 06:04:16.726 INFO source{component_kind="source" component_name=host_audit_logs component_type=file}: vector::sources::file: Starting file server. include=["/var/log/audit/audit.log"] exclude=[]
Jun 13 06:04:16.727 INFO source{component_kind="source" component_name=openshift_audit_logs component_type=file}:file_server: file_source::checkpointer: Loaded checkpoint data.
Jun 13 06:04:16.727 INFO source{component_kind="source" component_name=host_audit_logs component_type=file}:file_server: file_source::checkpointer: Loaded checkpoint data.
Jun 13 06:04:16.727 INFO vector::topology: Starting transform. name="route_container_logs.app"
Jun 13 06:04:16.727 INFO vector::topology: Starting transform. name="audit"
Jun 13 06:04:16.727 INFO vector::topology: Starting transform. name="default_add_es_id"
Jun 13 06:04:16.727 INFO vector::topology: Starting transform. name="route_container_logs.infra"
Jun 13 06:04:16.727 INFO vector::topology: Starting transform. name="default_dedot_and_flatten"
Jun 13 06:04:16.727 INFO vector::topology: Starting transform. name="journal_logs"
Jun 13 06:04:16.727 INFO vector::topology: Starting transform. name="all-to-default"
Jun 13 06:04:16.727 INFO vector::topology: Starting transform. name="container_logs"
Jun 13 06:04:16.727 INFO vector::topology: Starting transform. name="infrastructure"
Jun 13 06:04:16.727 INFO vector::topology: Starting transform. name="application"
Jun 13 06:04:16.727 INFO vector::topology: Starting sink. name="default"
Jun 13 06:04:16.728 INFO vector::topology: Starting sink. name="prometheus_output"
Jun 13 06:04:16.728 INFO source{component_kind="source" component_name=raw_container_logs component_type=kubernetes_logs}:file_server: file_source::checkpointer: Loaded checkpoint data.
Jun 13 06:04:16.728 INFO vector: Vector has started. version="0.14.1" arch="x86_64" build_id="none"
最下行に Vector has started. version="0.14.1" arch="x86_64" build_id="none" と表示されていることが分かります。
Kibana UI の確認
log collector を vector に変更した状態で Kibana UI から Elasticsearch の情報が参照できるか確認します。
Discover 画面で pipeline_metadata.collector.name が存在（exists）する log を Filter して、pipeline_metadata.collector.name と kubernetes.pod_name を表示してみます。
先頭の log の詳細を確認してみます。
pipeline_metadata.collector.name が vector、pipeline_metadata.collector.version が 0.14.1 となっており、oc logs コマンドで確認した内容と一致している事が分かります。
簡単に確認した限りではありますが、Clutster Logging の log store（Elasticsearch）、visualizer（Kibana）の設定変更を行わなくても、vector で実装した log collector で取得した情報が正常に参照できるようです。