Red Hat OpenShift Container Platform (OCP) 4.10 では、Logging 5.4 Technology Previews の位置付けながら log collector に Vector を使用することができます。

Release notes for Logging
Logging 5.4 Technology Previews
https://docs.openshift.com/container-platform/4.10/logging/cluster-logging-release-notes.html#cluster-logging-technology-previews-5.4

About Vector
Vector is a log collector offered as a tech-preview alternative to the current default collector for the logging subsystem.

The following outputs are supported:

elasticsearch. An external Elasticsearch instance. The elasticsearch output can use a TLS connection.

kafka. A Kafka broker. The kafka output can use an unsecured or TLS connection.

loki. Loki, a horizontally scalable, highly available, multi-tenant log aggregation system.

ここでは、上記の記述に従って、Cluster Logging 5.4 の log collector を fluentd から vector に変更してみます。

Cluster Logging Version の確認

Cluster Logging 5.4 が導入されていることを確認します。

$ oc version
Client Version: 4.10.13
Server Version: 4.10.13
Kubernetes Version: v1.23.5+b463d71

$ oc get csv -n openshift-logging
NAME                              DISPLAY                            VERSION    REPLACES   PHASE
cluster-logging.5.4.1-24          Red Hat OpenShift Logging          5.4.1-24              Succeeded
elasticsearch-operator.5.4.1-24   OpenShift Elasticsearch Operator   5.4.1-24              Succeeded

ClusterLogging custom resource (CR) の変更

以下のコマンドで、ClusterLogging CR を変更します。

$ oc -n openshift-logging edit ClusterLogging instance

通常は、.spec.collection に fluentd が設定されています。

  spec:
    collection:
      logs:
        type: "fluentd"
        fluentd:
          resources: 
            limits:
              memory: 736Mi   
            requests:
              cpu: 200m
              memory: 736Mi

これを、以下のように vector に変更します。

  spec:
    collection:
      logs:
        type: "vector"
        vector: {}

log collector Pod の確認

現在の実装では、log collector を vector に変更しても、Pod Name、Container Name は変更されません。
そこで、log collector Pod が再起動された後に、vector で稼働していることを oc logs コマンドで確認してみます。

$ oc get pod -l component=collector -n openshift-logging
NAME              READY   STATUS    RESTARTS   AGE
collector-7fq8w   2/2     Running   0          2m
collector-cm5n6   2/2     Running   0          2m
collector-gbwzv   2/2     Running   0          2m
collector-grkg4   2/2     Running   0          2m
collector-h4c8x   2/2     Running   0          2m
collector-rzqsm   2/2     Running   0          3m
collector-sgfhs   2/2     Running   0          2m
collector-zpg25   2/2     Running   0          2m

$ oc get pod -l component=collector -n openshift-logging -o jsonpath='{range .items[*]}{"POD : "}{.metadata.name}{" CONTAINER : "}{.spec.containers[*].name}{"\n"}{end}'
POD : collector-7fq8w CONTAINER : collector logfilesmetricexporter
POD : collector-cm5n6 CONTAINER : collector logfilesmetricexporter
POD : collector-gbwzv CONTAINER : collector logfilesmetricexporter
POD : collector-grkg4 CONTAINER : collector logfilesmetricexporter
POD : collector-h4c8x CONTAINER : collector logfilesmetricexporter
POD : collector-rzqsm CONTAINER : collector logfilesmetricexporter
POD : collector-sgfhs CONTAINER : collector logfilesmetricexporter
POD : collector-zpg25 CONTAINER : collector logfilesmetricexporter

$ oc logs collector-7fq8w -c collector | head -n 31
Jun 13 06:04:16.693  INFO vector::app: Log level is enabled. level="info"
Jun 13 06:04:16.694  INFO vector::app: Loading configs. path=[("/etc/vector/vector.toml", Some(Toml))]
Jun 13 06:04:16.698  INFO vector::sources::kubernetes_logs: Obtained Kubernetes Node name to collect logs for (self). self_node_name="infra01"
Jun 13 06:04:16.725  INFO vector::topology: Running healthchecks.
Jun 13 06:04:16.725  INFO vector::topology::builder: Healthcheck: Passed.
Jun 13 06:04:16.726  INFO vector::topology: Starting source. name="raw_journal_logs"
Jun 13 06:04:16.726  INFO vector::topology: Starting source. name="k8s_audit_logs"
Jun 13 06:04:16.726  INFO source{component_kind="source" component_name=k8s_audit_logs component_type=file}: vector::sources::file: Starting file server. include=["/var/log/kube-apiserver/audit.log"] exclude=[]
Jun 13 06:04:16.726  INFO vector::topology: Starting source. name="openshift_audit_logs"
Jun 13 06:04:16.726  INFO vector::topology: Starting source. name="host_audit_logs"
Jun 13 06:04:16.726  INFO vector::topology: Starting source. name="internal_metrics"
Jun 13 06:04:16.726  INFO vector::topology: Starting source. name="raw_container_logs"
Jun 13 06:04:16.726  INFO source{component_kind="source" component_name=openshift_audit_logs component_type=file}: vector::sources::file: Starting file server. include=["/var/log/oauth-apiserver.audit.log"] exclude=[]
Jun 13 06:04:16.726  INFO source{component_kind="source" component_name=k8s_audit_logs component_type=file}:file_server: file_source::checkpointer: Loaded checkpoint data.
Jun 13 06:04:16.726  INFO source{component_kind="source" component_name=host_audit_logs component_type=file}: vector::sources::file: Starting file server. include=["/var/log/audit/audit.log"] exclude=[]
Jun 13 06:04:16.727  INFO source{component_kind="source" component_name=openshift_audit_logs component_type=file}:file_server: file_source::checkpointer: Loaded checkpoint data.
Jun 13 06:04:16.727  INFO source{component_kind="source" component_name=host_audit_logs component_type=file}:file_server: file_source::checkpointer: Loaded checkpoint data.
Jun 13 06:04:16.727  INFO vector::topology: Starting transform. name="route_container_logs.app"
Jun 13 06:04:16.727  INFO vector::topology: Starting transform. name="audit"
Jun 13 06:04:16.727  INFO vector::topology: Starting transform. name="default_add_es_id"
Jun 13 06:04:16.727  INFO vector::topology: Starting transform. name="route_container_logs.infra"
Jun 13 06:04:16.727  INFO vector::topology: Starting transform. name="default_dedot_and_flatten"
Jun 13 06:04:16.727  INFO vector::topology: Starting transform. name="journal_logs"
Jun 13 06:04:16.727  INFO vector::topology: Starting transform. name="all-to-default"
Jun 13 06:04:16.727  INFO vector::topology: Starting transform. name="container_logs"
Jun 13 06:04:16.727  INFO vector::topology: Starting transform. name="infrastructure"
Jun 13 06:04:16.727  INFO vector::topology: Starting transform. name="application"
Jun 13 06:04:16.727  INFO vector::topology: Starting sink. name="default"
Jun 13 06:04:16.728  INFO vector::topology: Starting sink. name="prometheus_output"
Jun 13 06:04:16.728  INFO source{component_kind="source" component_name=raw_container_logs component_type=kubernetes_logs}:file_server: file_source::checkpointer: Loaded checkpoint data.
Jun 13 06:04:16.728  INFO vector: Vector has started. version="0.14.1" arch="x86_64" build_id="none"

最下行に Vector has started. version="0.14.1" arch="x86_64" build_id="none" と表示されていることが分かります。

Kibana UI の確認

log collector を vector に変更した状態で Kibana UI から Elasticsearch の情報が参照できるか確認します。
Discover 画面で pipeline_metadata.collector.name が存在（exists）する log を Filter して、pipeline_metadata.collector.name と kubernetes.pod_name を表示してみます。

先頭の log の詳細を確認してみます。

pipeline_metadata.collector.name が vector、pipeline_metadata.collector.version が 0.14.1 となっており、oc logs コマンドで確認した内容と一致している事が分かります。

簡単に確認した限りではありますが、Clutster Logging の log store（Elasticsearch）、visualizer（Kibana）の設定変更を行わなくても、vector で実装した log collector で取得した情報が正常に参照できるようです。