1
Help us understand the problem. What are the problem?

posted at

updated at

fluent-bit で log file を tail して elasticsearch に送る

初めて fluent-bit を使ってみた。軽量な fluentd だそうです。pluginの追加あたりで躓いたので、メモっておく

install fluent-bit

Ref https://docs.fluentbit.io/manual/installation/linux/redhat-centos#configure-yum

repoを登録する

/etc/yum.repos.d/fluent-bit.repo
[fluent-bit]
name = Fluent Bit
baseurl = https://packages.fluentbit.io/centos/$releasever/$basearch/
gpgcheck=1
gpgkey=https://packages.fluentbit.io/fluentbit.key
repo_gpgcheck=1
enabled=1

install

$ yum install fluent-bit

systemctl status fluent-bit

config

elasticsearchに出力するpluginを有効にします

/etc/fluent-bit/plugins.conf
[PLUGINS]
    @INCLUDE output-elasticsearch.conf             <--- 追加
    # Path /path/to/out_gstdout.so

defaultのconfigを書き換えます

/etc/fluent-bit/fluent-bit.conf

[SERVICE]
    daemon       Off
    log_level    debug
    plugins_file plugins.conf

# https://docs.fluentbit.io/manual/pipeline/inputs/tail
[INPUT]
    Name   tail
    Path   /path/to/your/log

# https://docs.fluentbit.io/manual/pipeline/outputs/elasticsearch
[OUTPUT]
    Name  es
    Match *
    Host  <your.elasticsearch.endpoint>
    Port  9200
    HTTP_User <es_user>
    HTTP_Passwd <es_password>
    tls   On
    tls.verify  On
    Index <your-index-name>
    Type  _doc

動作確認

start

systemctl restart fluent-bit
systemctl enable fluent-bit

date > /path/to/your/log

kibanaのdevtoolsとかで叩いてみる

GET <your-index-name>/_search
{
  "query": {
    "match_all": {}
  }
}

Voilà!

Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
Sign upLogin
1
Help us understand the problem. What are the problem?