Configuration example of Logstash syslog input module

　以下の流れのデータフローを構築するために、本記事ではLogstashのコンフィグレーションの設定方法について述べる。

[Logstash syslog] -> [Kafka] -> [Elasticsearch]

1. Logstashのインストールについて

Javaをインストールする

yum install java-1.8.0-openjdk
yum install java-1.8.0-openjdk-devel

鍵を取得する

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

リポジトリ情報を記述する

sudo cat <<EOF >> /etc/yum.repos.d/logstash.repo
[logstash-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF

インストールを実行する

sudo yum install logstash

起動コマンドはこちらから。

systemctl start logstash
systemctl status logstash  
systemctl enable logstash

以下は、LogstashのInputプラグインの全設定項目である。

input {
  syslog {
    ### port[number]
     # The port to listen on. Remember that ports less than 1024
      port => 514
    ### host[string]
     # The address to listen on. Default value is "0.0.0.0"
      # host => 127.0.0.1
    ### facility_labels[array]
     # Labels for facility levels. These are defined in RFC3164.
      # facility_labels => ["kernel", "user-level", "mail", "system", "security/authorization", "syslogd", "line printer", "network news", "UUCP", "clock", "security/authorization", "FTP", "NTP", "log audit", "log alert", "clock", "local0", "local1", "local2", "local3", "local4", "local5", "local6", "local7"]
    ### grok_pattern[string]
     # The default value should read and properly parse syslog lines which are fully compliant with RFC3164.
     # grok_pattern => "<%{POSINT:priority}>%{SYSLOGLINE}"
    ### locale[string]
     # Specify a locale to be used for date parsing using either IETF-BCP47 or POSIX language tag.
     # The locale is mostly necessary to be set for parsing month names (pattern with MMM) and weekday names.
      # locale => en-US
    ### proxy_protocol[boolean]
     # Proxy protocol support, only v1 is supported at this time
      # proxy_protocol => false
    ### severity_labels[array]
     # Labels for severity levels. These are defined in RFC3164.
      # Labels for severity levels. These are defined in RFC3164.
       # severity_labels => ["Emergency", "Alert", "Critical", "Error", "Warning", "Notice", "Informational", "Debug"]
    ### syslog_field[string]
     # Codecs process the data before the rest of the data is parsed.
      # syslog_field => "syslog"
    ### timezone[string]
     # Specify a time zone canonical ID to be used for date parsing. The valid IDs are listed on the [Joda.org available time zones page]
      # timezone => America
    ### use_labels[boolean]
     # Use label parsing for severity and facility levels.
      # use_labels => true
  }
}