IBM Cloud Virtual Server for Classic で Windows イベントログを IBM Log Analysis with LogDNA に送る

概要

Fluentd | LogDNA Docs のガイドに沿って、 Windows Server から IBM Log Analysis with LogDNA にイベントログを送る方法を確認します。

事前準備

• IBM Log Analysis with LogDNA サービスインスタンス作成
• IBM Cloud Virtual Server for Classic プロビジョニング
  • Windows Server 2019 STD 64 bit を使います

Ingestion Key の取得

作成済みの IBM Log Analysis with LogDNA サービスインスタンスにログを送付するための「Ingestion Key」（32桁の英数字）を以下のように取得します。

Fluentd インストール

こちらにある最新版の td-agent v4.0.1 を使用します。

• Download | Fluentd

PowerShell で以下のコマンドを実行します。

powershell

Invoke-WebRequest -Uri "http://packages.treasuredata.com.s3.amazonaws.com/4/windows/td-agent-4.0.1-x64.msi" -Outfile td-agent.msi ; & .\td-agent.msi /passive

インストール後、設定ファイルがデフォルトディレクトリに配置されたことを確認します。

powershell

PS C:\Users\Administrator> Get-ChildItem C:\opt\td-agent\etc\td-agent\


    Directory: C:\opt\td-agent\etc\td-agent


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        8/17/2020   7:52 AM           2562 td-agent.conf

また、この時点でサービスが追加・起動されることも確認できます。

Install by .msi Installer (Windows) - Fluentd
Since version 4.0.0, td-agent is registered as a Windows service permanently by the msi installer.

powershell

services.msc

Fluentd Windows Service のログは以下の場所から確認できます。

Install by .msi Installer (Windows) - Fluentd
The log file will be located at C:/opt/td-agent/td-agent.log

powershell

PS C:\Users\Administrator> Get-Content C:\opt\td-agent\td-agent.log
2021-02-09 14:22:42 -0600 [info]: parsing config file is succeeded path="C:\\opt\\td-agent\\etc\\td-agent\\td-agent.conf"
2021-02-09 14:22:42 -0600 [info]: gem 'fluent-plugin-elasticsearch' version '4.1.1'
2021-02-09 14:22:42 -0600 [info]: gem 'fluent-plugin-kafka' version '0.14.1'
2021-02-09 14:22:42 -0600 [info]: gem 'fluent-plugin-parser-winevt_xml' version '0.2.2'
2021-02-09 14:22:42 -0600 [info]: gem 'fluent-plugin-prometheus' version '1.8.2'
2021-02-09 14:22:42 -0600 [info]: gem 'fluent-plugin-prometheus_pushgateway' version '0.0.2'
2021-02-09 14:22:42 -0600 [info]: gem 'fluent-plugin-record-modifier' version '2.1.0'
2021-02-09 14:22:42 -0600 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.3.0'
2021-02-09 14:22:42 -0600 [info]: gem 'fluent-plugin-s3' version '1.4.0'
2021-02-09 14:22:42 -0600 [info]: gem 'fluent-plugin-td' version '1.1.0'
2021-02-09 14:22:42 -0600 [info]: gem 'fluent-plugin-webhdfs' version '1.2.5'
2021-02-09 14:22:42 -0600 [info]: gem 'fluent-plugin-windows-eventlog' version '0.7.0'
2021-02-09 14:22:42 -0600 [info]: gem 'fluentd' version '1.11.2'
2021-02-09 14:22:43 -0600 [warn]: [output_td] Use different plugin for secondary. Check the plugin works with primary like secondary_file primary="Fluent::Plugin::TreasureDataLogOutput" secondary="Fluent::Plugin::FileOutput"
2021-02-09 14:22:44 -0600 [info]: using configuration file: <ROOT>
  <match td.*.*>
    @type tdlog
    @id output_td
    apikey xxxxxx
    auto_create_table
    <buffer>
      @type "file"
      path "/var/log/td-agent/buffer/td"
    </buffer>
    <secondary>
      @type "file"
      path "/var/log/td-agent/failed_records"
      <buffer time>
        path /var/log/td-agent/failed_records
      </buffer>
    </secondary>
  </match>
  <match debug.**>
    @type stdout
    @id output_stdout
  </match>
  <source>
    @type forward
    @id input_forward
  </source>
  <source>
    @type http
    @id input_http
    port 8888
  </source>
  <source>
    @type debug_agent
    @id input_debug_agent
    bind "127.0.0.1"
    port 24230
  </source>
</ROOT>
2021-02-09 14:22:44 -0600 [info]: starting fluentd-1.11.2 pid=5480 ruby="2.7.1"
2021-02-09 14:22:44 -0600 [info]: spawn command to main:  cmdline=["C:/opt/td-agent/bin/ruby.exe", "-Eascii-8bit:ascii-8bit", "C:/opt/td-agent/bin/fluentd", "-c", "C:\\opt\\td-agent\\etc\\td-agent\\td-agent.conf", "-o", "C:\\opt\\td-agent\\td-agent.log", "-x", "fluentdwinsvc", "--under-supervisor"]
2021-02-09 14:22:49 -0600 [info]: adding match pattern="td.*.*" type="tdlog"
2021-02-09 14:22:49 -0600 [warn]: #0 [output_td] Use different plugin for secondary. Check the plugin works with primary like secondary_file primary="Fluent::Plugin::TreasureDataLogOutput" secondary="Fluent::Plugin::FileOutput"
2021-02-09 14:22:49 -0600 [info]: adding match pattern="debug.**" type="stdout"
2021-02-09 14:22:50 -0600 [info]: adding source type="forward"
2021-02-09 14:22:50 -0600 [info]: adding source type="http"
2021-02-09 14:22:50 -0600 [info]: adding source type="debug_agent"
2021-02-09 14:22:50 -0600 [info]: #0 starting fluentd worker pid=6476 ppid=5480 worker=0
2021-02-09 14:22:51 -0600 [info]: #0 [input_debug_agent] listening dRuby uri="druby://127.0.0.1:24230" object="Fluent::Engine" worker=0
2021-02-09 14:22:51 -0600 [info]: #0 [input_forward] listening port port=24224 bind="0.0.0.0"
2021-02-09 14:22:51 -0600 [info]: #0 fluentd worker is now running worker=0

Fluentd 設定ファイル準備

Fluentd Windows Service が使う設定ファイルの場所は C:/opt/td-agent/etc/td-agent/td-agent.conf がデフォルトです。
オリジナルのデフォルト設定ファイルを別ファイルにコピーしバックアップとして保存します。

powershell

Copy-Item C:\opt\td-agent\etc\td-agent\td-agent.conf C:\opt\td-agent\etc\td-agent\td-agent.conf.bak
Get-ChildItem C:\opt\td-agent\etc\td-agent\

先ほど取得した「Ingestion Key」を変数にセットします。

powershell

$INGESTION_KEY = "YOUR_INGESTION_KEY"
$INGESTION_KEY

Ingestion 先を「Ingestion endpoints」から設定します。
今回は IBM Log Analysis with LogDNA 東京リージョンのプライベートエンドポイントを設定します。

powershell

$INGESTER_DOMAIN = "https://logs.private.jp-tok.logging.cloud.ibm.com"
$INGESTER_DOMAIN

ヒアドキュメントを使って以下の内容を設定ファイルに追記します。

powershell

$TD_AGENT_CONF = @"
<source>
  @type windows_eventlog2
  @id windows_eventlog2
  channels application,system,setup,security # Also be able to use `<subscribe>` directive.
  read_existing_events false
  read_interval 2
  tag winevt.raw
  render_as_xml false       # default is false.
  rate_limit 200            # default is -1(Winevt::EventLog::Subscribe::RATE_INFINITE).
  # preserve_qualifiers_on_hash true # default is false.
  # read_all_channels false # default is false.
  # description_locale en_US # default is nil. It means that system locale is used for obtaining description.
  <storage>
    @type local             # @type local is the default.
    persistent true         # default is true. Set to false to use in-memory storage.
    path ./tmp/storage.json # This is required when persistent is true.
                            # Or, please consider using <system> section's `root_dir` parameter.
  </storage>
  <parse>
    @type winevt_xml # @type winevt_xml is the default. winevt_xml and none parsers are supported for now.
    # When set up it as true, this plugin preserves "Qualifiers" and "EventID" keys.
    # When set up it as false, this plugin calculates actual "EventID" from "Qualifiers" and removing "Qualifiers".
    # With the following equation:
    # (EventID & 0xffff) | (Qualifiers & 0xffff) << 16
    preserve_qualifiers true
  </parse>
  # <subscribe>
  #   channles application, system, setup, security
  #   read_existing_events false # read_existing_events should be applied each of subscribe directive(s)
  # </subscribe>
</source>
 
<match **>
  @type logdna
  api_key $INGESTION_KEY   # paste your api key here (required)
  ingester_domain $INGESTER_DOMAIN   #Replace with your specific LogDNA endpoint
  hostname "#{Socket.gethostname}"      #your hostname (required)
  app my_app                            # replace with your app name
  #mac C0:FF:EE:C0:FF:EE                    # optional mac address
  #ip 127.0.0.1                                 # optional ip address
  #tags web,dev                                 # optional tags
  slow_flush_log_threshold 30.0
  request_timeout 30000 ms                  # optional timeout for upload request, supports seconds (s, default) and milliseconds (ms) suffixes, default 30 seconds
  buffer_chunk_limit 1m                         # do not increase past 8m (8MB) or your logs will be rejected by our server.
  flush_at_shutdown true                        # only needed with file buffer
</match>
"@

$TD_AGENT_CONF | Add-Content C:\opt\td-agent\etc\td-agent\td-agent.conf
Get-Content C:\opt\td-agent\etc\td-agent\td-agent.conf

LogDNA Fluentd プラグインインストール

IBM Log Analysis with LogDNA 用のプラグインをインストールします。

powershell

Start-Process cmd "/c C:\opt\td-agent\bin\td-agent-gem install fluent-plugin-logdna"

Fluentd Windows Service 再起動

log ファイル更新をモニターするために PowerShell のウインドウをもうひとつ立ち上げておき、以下のコマンドを実行します。

powershell

Get-Content C:\opt\td-agent\td-agent.log -Wait -Tail 10

もう一方の PowerShell のウインドウで Fluentd Windows Service を再起動します。

powershell

Restart-Service fluentdwinsvc

以下のようなログで、設定ファイルやプラグインが正しく読み込まれているかを確認できます。

powershell

2021-02-09 09:00:56 -0600 [info]: Received graceful stop
2021-02-09 09:00:57 -0600 [info]: Worker 0 finished with status 0
2021-02-09 09:01:08 -0600 [info]: parsing config file is succeeded path="C:\\opt\\td-agent\\etc\\td-agent\\td-agent.conf"
2021-02-09 09:01:08 -0600 [info]: gem 'fluent-plugin-elasticsearch' version '4.1.1'
2021-02-09 09:01:08 -0600 [info]: gem 'fluent-plugin-kafka' version '0.14.1'
2021-02-09 09:01:08 -0600 [info]: gem 'fluent-plugin-logdna' version '0.4.0'
2021-02-09 09:01:08 -0600 [info]: gem 'fluent-plugin-parser-winevt_xml' version '0.2.2'
2021-02-09 09:01:08 -0600 [info]: gem 'fluent-plugin-prometheus' version '1.8.2'
2021-02-09 09:01:08 -0600 [info]: gem 'fluent-plugin-prometheus_pushgateway' version '0.0.2'
2021-02-09 09:01:08 -0600 [info]: gem 'fluent-plugin-record-modifier' version '2.1.0'
2021-02-09 09:01:08 -0600 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.3.0'
2021-02-09 09:01:08 -0600 [info]: gem 'fluent-plugin-s3' version '1.4.0'
2021-02-09 09:01:08 -0600 [info]: gem 'fluent-plugin-td' version '1.1.0'
2021-02-09 09:01:08 -0600 [info]: gem 'fluent-plugin-webhdfs' version '1.2.5'
2021-02-09 09:01:08 -0600 [info]: gem 'fluent-plugin-windows-eventlog' version '0.7.0'
2021-02-09 09:01:08 -0600 [info]: gem 'fluentd' version '1.11.2'
2021-02-09 09:01:08 -0600 [warn]: [output_td] Use different plugin for secondary. Check the plugin works with primary like secondary_file primary="Fluent::Plugin::TreasureDataLogOutput" secondary="Fluent::Plugin::FileOutput"
2021-02-09 09:01:10 -0600 [warn]: define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead
2021-02-09 09:01:10 -0600 [info]: using configuration file: <ROOT>
  <match td.*.*>
    @type tdlog
    @id output_td
    apikey xxxxxx
    auto_create_table
    <buffer>
      @type "file"
      path "C:\\opt\\td-agent\\/var/log/td-agent/buffer/td"
    </buffer>
    <secondary>
      @type "file"
      path "C:\\opt\\td-agent\\/var/log/td-agent/failed_records"
      <buffer time>
        path C:\opt\td-agent\/var/log/td-agent/failed_records
      </buffer>
    </secondary>
  </match>
  <match debug.**>
    @type stdout
    @id output_stdout
  </match>
  <source>
    @type forward
    @id input_forward
  </source>
  <source>
    @type http
    @id input_http
    port 8888
  </source>
  <source>
    @type debug_agent
    @id input_debug_agent
    bind "127.0.0.1"
    port 24230
  </source>
  <source>
    @type windows_eventlog2
    @id windows_eventlog2
    channels application,system,setup,security
    read_existing_events false
    read_interval 2
    tag "winevt.raw"
    render_as_xml false
    rate_limit 200
    <storage>
      @type "local"
      persistent true
      path "./tmp/storage.json"
    </storage>
    <parse>
      @type "winevt_xml"
      preserve_qualifiers true
    </parse>
  </source>
  <match **>
    @type logdna
    api_key xxxxxx
    ingester_domain "https://logs.private.jp-tok.logging.cloud.ibm.com"
    hostname "khayama-win"
    app "my_app"
    slow_flush_log_threshold 30.0
    request_timeout "30000 ms"
    buffer_chunk_limit 1m
    flush_at_shutdown true
    <buffer>
      flush_mode interval
      retry_type exponential_backoff
      chunk_limit_size 1m
      flush_at_shutdown true
    </buffer>
  </match>
</ROOT>
2021-02-09 09:01:10 -0600 [info]: starting fluentd-1.11.2 pid=5588 ruby="2.7.1"
2021-02-09 09:01:10 -0600 [info]: spawn command to main:  cmdline=["C:/opt/td-agent/bin/ruby.exe", "-Eascii-8bit:ascii-8bit", "C:/opt/td-agent/bin/fluentd", "-c", "C:\\opt\\td-agent\\etc\\td-agent\\td-agent.conf", "-o", "C:\\opt\\td-agent\\td-agent.log", "-x", "fluentdwinsvc", "--under-supervisor"]
2021-02-09 09:01:15 -0600 [info]: adding match pattern="td.*.*" type="tdlog"
2021-02-09 09:01:15 -0600 [warn]: #0 [output_td] Use different plugin for secondary. Check the plugin works with primary like secondary_file primary="Fluent::Plugin::TreasureDataLogOutput" secondary="Fluent::Plugin::FileOutput"
2021-02-09 09:01:15 -0600 [info]: adding match pattern="debug.**" type="stdout"
2021-02-09 09:01:16 -0600 [info]: adding match pattern="**" type="logdna"
2021-02-09 09:01:16 -0600 [info]: adding source type="forward"
2021-02-09 09:01:16 -0600 [info]: adding source type="http"
2021-02-09 09:01:16 -0600 [info]: adding source type="debug_agent"
2021-02-09 09:01:16 -0600 [info]: adding source type="windows_eventlog2"
2021-02-09 09:01:17 -0600 [warn]: #0 define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead
2021-02-09 09:01:17 -0600 [info]: #0 starting fluentd worker pid=3952 ppid=5588 worker=0
2021-02-09 09:01:19 -0600 [info]: #0 [input_debug_agent] listening dRuby uri="druby://127.0.0.1:24230" object="Fluent::Engine" worker=0
2021-02-09 09:01:19 -0600 [info]: #0 [input_forward] listening port port=24224 bind="0.0.0.0"
2021-02-09 09:01:19 -0600 [info]: #0 fluentd worker is now running worker=0

IBM Log Analysis with LogDNA 確認

IBM Cloud カスタマーポータルから IBM Log Analysis with LogDNA ダッシュボードへのリンクをたどると
先ほどの Fluentd Windows Service 再起動時のログに一致するような内容が確認できます。

また、以下のコマンドでテスト用の Windows イベントログを作成できます。

powershell

Write-EventLog -LogName Application -Source Application -EventID 1234 -EntryType Information -Message "This is a test for IBM Log Analysis with LogDNA"

作成したテスト用の Windows イベントログが IBM Log Analysis with LogDNA ダッシュボードでも確認できます。
Config: Buffer Section - Fluentd の flush_interval (Default: 60s) に基づき、約 1 分おきに送られます。

参考

• fluentd Windows版をWindowsサービスとして動かす - Qiita
• td-agent 3.0 for Windows - Qiita
• PowerShellでtail -fを再現する - tech.guitarrapc.cóm
• 【Powershell】イベントログ を書き込みたい！監視のテストにも使える！ | ぴぐろぐ
• fluent/fluent-plugin-windows-eventlog: Fluentd plugin to collect windows event logs