概要
アクセスログをLTSVフォーマットに変更した。
既存のログと大きな違いは以下の２点
- タブ区切り
- ラベルが付けられる
今までgrepやawkを駆使して該当項目を抽出していたが、
lltsvコマンドをインストールすることにより、作業がグッと楽になった。
使い方
例えば以下のようなアクセスログがあるとする。
time_local:11/Dec/2017:12:09:25 +0900 request:"GET /admin/widget/ HTTP/1.1" http_x_forwarded_for:"-" uid_got:"-" req_body_start:""
time_local:11/Dec/2017:12:09:25 +0900 request:"GET /admin/widget/ HTTP/1.0" http_x_forwarded_for:"192.168.120.120" uid_got:"app_uid=060012AC5A99275A050070A502030303" req_body_start:""
time_local:11/Dec/2017:12:09:26 +0900 request:"GET /admin/event/ HTTP/1.1" http_x_forwarded_for:"-" uid_got:"-" req_body_start:""
time_local:11/Dec/2017:12:09:26 +0900 request:"GET /admin/event/ HTTP/1.0" http_x_forwarded_for:"192.168.120.120" uid_got:"app_uid=060012AC5A99275A050070A502030303" req_body_start:""
このログから
- time_local
- http_x_forwarded_for
のみを抽出したい場合、以下のようなコマンドで簡単に抽出できる
# tail access.log | lltsv -k time_local,http_x_forwarded_for
time_local:11/Dec/2017:12:09:22 +0900 http_x_forwarded_for:"-"
time_local:11/Dec/2017:12:09:22 +0900 http_x_forwarded_for:"192.168.120.120"
time_local:11/Dec/2017:12:09:22 +0900 http_x_forwarded_for:"-"
time_local:11/Dec/2017:12:09:22 +0900 http_x_forwarded_for:"192.168.120.120"
time_local:11/Dec/2017:12:09:24 +0900 http_x_forwarded_for:"-"
time_local:11/Dec/2017:12:09:24 +0900 http_x_forwarded_for:"192.168.120.120"
time_local:11/Dec/2017:12:09:25 +0900 http_x_forwarded_for:"-"
time_local:11/Dec/2017:12:09:25 +0900 http_x_forwarded_for:"192.168.120.120"
time_local:11/Dec/2017:12:09:26 +0900 http_x_forwarded_for:"-"
time_local:11/Dec/2017:12:09:26 +0900 http_x_forwarded_for:"192.168.120.120"
#
インストール方法
yum install wget -y
wget https://github.com/sonots/lltsv/releases/download/v0.3.0/lltsv_linux_amd64 -O lltsv
chmod a+x lltsv
mv lltsv /usr/sbin/.
参考情報
http://blog.livedoor.jp/sonots/archives/39628163.html
http://tech.mercari.com/entry/2017/05/09/152428