LoginSignup
106
86

More than 3 years have passed since last update.

@shimajiri(Ryota Shimajiri)inエルバークグループ

AWS CloudWatchのログフィルタパターンを分かりやすく解説

Last updated at Posted at 2020-04-27

こちらのサンプルログイベントメッセージを使用します。

1   [XXX] BENCHMARK : Running Start Crawl for Crawler TestCrawler2
2   [XXX] BENCHMARK : Classification complete, writing results to database mygluedatabase
3   [XXX] INFO : Crawler configured with SchemaChangePolicy {"UpdateBehavior":"UPDATE_IN_DATABASE","DeleteBehavior":"DEPRECATE_IN_DATABASE"}.
4   [XXX] INFO : Created table gluetest in database mygluedatabase
5   [XXX] BENCHMARK : Finished writing to Catalog
6   [XXX] BENCHMARK : Crawler has finished running and is in state READY

検索したい語句を""で囲むまたはそのまま入力することで、一致したログイベントメッセージを取得できます。

検索条件

BENCHMARK

または、

"BENCHMARK"

検索結果

1   [XXX] BENCHMARK : Running Start Crawl for Crawler TestCrawler2
2   [XXX] BENCHMARK : Classification complete, writing results to database mygluedatabase
5   [XXX] BENCHMARK : Finished writing to Catalog
6   [XXX] BENCHMARK : Crawler has finished running and is in state READY

""で囲むことで、スペースが存在する場合も単一文字列として認識させることが出来ます。

検索条件

"Running Start Crawl for Crawler"

検索結果

1   [XXX] BENCHMARK : Running Start Crawl for Crawler TestCrawler2

語句を複数並べて絞り込み検索を行うことも出来ます。

検索条件

BENCHMARK Crawler

または、

"BENCHMARK" "Crawler"

検索結果

1   [XXX] BENCHMARK : Running Start Crawl for Crawler TestCrawler2
6   [XXX] BENCHMARK : Crawler has finished running and is in state READY

語句の後に負符号 (-) をつけたあと、語句を指定すると検索条件から外すことができます。

検索条件

BENCHMARK - Crawler

または、

"BENCHMARK" - "Crawler"

検索結果

2   [XXX] BENCHMARK : Classification complete, writing results to database mygluedatabase
5   [XXX] BENCHMARK : Finished writing to Catalog

?を使ってORパターンマッチングを行うことも出来ます。

検索条件

?INFO ?Finished

検索結果

3   [XXX] INFO : Crawler configured with SchemaChangePolicy {"UpdateBehavior":"UPDATE_IN_DATABASE","DeleteBehavior":"DEPRECATE_IN_DATABASE"}.
4   [XXX] INFO : Created table gluetest in database mygluedatabase
5   [XXX] BENCHMARK : Finished writing to Catalog

JSON ログについて

こちらのサンプルJSONログを使用します。

1   {"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
2   {"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}
3   {"eventType": "UpdateTrail3", "sourceIPAddress": "120.0.0.1", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 500},{"name": "f", "id": 600}], "SomeObject": "a", "ThisFlag": true}
4   {"eventType": "UpdateTrail4", "sourceIPAddress": "120.0.0.2", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 501},{"name": "f", "id": 601}], "SomeObject": "b", "ThisFlag": true}

{}で囲み、keyに$をつけて検索します。

{ $.eventType = "UpdateTrail" }

検索結果

1   {"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}

~ではない場合の条件は=に!をつけることで可能です。

{ $.sourceIPAddress != 123.123.* }

検索結果

1   {"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
3   {"eventType": "UpdateTrail3", "sourceIPAddress": "120.0.0.1", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 500},{"name": "f", "id": 600}], "SomeObject": "a", "ThisFlag": true}
4   {"eventType": "UpdateTrail4", "sourceIPAddress": "120.0.0.2", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 501},{"name": "f", "id": 601}], "SomeObject": "b", "ThisFlag": true}

リスト内の検索も可能です。この場合は2のJSONが検索に引っかかります。

{ $.objectList[0].id = 3 }

検索結果

2   {"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}

数値検索では、>、<、>=、<=、=、 != 演算子を使用することが出来ます。

{ $.objectList[0].id < 5 }

検索結果

1   {"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
2   {"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}

数字ではアスタリスクも使用することが出来ます。

{ $.objectList[0].id = 5* }

検索結果

3   {"eventType": "UpdateTrail3", "sourceIPAddress": "120.0.0.1", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 500},{"name": "f", "id": 600}], "SomeObject": "a", "ThisFlag": true}
4   {"eventType": "UpdateTrail4", "sourceIPAddress": "120.0.0.2", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 501},{"name": "f", "id": 601}], "SomeObject": "b", "ThisFlag": true}

値がNULLの場合はIS NULLで引っ掛けることが出来ます。

{ $.SomeObject IS NULL }
*小文字でもOK 
{ $.SomeObject is null }

検索結果

1   {"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
2   {"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}

値が真偽値の場合はIS TRUE または IS FALSEで引っ掛けることが出来ます。

{ $.ThisFlag IS TRUE }
小文字でもOK 
{ $.ThisFlag is true }

検索結果

1   {"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
3   {"eventType": "UpdateTrail3", "sourceIPAddress": "120.0.0.1", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 500},{"name": "f", "id": 600}], "SomeObject": "a", "ThisFlag": true}
4   {"eventType": "UpdateTrail4", "sourceIPAddress": "120.0.0.2", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 501},{"name": "f", "id": 601}], "SomeObject": "b", "ThisFlag": true}

JSONの複合条件について

括弧()、OR (||) と AND (&&) を使用して複合式も可能です。

構文は演算子の標準の順序に従い、() > && > || となります。

OR (||) 条件

{ ( $.SomeObject IS NULL ) || ( $.ThisFlag IS FALSE ) }

検索結果

1   {"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
2   {"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}

AND (&&)条件

{ ( $.SomeObject IS NULL ) && ( $.ThisFlag IS TRUE ) }

検索結果

1   {"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}

スペース区切りで出力されるログから値を取得する

スペース区切りの項目ごとが何を示しているのか指定し、
その項目ごとに条件を指定して、値を取得することができます。

こちらのサンプルログを使用します。

1   frank   10/Oct/2000:13:25:15 -0700  GET /apache_pb.gif HTTP/1.0 200 1534    127.0.0.1   -
2   frank   10/Oct/2000:13:35:22 -0700  GET /apache_pb.gif HTTP/1.0 500 5324    127.0.0.1   -
3   frank   10/Oct/2000:13:50:35 -0700  GET /apache_pb.gif HTTP/1.0 200 4355    127.0.0.1   -
4   frank   10/Oct/2000:13:50:40 -0700  GET /apache_pb.gif HTTP/1.0 401 5423    127.0.0.1   -

フィールド数が不明な場合は、省略符号 (…) を使用した省略通知を使用できます。

前方を省略した検索条件

[..., status_code, bytes]

検索結果

イベント番号  $1  $2  $3  $4  $5  $6  $7  $8  $9  $bytes  $status_code
1   127.0.0.1   -   frank   10/Oct/2000:13:25:15 -0700  GET /apache_pb.gif HTTP/1.0 1534    200
2   127.0.0.1   -   frank   10/Oct/2000:13:35:22 -0700  GET /apache_pb.gif HTTP/1.0 5324    500
3   127.0.0.1   -   frank   10/Oct/2000:13:50:35 -0700  GET /apache_pb.gif HTTP/1.0 4355    200
4   127.0.0.1   -   frank   10/Oct/2000:13:50:40 -0700  GET /apache_pb.gif HTTP/1.0 5423    401

間を省略した検索条件

[ip, user, ..., status_code, bytes]

検索結果

イベント番号  $3  $4  $5  $6  $7  $8  $9  $bytes  $ip $status_code    $user
1   frank   10/Oct/2000:13:25:15 -0700  GET /apache_pb.gif HTTP/1.0 1534    127.0.0.1   200 -
2   frank   10/Oct/2000:13:35:22 -0700  GET /apache_pb.gif HTTP/1.0 5324    127.0.0.1   500 -
3   frank   10/Oct/2000:13:50:35 -0700  GET /apache_pb.gif HTTP/1.0 4355    127.0.0.1   200 -
4   frank   10/Oct/2000:13:50:40 -0700  GET /apache_pb.gif HTTP/1.0 5423    127.0.0.1   401 -

後方を省略した検索条件

[ip, user, ...]

検索結果

イベント番号  $3  $4  $5  $6  $7  $ip $user
1   frank   10/Oct/2000:13:25:15 -0700  GET /apache_pb.gif HTTP/1.0 200 1534    127.0.0.1   -
2   frank   10/Oct/2000:13:35:22 -0700  GET /apache_pb.gif HTTP/1.0 500 5324    127.0.0.1   -
3   frank   10/Oct/2000:13:50:35 -0700  GET /apache_pb.gif HTTP/1.0 200 4355    127.0.0.1   -
4   frank   10/Oct/2000:13:50:40 -0700  GET /apache_pb.gif HTTP/1.0 401 5423    127.0.0.1   -

条件を指定して検索することも出来ます。
>、<、>=、<=、=, != 演算子やアスタリスク、OR (||) と AND (&&)も使用することが出来ます。

status_codeが200である場合の検索条件

[ip, user, username, timestamp, request, status_code = 200, bytes]

検索結果

1   1534    127.0.0.1   GET /apache_pb.gif HTTP/1.0 200 10/Oct/2000:13:25:15 -0700  -   frank
3   4355    127.0.0.1   GET /apache_pb.gif HTTP/1.0 200 10/Oct/2000:13:50:35 -0700  -   frank

こちらはOR (||) を試しています。
status_codeが200、または500である場合の検索条件

[ip, user, username, timestamp, request, status_code = 200 || status_code = 500, bytes]

検索結果

1   1534    127.0.0.1   GET /apache_pb.gif HTTP/1.0 200 10/Oct/2000:13:25:15 -0700  -   frank
2   5324    127.0.0.1   GET /apache_pb.gif HTTP/1.0 500 10/Oct/2000:13:35:22 -0700  -   frank
3   4355    127.0.0.1   GET /apache_pb.gif HTTP/1.0 200 10/Oct/2000:13:50:35 -0700  -   frank

参考

フィルターとパターンの構文

106
86
1

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
106
86