111
88

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

AWS CloudWatchのログフィルタパターンを分かりやすく解説

Last updated at Posted at 2020-04-27

こちらのサンプルログイベントメッセージを使用します。

1	[XXX] BENCHMARK : Running Start Crawl for Crawler TestCrawler2
2	[XXX] BENCHMARK : Classification complete, writing results to database mygluedatabase
3	[XXX] INFO : Crawler configured with SchemaChangePolicy {"UpdateBehavior":"UPDATE_IN_DATABASE","DeleteBehavior":"DEPRECATE_IN_DATABASE"}.
4	[XXX] INFO : Created table gluetest in database mygluedatabase
5	[XXX] BENCHMARK : Finished writing to Catalog
6	[XXX] BENCHMARK : Crawler has finished running and is in state READY

###検索したい語句を""で囲むまたはそのまま入力することで、一致したログイベントメッセージを取得できます。

検索条件

BENCHMARK

または、

"BENCHMARK"

検索結果

1	[XXX] BENCHMARK : Running Start Crawl for Crawler TestCrawler2
2	[XXX] BENCHMARK : Classification complete, writing results to database mygluedatabase
5	[XXX] BENCHMARK : Finished writing to Catalog
6	[XXX] BENCHMARK : Crawler has finished running and is in state READY

###""で囲むことで、スペースが存在する場合も単一文字列として認識させることが出来ます。

検索条件

"Running Start Crawl for Crawler"

検索結果

1	[XXX] BENCHMARK : Running Start Crawl for Crawler TestCrawler2

###語句を複数並べて絞り込み検索を行うことも出来ます。

検索条件

BENCHMARK Crawler

または、

"BENCHMARK" "Crawler"

検索結果

1	[XXX] BENCHMARK : Running Start Crawl for Crawler TestCrawler2
6	[XXX] BENCHMARK : Crawler has finished running and is in state READY

###語句の後に負符号 (-) をつけたあと、語句を指定すると検索条件から外すことができます。

検索条件

BENCHMARK - Crawler

または、

"BENCHMARK" - "Crawler"

検索結果

2	[XXX] BENCHMARK : Classification complete, writing results to database mygluedatabase
5	[XXX] BENCHMARK : Finished writing to Catalog

###?を使ってORパターンマッチングを行うことも出来ます。

検索条件

?INFO ?Finished

検索結果

3	[XXX] INFO : Crawler configured with SchemaChangePolicy {"UpdateBehavior":"UPDATE_IN_DATABASE","DeleteBehavior":"DEPRECATE_IN_DATABASE"}.
4	[XXX] INFO : Created table gluetest in database mygluedatabase
5	[XXX] BENCHMARK : Finished writing to Catalog

##JSON ログについて

こちらのサンプルJSONログを使用します。

1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
2	{"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}
3	{"eventType": "UpdateTrail3", "sourceIPAddress": "120.0.0.1", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 500},{"name": "f", "id": 600}], "SomeObject": "a", "ThisFlag": true}
4	{"eventType": "UpdateTrail4", "sourceIPAddress": "120.0.0.2", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 501},{"name": "f", "id": 601}], "SomeObject": "b", "ThisFlag": true}

###{}で囲み、keyに$をつけて検索します。

{ $.eventType = "UpdateTrail" }

検索結果

1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}

###~ではない場合の条件は=に!をつけることで可能です。

{ $.sourceIPAddress != 123.123.* }

検索結果

1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
3	{"eventType": "UpdateTrail3", "sourceIPAddress": "120.0.0.1", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 500},{"name": "f", "id": 600}], "SomeObject": "a", "ThisFlag": true}
4	{"eventType": "UpdateTrail4", "sourceIPAddress": "120.0.0.2", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 501},{"name": "f", "id": 601}], "SomeObject": "b", "ThisFlag": true}

###リスト内の検索も可能です。この場合は2のJSONが検索に引っかかります。

{ $.objectList[0].id = 3 }

検索結果

2	{"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}

###数値検索では、>、<、>=、<=、=、 != 演算子を使用することが出来ます。

{ $.objectList[0].id < 5 }

検索結果

1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
2	{"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}

###数字ではアスタリスクも使用することが出来ます。

{ $.objectList[0].id = 5* }

検索結果

3	{"eventType": "UpdateTrail3", "sourceIPAddress": "120.0.0.1", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 500},{"name": "f", "id": 600}], "SomeObject": "a", "ThisFlag": true}
4	{"eventType": "UpdateTrail4", "sourceIPAddress": "120.0.0.2", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 501},{"name": "f", "id": 601}], "SomeObject": "b", "ThisFlag": true}

###値がNULLの場合はIS NULLで引っ掛けることが出来ます。

{ $.SomeObject IS NULL }
*小文字でもOK 
{ $.SomeObject is null }

検索結果

1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
2	{"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}

###値が真偽値の場合はIS TRUE または IS FALSEで引っ掛けることが出来ます。

{ $.ThisFlag IS TRUE }
小文字でもOK 
{ $.ThisFlag is true }

検索結果

1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
3	{"eventType": "UpdateTrail3", "sourceIPAddress": "120.0.0.1", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 500},{"name": "f", "id": 600}], "SomeObject": "a", "ThisFlag": true}
4	{"eventType": "UpdateTrail4", "sourceIPAddress": "120.0.0.2", "arrayKey": ["value", "another value"], "objectList": [{ "name": "e", "id": 501},{"name": "f", "id": 601}], "SomeObject": "b", "ThisFlag": true}

##JSONの複合条件について

###括弧()、OR (||) と AND (&&) を使用して複合式も可能です。
構文は演算子の標準の順序に従い、() > && > || となります。

###OR (||) 条件

{ ( $.SomeObject IS NULL ) || ( $.ThisFlag IS FALSE ) }

検索結果

1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}
2	{"eventType": "UpdateTrail2", "sourceIPAddress": "123.123.123.123", "arrayKey": ["value", "another value"], "objectList": [{ "name": "c", "id": 3},{"name": "d", "id": 4}], "SomeObject": null, "ThisFlag": false}

###AND (&&)条件

{ ( $.SomeObject IS NULL ) && ( $.ThisFlag IS TRUE ) }

検索結果

1	{"eventType": "UpdateTrail", "sourceIPAddress": "111.111.111.111", "arrayKey": ["value", "another value"], "objectList": [{ "name": "a", "id": 1},{"name": "b", "id": 2}], "SomeObject": null, "ThisFlag": true}

##スペース区切りで出力されるログから値を取得する

スペース区切りの項目ごとが何を示しているのか指定し、
その項目ごとに条件を指定して、値を取得することができます。

こちらのサンプルログを使用します。

1   frank   10/Oct/2000:13:25:15 -0700  GET /apache_pb.gif HTTP/1.0 200 1534    127.0.0.1   -
2   frank   10/Oct/2000:13:35:22 -0700  GET /apache_pb.gif HTTP/1.0 500 5324    127.0.0.1   -
3   frank   10/Oct/2000:13:50:35 -0700  GET /apache_pb.gif HTTP/1.0 200 4355    127.0.0.1   -
4   frank   10/Oct/2000:13:50:40 -0700  GET /apache_pb.gif HTTP/1.0 401 5423    127.0.0.1   -

###フィールド数が不明な場合は、省略符号 (…) を使用した省略通知を使用できます。

前方を省略した検索条件

[..., status_code, bytes]

検索結果

イベント番号	$1	$2	$3	$4	$5	$6	$7	$8	$9	$bytes	$status_code
1	127.0.0.1	-	frank	10/Oct/2000:13:25:15 -0700	GET /apache_pb.gif HTTP/1.0	1534	200
2	127.0.0.1	-	frank	10/Oct/2000:13:35:22 -0700	GET /apache_pb.gif HTTP/1.0	5324	500
3	127.0.0.1	-	frank	10/Oct/2000:13:50:35 -0700	GET /apache_pb.gif HTTP/1.0	4355	200
4	127.0.0.1	-	frank	10/Oct/2000:13:50:40 -0700	GET /apache_pb.gif HTTP/1.0	5423	401

間を省略した検索条件

[ip, user, ..., status_code, bytes]

検索結果

イベント番号	$3	$4	$5	$6	$7	$8	$9	$bytes	$ip	$status_code	$user
1	frank	10/Oct/2000:13:25:15 -0700	GET /apache_pb.gif HTTP/1.0	1534	127.0.0.1	200	-
2	frank	10/Oct/2000:13:35:22 -0700	GET /apache_pb.gif HTTP/1.0	5324	127.0.0.1	500	-
3	frank	10/Oct/2000:13:50:35 -0700	GET /apache_pb.gif HTTP/1.0	4355	127.0.0.1	200	-
4	frank	10/Oct/2000:13:50:40 -0700	GET /apache_pb.gif HTTP/1.0	5423	127.0.0.1	401	-

後方を省略した検索条件

[ip, user, ...]

検索結果

イベント番号	$3	$4	$5	$6	$7	$ip	$user
1	frank	10/Oct/2000:13:25:15 -0700	GET /apache_pb.gif HTTP/1.0	200	1534	127.0.0.1	-
2	frank	10/Oct/2000:13:35:22 -0700	GET /apache_pb.gif HTTP/1.0	500	5324	127.0.0.1	-
3	frank	10/Oct/2000:13:50:35 -0700	GET /apache_pb.gif HTTP/1.0	200	4355	127.0.0.1	-
4	frank	10/Oct/2000:13:50:40 -0700	GET /apache_pb.gif HTTP/1.0	401	5423	127.0.0.1	-

条件を指定して検索することも出来ます。
>、<、>=、<=、=, != 演算子やアスタリスク、OR (||) と AND (&&)も使用することが出来ます。

status_codeが200である場合の検索条件

[ip, user, username, timestamp, request, status_code = 200, bytes]

検索結果

1	1534	127.0.0.1	GET /apache_pb.gif HTTP/1.0	200	10/Oct/2000:13:25:15 -0700	-	frank
3	4355	127.0.0.1	GET /apache_pb.gif HTTP/1.0	200	10/Oct/2000:13:50:35 -0700	-	frank

こちらはOR (||) を試しています。
status_codeが200、または500である場合の検索条件

[ip, user, username, timestamp, request, status_code = 200 || status_code = 500, bytes]

検索結果

1	1534	127.0.0.1	GET /apache_pb.gif HTTP/1.0	200	10/Oct/2000:13:25:15 -0700	-	frank
2	5324	127.0.0.1	GET /apache_pb.gif HTTP/1.0	500	10/Oct/2000:13:35:22 -0700	-	frank
3	4355	127.0.0.1	GET /apache_pb.gif HTTP/1.0	200	10/Oct/2000:13:50:35 -0700	-	frank

##参考
フィルターとパターンの構文

111
88
1

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
111
88

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?