Edited at

fluentdからAmazon Elasticsearch ServiceにPOSTする

More than 3 years have passed since last update.

403エラーに悩まされたのでメモ。


結論

これと同じエラーで悩まされたが、以下の設定でfluent-plugin-aws-elasticsearch-serviceは正常に動いた。


fluentd側


fluent-plugin-aws-elasticsearch-serviceのインストール

sudo td-agent-gem install fluent-plugin-aws-elasticsearch-service


configサンプル

type_nameが無いと403エラーになる

<match **.**>

type copy

<store>
type aws-elasticsearch-service
type_name access_log
logstash_format true
include_tag_key true
tag_key @log_name
flush_interval 10s

<endpoint>
url https://search-***.ap-northeast-1.es.amazonaws.com
region ap-northeast-1
access_key_id ***
secret_access_key ***
</endpoint>
</store>
</match>


AWS ES側


policy

{

"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::***:user/***"
]
},
"Action": "es:*",
"Resource": "arn:aws:es:ap-northeast-1:***:domain/***/*"
}
]
}