LoginSignup
1
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 1 year has passed since last update.

@gaichi(gaichi)

EC2インスタンスの仮想OS上のログファイルをCloudwatch Logsに転送する設定手順

Last updated at Posted at 2023-10-28

これは何？

EC2インスタンス上で稼働している仮想OS上のログ(/var/log/syslogなど）を，Cloudwatch Logsに転送する設定手順です。
<参考>コマンドラインを使用した CloudWatch エージェントのインストール（公式）

環境（2023/10現在でテスト）

  • ubuntu2204のEC2インスタンス（AMIから展開）

IAMロールの作成 or 編集

EC2インスタンスに割り当てるIAMロールに，以下のポリシーを追加します。このポリシーによって，Cloudwatch Logsにログ送信できる権限をEC2インスタンスに付与できます。

仮想OSへCloudwatch Agentをインストール

EC2インスタンス作成時に標準のAMIから展開した場合は，すでにCloudwatch Agentが導入済の場合があります。この場合には，追加でインストールする必要はありません。ubuntu2204をAMIから展開した場合にはすでに導入済でした。
<参考>CloudWatchエージェントのダウンロードおよび設定(公式)

collectdを導入

$ sudo apt install collectd

<参考>https://knhk.hatenablog.com/entry/2018/10/29/112956

Cloudwatch Agentの設定ウィザード実行

仮想OSにログインし，Cloudwatch Agentの設定ウィザードを実行します。このウィザードではログ送信の設定に加え，メトリクス送信に関するパラメータも設定できます。

$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

================================================================
= Welcome to the Amazon CloudWatch Agent Configuration Manager =
=                                                              =
= CloudWatch Agent allows you to collect metrics and logs from =
= your host and send them to CloudWatch. Additional CloudWatch =
= charges may apply.                                           =
================================================================
On which OS are you planning to use the agent?
1. linux
2. windows
3. darwin
default choice: [1]: 
※デフォルトの1でOK。今回はUbuntuなので1です。

Trying to fetch the default region based on ec2 metadata...
I! imds retry client will retry 1 timesAre you using EC2 or On-Premises hosts?
1. EC2
2. On-Premises
default choice: [1]: 
※デフォルトの1でOK。今回はEC2インスタンスなので1です。オンプレサーバからCloudwatchにメトリクスやログ送信する場合には2です。

Which user are you planning to run the agent?
1. root
2. cwagent
3. others
default choice: [1]: ※デフォルトでOK

Do you want to turn on StatsD daemon?
1. yes
2. no
default choice: [1]: ※デフォルトでOK

Which port do you want StatsD daemon to listen to?
default choice: [8125] ※デフォルトでOK

What is the collect interval for StatsD daemon?
1. 10s
2. 30s
3. 60s
default choice: [1]: ※デフォルトでOK

What is the aggregation interval for metrics collected by StatsD daemon?
1. Do not aggregate
2. 10s
3. 30s
4. 60s
default choice: [4]: ※デフォルトでOK

Do you want to monitor metrics from CollectD? WARNING: CollectD must be installed or the Agent will fail to start
1. yes
2. no
default choice: [1]: ※デフォルトでOK

Do you want to monitor any host metrics? e.g. CPU, memory, etc.
1. yes
2. no
default choice: [1]: ※デフォルトでOK

Do you want to monitor cpu metrics per core?
1. yes
2. no
default choice: [1]: ※デフォルトでOK

Do you want to add ec2 dimensions (ImageId, InstanceId, InstanceType, AutoScalingGroupName) into all of your metrics if the info is available?
1. yes
2. no
default choice: [1]: ※デフォルトでOK

Do you want to aggregate ec2 dimensions (InstanceId)?
1. yes
2. no
default choice: [1]: ※デフォルトでOK

Would you like to collect your metrics at high resolution (sub-minute resolution)? This enables sub-minute resolution for all metrics, but you can customize for specific metrics in the output json file.
1. 1s
2. 10s
3. 30s
4. 60s
default choice: [4]: ※デフォルトでOK

Which default metrics config do you want?
1. Basic
2. Standard
3. Advanced
4. None
default choice: [1]: ※デフォルトでOK

Current config as follows:
{
        "agent": {
                "metrics_collection_interval": 60,
                "run_as_user": "root"
        },
        "metrics": {
                "aggregation_dimensions": [
                        [
                                "InstanceId"
                        ]
                ],
                "append_dimensions": {
                        "AutoScalingGroupName": "${aws:AutoScalingGroupName}",
                        "ImageId": "${aws:ImageId}",
                        "InstanceId": "${aws:InstanceId}",
                        "InstanceType": "${aws:InstanceType}"
                },
                "metrics_collected": {
                        "collectd": {
                                "metrics_aggregation_interval": 60
                        },
                        "disk": {
                                "measurement": [
                                        "used_percent"
                                ],
                                "metrics_collection_interval": 60,
                                "resources": [
                                        "*"
                                ]
                        },
                        "mem": {
                                "measurement": [
                                        "mem_used_percent"
                                ],
                                "metrics_collection_interval": 60
                        },
                        "statsd": {
                                "metrics_aggregation_interval": 60,
                                "metrics_collection_interval": 10,
                                "service_address": ":8125"
                        }
                }
        }
}
Are you satisfied with the above config? Note: it can be manually customized after the wizard completes to add additional items.
1. yes
2. no
default choice: [1]: ※デフォルトでOK

Do you have any existing CloudWatch Log Agent (http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html) configuration file to import for migration?
1. yes
2. no
default choice: [2]: ※デフォルトでOK

Do you want to monitor any log files?
1. yes
2. no
default choice: [1]: 
※デフォルトでOK。ここからログ送信の設定です。

Log file path:
/var/log/syslog　 ※転送したいログの絶対パスを入力
Log group name:
default choice: [syslog]　 ※CloudwatchLogs上でのロググループ名を入力

Log stream name:
default choice: [{instance_id}] ※デフォルトでOK。ログのストリーム名をインスタンスIDで表示する設定です。サーバ名とかでもいいかもしれません。

Log Group Retention in days
1. -1
2. 1
3. 3
4. 5
5. 7
6. 14
7. 30
8. 60
9. 90
10. 120
11. 150
12. 180
13. 365
14. 400
15. 545
16. 731
17. 1096
18. 1827
19. 2192
20. 2557
21. 2922
22. 3288
23. 3653
default choice: [1]: 
※ログのリテンション期間を設定。-1は無期限（＝削除しない）

Do you want to specify any additional log files to monitor?
1. yes
2. no
default choice: [1]: ※追加のログがあれば1，これで最後なら2

Do you want the CloudWatch agent to also retrieve X-ray traces?
1. yes
2. no
default choice: [1]: 
※デフォルトでOK。X-rayを使うのならIAMロールのポリシーにAWSXRayDaemonWriteAccessを追加する必要があるはずです。（←このあたりはまだ試していません）

Do you have an existing X-Ray Daemon configuration file to import for migration?
1. yes
2. no
default choice: [1]: ※デフォルトでOK

What is the file path for the existing X-Ray Daemon configuration file?

There was an error reading X-Ray Daemon config file. Using default traces configurations
Current Traces Configurations:
{
        "traces": {
                "traces_collected": {
                        "xray": {
                                "bind_address": "127.0.0.1:2000",
                                "tcp_proxy": {
                                        "bind_address": "127.0.0.1:2000"
                                }
                        }
                },
                "concurrency": 8,
                "buffer_size_mb": 3,
                "insecure": false
        }
}
Enter a number of the field you would like to update (or 0 to exit)
0: Keep this configuration and exit
1: UDP BindAddress
2: TCP BindAddress
3: concurrency
4: buffer_size_mb
5: resource_arn
6: local_mode
7: insecure
8: role_arn
9: endpoint_override
10: region_override
11: proxy_override

※0でよいはず。
Existing config JSON identified and copied to:  /opt/aws/amazon-cloudwatch-agent/etc/backup-configs
Saved config file to /opt/aws/amazon-cloudwatch-agent/bin/config.json successfully.
Current config as follows:
{
        "agent": {
                "metrics_collection_interval": 60,
                "run_as_user": "root"
        },
        "logs": {
                "logs_collected": {
                        "files": {
                                "collect_list": [
                                        {
                                                "file_path": "/var/log/syslog",
                                                "log_group_name": "syslog",
                                                "log_stream_name": "{instance_id}",
                                                "retention_in_days": 90
                                        }
                                ]
                        }
                }
        },
        "metrics": {
                "aggregation_dimensions": [
                        [
                                "InstanceId"
                        ]
                ],
                "append_dimensions": {
                        "AutoScalingGroupName": "${aws:AutoScalingGroupName}",
                        "ImageId": "${aws:ImageId}",
                        "InstanceId": "${aws:InstanceId}",
                        "InstanceType": "${aws:InstanceType}"
                },
                "metrics_collected": {
                        "collectd": {
                                "metrics_aggregation_interval": 60
                        },
                        "disk": {
                                "measurement": [
                                        "used_percent"
                                ],
                                "metrics_collection_interval": 60,
                                "resources": [
                                        "*"
                                ]
                        },
                        "mem": {
                                "measurement": [
                                        "mem_used_percent"
                                ],
                                "metrics_collection_interval": 60
                        },
                        "statsd": {
                                "metrics_aggregation_interval": 60,
                                "metrics_collection_interval": 10,
                                "service_address": ":8125"
                        }
                }
        },
        "traces": {
                "buffer_size_mb": 3,
                "concurrency": 8,
                "insecure": false,
                "traces_collected": {
                        "xray": {
                                "bind_address": "127.0.0.1:2000",
                                "tcp_proxy": {
                                        "bind_address": "127.0.0.1:2000"
                                }
                        }
                }
        }
}
Please check the above content of the config.
The config file is also located at /opt/aws/amazon-cloudwatch-agent/bin/config.json.
Edit it manually if needed.
Do you want to store the config in the SSM parameter store?
1. yes
2. no
default choice: [1]: ※デフォルトでOK

What parameter store name do you want to use to store your config? (Use 'AmazonCloudWatch-' prefix if you use our managed AWS policy)
default choice: [AmazonCloudWatch-linux] ※デフォルトでOK

Trying to fetch the default region based on ec2 metadata...
I! imds retry client will retry 1 timesWhich region do you want to store the config in the parameter store?
default choice: [us-east-1] ※デフォルトでOK

Which AWS credential should be used to send json config to parameter store?
1. ***********************:(From SDK)
2. Other
default choice: [1]: ※デフォルトでOK（SDKからクレデンシャルを自動的に取得しているようです）

Successfully put config to parameter store AmazonCloudWatch-linux.
Program exits now.

設定ファイルの更新処理

$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json

サービス再起動

$ sudo systemctl stop amazon-cloudwatch-agent.service
$ sudo systemctl start amazon-cloudwatch-agent.service

トラブルシューティング

"open /usr/share/collectd/types.db: no such file or directory"が表示される

2023-10-22T14:08:05Z E! [telegraf] Error running agent: Error loading config file /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml: error parsing socket_listener, open /usr/share/collectd/types.db: no such file or directory

collectdが入っておらず，types.dbファイルが存在しないことが原因。
collectdをインストールすると解消する。

"no identity-based policy allows the ssm:PutParameter action"が表示される

Please make sure the creds you used have the right permissions configured for SSM access.
Error in putting config to parameter store AmazonCloudWatch-linux: AccessDeniedException: User: arn:aws:sts::************:assumed-role/ロール名/インスタンスID is not authorized to perform: ssm:PutParameter on resource: arn:aws:ssm:us-east-1:202676725572:parameter/AmazonCloudWatch-linux because no identity-based policy allows the ssm:PutParameter action
        status code: 400, request id:*********************
Program exits now.

ssm:PutParameter権限がないことが原因。IAMロールにCloudWatchAgentAdminPolicyのポリシーを追加すれば解消する。

1
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?