はじめに
EC2内にApacheを入れたのですが、アクセスログとエラーログをCloudWatch Logsに送って、CloudWatch上から確認できるようにしたかったのでセットアップをしてみます。
注意事項・前提事項
- 細かい説明は省いて設定方法のみを記載していますので理解しながら進めたい人は要望に応えられないかもしれません。
- EC2構築とApache導入・IAMロールの作成/適用は既に終わっているものとします。
やってみる
EC2へCloudWatchAgentインストール
# yum -y install amazon-cloudwat-agent
インストール確認
yum list installed| grep amazon-cloudwatch-agent
amazon-cloudwatch-agent.x86_64 1.247352.0-1.amzn2 @amzn2-core
ウィザードを使用してCloudWatchAgent設定ファイル作成
# /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
ウィザードの中身
# /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
================================================================
= Welcome to the Amazon CloudWatch Agent Configuration Manager =
= =
= CloudWatch Agent allows you to collect metrics and logs from =
= your host and send them to CloudWatch. Additional CloudWatch =
= charges may apply. =
================================================================
On which OS are you planning to use the agent?
1. linux
2. windows
3. darwin
default choice: [1]:
Trying to fetch the default region based on ec2 metadata...
Are you using EC2 or On-Premises hosts?
1. EC2
2. On-Premises
default choice: [1]:
Which user are you planning to run the agent?
1. root
2. cwagent
3. others
default choice: [1]:
Do you want to turn on StatsD daemon?
1. yes
2. no
default choice: [1]:
2
Do you want to monitor metrics from CollectD? WARNING: CollectD must be installed or the Agent will fail to start
1. yes
2. no
default choice: [1]:
2
Do you want to monitor any host metrics? e.g. CPU, memory, etc.
1. yes
2. no
default choice: [1]:
2
Do you have any existing CloudWatch Log Agent (http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html) configuration file to import for migration?
1. yes
2. no
default choice: [2]:
Do you want to monitor any log files?
1. yes
2. no
default choice: [1]:
Log file path:
/var/log/httpd/access_log
Log group name:
default choice: [access_log]
Log stream name:
default choice: [{instance_id}]
Log Group Retention in days
1. -1
2. 1
3. 3
4. 5
5. 7
6. 14
7. 30
8. 60
9. 90
10. 120
11. 150
12. 180
13. 365
14. 400
15. 545
16. 731
17. 1827
18. 2192
19. 2557
20. 2922
21. 3288
22. 3653
default choice: [1]:
Do you want to specify any additional log files to monitor?
1. yes
2. no
default choice: [1]:
Log file path:
/var/log/httpd/error_log
Log group name:
default choice: [error_log]
Log stream name:
default choice: [{instance_id}]
Log Group Retention in days
1. -1
2. 1
3. 3
4. 5
5. 7
6. 14
7. 30
8. 60
9. 90
10. 120
11. 150
12. 180
13. 365
14. 400
15. 545
16. 731
17. 1827
18. 2192
19. 2557
20. 2922
21. 3288
22. 3653
default choice: [1]:
Do you want to specify any additional log files to monitor?
1. yes
2. no
default choice: [1]:
2
Saved config file to /opt/aws/amazon-cloudwatch-agent/bin/config.json successfully.
Current config as follows:
{
"agent": {
"run_as_user": "root"
},
"logs": {
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/var/log/httpd/access_log",
"log_group_name": "access_log",
"log_stream_name": "{instance_id}",
"retention_in_days": -1
},
{
"file_path": "/var/log/httpd/error_log",
"log_group_name": "error_log",
"log_stream_name": "{instance_id}",
"retention_in_days": -1
}
]
}
}
}
}
Please check the above content of the config.
The config file is also located at /opt/aws/amazon-cloudwatch-agent/bin/config.json.
Edit it manually if needed.
Do you want to store the config in the SSM parameter store?
1. yes
2. no
default choice: [1]:
2
Program exits now.
SSMパラメータストアへ送信する場合は、IAMロールに記載している"PutParameter"アクションに対しての対象リソースセクションは確認すること。
設定ファイル(config.hson)生成確認
# ll /opt/aws/amazon-cloudwatch-agent/bin/
total 207948
-rwxr-xr-x 1 root root 66556280 Jun 9 19:14 amazon-cloudwatch-agent
-rwxr-xr-x 1 root root 9261336 Jun 9 19:14 amazon-cloudwatch-agent-config-wizard
-rwxr-xr-x 1 root root 19318 Jun 9 19:14 amazon-cloudwatch-agent-ctl
-rwxr-xr-x 1 root root 8255864 Jun 9 19:14 config-downloader
-rwxr-xr-x 1 root root 473 Aug 30 05:21 config.json
-rwxr-xr-x 1 root root 18741720 Jun 9 19:14 config-translator
-rwxr-xr-x 1 root root 91570232 Jun 9 19:14 cwagent-otel-collector
-rw-r--r-- 1 root root 11 Jun 9 19:14 CWAGENT_VERSION
-rwxr-xr-x 1 root root 18508024 Jun 9 19:14 start-amazon-cloudwatch-agent
# cat /opt/aws/amazon-cloudwatch-agent/bin/config.json
{
"agent": {
"run_as_user": "root"
},
"logs": {
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/var/log/httpd/access_log",
"log_group_name": "access_log",
"log_stream_name": "{instance_id}",
"retention_in_days": -1
},
{
"file_path": "/var/log/httpd/error_log",
"log_group_name": "error_log",
"log_stream_name": "{instance_id}",
"retention_in_days": -1
}
]
}
}
}
amazon-cloudwatch-agent-ctlの起動ヘルプをサラッと紹介
# /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl
Invalid action:
usage: amazon-cloudwatch-agent-ctl -a
stop|start|status|fetch-config|append-config|remove-config|set-log-level
[-m ec2|onPremise|auto]
[-c default|all|ssm:<parameter-store-name>|file:<file-path>]
[-o default|all|ssm:<parameter-store-name>|file:<file-path>]
[-s]
[-l INFO|DEBUG|WARN|ERROR|OFF]
e.g.
1. apply a SSM parameter store config on EC2 instance and restart the agent afterwards:
amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:AmazonCloudWatch-Config.json -s
2. append a local json config file on onPremise host and restart the agent afterwards:
amazon-cloudwatch-agent-ctl -a append-config -m onPremise -c file:/tmp/config.json -s
3. query agent status:
amazon-cloudwatch-agent-ctl -a status
-a: action
stop: stop the agent process.
start: start the agent process.
status: get the status of the agent process.
fetch-config: apply config for agent, followed by -c or -o or both. Target config can be based on location (ssm parameter store name, file name), or 'default'.
append-config: append json config with the existing json configs if any, followed by -c. Target config can be based on the location (ssm parameter store name, file name), or 'default'.
remove-config: remove config for agent, followed by -c or -o or both. Target config can be based on the location (ssm parameter store name, file name), or 'all'.
set-log-level: sets the log level, followed by -l to provide the level in all caps.
-m: mode
ec2: indicate this is on ec2 host.
onPremise: indicate this is on onPremise host.
auto: use ec2 metadata to determine the environment, may not be accurate if ec2 metadata is not available for some reason on EC2.
-c: amazon-cloudwatch-agent configuration
default: default configuration for quick trial.
ssm:<parameter-store-name>: ssm parameter store name.
file:<file-path>: file path on the host.
all: all existing configs. Only apply to remove-config action.
-o: cwagent-otel-collector configuration
default: default configuration for quick trial.
ssm:<parameter-store-name>: ssm parameter store name.
file:<file-path>: file path on the host.
all: all existing configs. Only apply to remove-config action.
-s: optionally restart after configuring the agent configuration
this parameter is used for 'fetch-config', 'append-config', 'remove-config' action only.
-l: log level to set the agent to INFO, DEBUG, WARN, ERROR, or OFF
this parameter is used for 'set-log-level' only.
サービス起動
# /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file://opt/aws/amazon-cloudwatch-agent/bin/config.json
****** processing amazon-cloudwatch-agent ******
/opt/aws/amazon-cloudwatch-agent/bin/config-downloader --output-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --download-source file://opt/aws/amazon-cloudwatch-agent/bin/config.json --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default
I! Trying to detect region from ec2
2022/08/30 05:30:52 D! [EC2] Found active network interface
Successfully fetched the config and saved in /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_config.json.tmp
Start configuration validation...
/opt/aws/amazon-cloudwatch-agent/bin/config-translator --input /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json --input-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --output /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default
2022/08/30 05:30:52 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_config.json.tmp ...
2022/08/30 05:30:52 I! Valid Json input schema.
I! Detecting run_as_user...
I! Trying to detect region from ec2
2022/08/30 05:30:52 D! [EC2] Found active network interface
No csm configuration found.
No metric configuration found.
Configuration validation first phase succeeded
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent -schematest -config /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml
Configuration validation second phase succeeded
Configuration validation succeeded
amazon-cloudwatch-agent has already been stopped
Created symlink from /etc/systemd/system/multi-user.target.wants/amazon-cloudwatch-agent.service to /etc/systemd/system/amazon-cloudwatch-agent.service.
Redirecting to /bin/systemctl restart amazon-cloudwatch-agent.service
CloudWatchにて確認
