前提
会社の業務でTerraformでリソースを管理することが多いのですが、CloudWatch AlarmsをTerraformのmoduleで管理し、for_eachを使って似たようなCloudWatch Alarmを作ろうとしたらエラーがでたお話です。
API Gateway + Lambda関数のREST APIの構成に対して、それぞれのLambdaに対して1つのCloudWatch Alarmを作成しようとしました。基本的には同じメトリクス、同じ名前空間で、関数名が違うだけのCloudWatch Alarmをいくつか作ることになります。そこでmodule＋for_eachで簡単に作ろうとしたのですが、エラーが出た話です。
また、監視対象のLambda関数は特定のaliasを持っているものとします。例えば、本番用のLambdaだけ監視したいなど。
筆者はTerraform初心者ですので、何か間違っている点などがあれば、コメントをいただけるとありがたいです。
結論
terraformのバージョンが0.12系だと、moduleで定義されたリソースに対してfor_eachを使うことができない。0.13系だと可能。以下が参考文献です。
https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each#module-count-and-for_each
ファイルの構成
Terraformのversionは0.12.28です。
> terraform --version
Terraform v0.12.28
Your version of Terraform is out of date! The latest version
is 0.13.4. You can update by downloading from https://www.terraform.io/downloads.html
module/cloudwatch_alarm/lambda/ ディレクトリ以下にmain.tfとvariables.tfを以下のように作成します。
module/cloudwatch_alarm/lambda/main.tf
provider "aws" {
region = "ap-northeast-1"
shared_credentials_file = "~/.aws/credentials"
profile = "hogehoge"
}
resource "aws_cloudwatch_metric_alarm" "lambda_alarm" {
alarm_name = var.alarm_name
comparison_operator = var.comparison_operator
evaluation_periods = var.evaluation_periods
metric_name = var.metric_name
namespace = var.namespace
period = var.period
statistic = var.statistic
threshold = var.threshold
insufficient_data_actions = var.insufficient_data_actions
alarm_actions = var.alarm_actions
}
module/cloudwatch_alarm/lambda/variables.tf
variable "alarm_name" {
default = ""
type = string
description = "The name of CloudWatch Alarm"
}
variable "comparison_operator" {
default = ""
type = string
description = "The operator to evaluate the metrics"
}
variable "evaluation_periods" {
default = "1"
type = string
description = "The number of the most recent periods, or data points, to evaluate when determining alarm state"
}
variable "metric_name" {
default = ""
type = string
description = "metrics which you want to monitor"
}
variable "namespace" {
default = "AWS/Lambda"
type = string
description = "namespace which you want to monitor"
}
variable "period" {
default = "300"
type = string
description = "The length of time to evaluate the metric or expression to create each individual data point for an alarm"
}
variable "statistic" {
default = ""
type = string
description = "statistic to evaluate a metrics"
}
variable "threshold" {
default = ""
type = string
description = "threshold to change the status of an alarm"
}
variable "alarm_description" {
default = ""
type = string
description = "the explanation of an alarm"
}
variable "insufficient_data_actions" {
default = ["specify the actions when the data is not sufficient"]
type = list
}
variable "dimensions" {
default = {}
type = map(string)
description = "If you want to monitor specific resource in namespace, you can specify it"
}
variable "alarm_actions" {
default = []
type = list
description = "Actions which are executed when the status of an alarm is changed"
}
そして、moduleを使ってCloudWatch Alarmsを作成します。alarmの設定をconfig.tfに記載し、main.tfで使用します。
main.tf
module "cloudwatch_for_lambda" {
source = "./module/cloudwatch_alarm/lambda"
for_each = local.lambda_func_list
alarm_name = "${each.key}-${local.alarm_settings["metric"]}-alarm"
comparison_operator = local.alarm_settings["comparison_operator"]
evaluation_periods = local.alarm_settings["evaluation_periods"]
metric_name = local.alarm_settings["metric"]
namespace = local.alarm_settings["namespace"]
period = local.alarm_settings["period"]
statistic = local.alarm_settings["statistic"]
threshold = local.alarm_settings["threshold"]
alarm_description = "Detect errors from Lambda function ${each.key} "
insufficient_data_actions = []
dimensions = {
FunctionName = "${each.key}",
Resource = "${each.key}:${each.value}"
}
}
config.tf
locals {
lambda_func_list = {
func1 = "alias1",
func2 = "alias2",
func3 = "alias3"
}
alarm_settings = {
comparison_operator = "GreaterThanThreshold"
evaluation_periods = "1"
metric = "Errors"
namespace = "AWS/Lambda"
period = "300"
statistic = "Sum"
threshold = "0.0"
}
}
いざterraform init and terraform planへ
terraform initしようとしますが、、、
> terraform init
Initializing modules...
- cloudwatch_for_lambda in module/cloudwatch_alarm/lambda
Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "aws" (hashicorp/aws) 3.9.0...
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.
* provider.aws: version = "~> 3.9"
Warning: Skipping backend initialization pending configuration upgrade
The root module configuration contains errors that may be fixed by running the
configuration upgrade tool, so Terraform is skipping backend initialization.
See below for more information.
Error: Reserved argument name in module block
on main.tf line 4, in module "cloudwatch_for_lambda":
4: for_each = local.lambda_func_list
The name "for_each" is reserved for use in a future version of Terraform.
Terraform has initialized, but configuration upgrades may be needed.
Terraform found syntax errors in the configuration that prevented full
initialization. If you've recently upgraded to Terraform v0.12, this may be
because your configuration uses syntax constructs that are no longer valid,
and so must be updated before full initialization is possible.
Terraform has installed the required providers to support the configuration
upgrade process. To begin upgrading your configuration, run the following:
terraform 0.12upgrade
To see the full set of errors that led to this message, run:
terraform validate
The name "for_each" is reserved for use in a future version of Terraform.というエラーを調べてみると、以下の記事を見つけました。
https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each#module-count-and-for_each
以下は記事からの引用です。
Module count and for_each
For a long time, users have wished to be able to use the count meta-argument within module blocks, allowing multiple instances of the same module to be created more easily.
Again, we have been laying the groundwork for this during Terraform 0.12 development and expect to complete this work in a later release. Along with count, module blocks will also accept the new for_each argument described for resources above, with similar results.
This feature is particularly complicated to implement within Terraform's existing architecture, so some more work will certainly be required before we can support this. To avoid further breaking changes in later releases, 0.12 will reserve the module input variable names count and for_each in preparation for the completion of this feature.
versionの問題のようです。0.12.xxだとmoduleに対してcount, for_eachが使えないようです。0.13.1にしてもう一度試してみます。
> tfenv list
0.13.1
* 0.12.28 (set by /usr/local/Cellar/tfenv/2.0.0/version)
> tfenv use 0.13.1
Switching default version to v0.13.1
Switching completed
> terraform init
Initializing modules...
There are some problems with the configuration, described below.
The Terraform configuration must be valid before initialization so that
Terraform can determine which modules and providers need to be installed.
Error: Module does not support for_each
on main.tf line 4, in module "cloudwatch_for_lambda":
4: for_each = local.lambda_func_list
Module "cloudwatch_for_lambda" cannot be used with for_each because it
contains a nested provider configuration for "aws", at
module/cloudwatch_alarm/lambda/main.tf:1,10-15.
This module can be made compatible with for_each by changing it to receive all
of its provider configurations from the calling module, by using the
"providers" argument in the calling module block.
エラーが出ました。moduleのmain.tfの中に、providerの情報が入ってるのがいけないようです。以下のようにmodule/cloudwatch_alarm/lambda/main.tfのproviderの部分をprovider.tfに分離します。
module/cloudwatch_alarm/lambda/provider.tf
provider "aws" {
region = "ap-northeast-1"
shared_credentials_file = "~/.aws/credentials"
profile = "hogehoge"
}
module/cloudwatch_alarm/lambda/main.tf
resource "aws_cloudwatch_metric_alarm" "lambda_alarm" {
alarm_name = var.alarm_name
comparison_operator = var.comparison_operator
evaluation_periods = var.evaluation_periods
metric_name = var.metric_name
namespace = var.namespace
period = var.period
statistic = var.statistic
threshold = var.threshold
insufficient_data_actions = var.insufficient_data_actions
alarm_actions = var.alarm_actions
}
もう一度terraform initします。
> terraform init
Initializing modules...
Initializing the backend...
Initializing provider plugins...
- Finding latest version of hashicorp/aws...
- Installing hashicorp/aws v3.9.0...
- Installed hashicorp/aws v3.9.0 (signed by HashiCorp)
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, we recommend adding version constraints in a required_providers block
in your configuration, with the constraint strings suggested below.
* hashicorp/aws: version = "~> 3.9.0"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
terraform planもします。
> terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# module.cloudwatch_for_lambda["func1"].aws_cloudwatch_metric_alarm.lambda_alarm will be created
+ resource "aws_cloudwatch_metric_alarm" "lambda_alarm" {
+ actions_enabled = true
+ alarm_name = "func1-Errors-alarm"
+ arn = (known after apply)
+ comparison_operator = "GreaterThanThreshold"
+ evaluate_low_sample_count_percentiles = (known after apply)
+ evaluation_periods = 1
+ id = (known after apply)
+ metric_name = "Errors"
+ namespace = "AWS/Lambda"
+ period = 300
+ statistic = "Sum"
+ threshold = 0
+ treat_missing_data = "missing"
}
# module.cloudwatch_for_lambda["func2"].aws_cloudwatch_metric_alarm.lambda_alarm will be created
+ resource "aws_cloudwatch_metric_alarm" "lambda_alarm" {
+ actions_enabled = true
+ alarm_name = "func2-Errors-alarm"
+ arn = (known after apply)
+ comparison_operator = "GreaterThanThreshold"
+ evaluate_low_sample_count_percentiles = (known after apply)
+ evaluation_periods = 1
+ id = (known after apply)
+ metric_name = "Errors"
+ namespace = "AWS/Lambda"
+ period = 300
+ statistic = "Sum"
+ threshold = 0
+ treat_missing_data = "missing"
}
# module.cloudwatch_for_lambda["func3"].aws_cloudwatch_metric_alarm.lambda_alarm will be created
+ resource "aws_cloudwatch_metric_alarm" "lambda_alarm" {
+ actions_enabled = true
+ alarm_name = "func3-Errors-alarm"
+ arn = (known after apply)
+ comparison_operator = "GreaterThanThreshold"
+ evaluate_low_sample_count_percentiles = (known after apply)
+ evaluation_periods = 1
+ id = (known after apply)
+ metric_name = "Errors"
+ namespace = "AWS/Lambda"
+ period = 300
+ statistic = "Sum"
+ threshold = 0
+ treat_missing_data = "missing"
}
Plan: 3 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.
期待した通りに3つの名前が違うだけのCloudWatch Alarmができるようです。
まとめ
- 0.12系のterraformを使用していて、moduleで定義されたリソース対してfor_eachを使おうとするとエラーが出るようです。0.13系だとうまくいきます。
- 無理やりmoduleの中に持っていく必要性はないかもしれませんが、それはterraformファイルをどうやって管理しているかにもよるのではないかと思います。