What's？

AWS FireLens＋Fluent Bitを組み込んだAWS FargateクラスターをTerraformで構成した時に、タスク定義でterraform plan時に妙に差分が出てきて困ったという話。

なお、原因は不明です。

お題

Terraformで、AWS Fargateクラスターを構築します。

タスク定義に含めるコンテナは、nginxとしましょう。

最初はnginxのみのタスク定義とし、あとでFluent Bit（AWS for Fluent Bit）も追加します。

この時のterraform planの挙動を見てみます。

環境

今回の環境は、こちら。

$ terraform version
Terraform v0.13.5
+ provider registry.terraform.io/hashicorp/aws v3.15.0


$ aws --version
aws-cli/2.1.1 Python/3.7.3 Linux/5.4.0-53-generic exe/x86_64.ubuntu.20

AWSのクレデンシャルは、環境変数で指定しています。

$ export AWS_ACCESS_KEY_ID=.....
$ export AWS_SECRET_ACCESS_KEY=.....
$ export AWS_DEFAULT_REGION=.....

AWS Fargateのタスク定義を行う（nginxのみ）

ここからは、AWS Fargateクラスターを構築していきますが、タスク定義と補足的なリソースのみを最初に記載します。

リソース定義全体は、最後に載せます。

最初に、nginxのみを含めたタスクを定義します。

resource "aws_ecs_task_definition" "nginx" {
  family                   = "nginx-task-definition"
  cpu                      = "512"
  memory                   = "1024"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  execution_role_arn       = aws_iam_role.ecs_task_execution_role.arn
  task_role_arn            = aws_iam_role.ecs_task_role.arn

  container_definitions = <<JSON
  [
    {
      "name": "nginx",
      "image": "nginx:1.19.4",
      "essential": true,
      "portMappings": [
        {
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/nginx",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "nginx"
        }
      }
    }
  ]
  JSON
}

とってもシンプル。

ログは、Amazon CloudWatch Logsに出力するようにしています。あとでAWS FireLensも使うので、ここまでをミニマムとします。

resource "aws_cloudwatch_log_group" "nginx" {
  name = "/fargate/containers/nginx"
}

apply。

$ terraform apply

動作確認。

$ curl -i [ALBのDNS名]
HTTP/1.1 200 OK
Date: Sun, 15 Nov 2020 03:32:23 GMT
Content-Type: text/html
Content-Length: 612
Connection: keep-alive
Server: nginx/1.19.4
Last-Modified: Tue, 27 Oct 2020 15:09:20 GMT
ETag: "5f983820-264"
Accept-Ranges: bytes

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

ここで、terraform planを実行してみます。

$ terraform plan

リソースを構築したばかりですし、定義の変更も行っていないので特に差分はありません。

No changes. Infrastructure is up-to-date.

terraform showを実行して

$ terraform show

タスク定義の部分を見てみます。

# aws_ecs_task_definition.nginx:
resource "aws_ecs_task_definition" "nginx" {
    arn                      = "arn:aws:ecs:ap-northeast-1:[AWSアカウントID]:task-definition/nginx-task-definition:37"
    container_definitions    = jsonencode(
        [
            {
                cpu              = 0
                environment      = []
                essential        = true
                image            = "nginx:1.19.4"
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/fargate/containers/nginx"
                        awslogs-region        = "ap-northeast-1"
                        awslogs-stream-prefix = "nginx"
                    }
                }
                mountPoints      = []
                name             = "nginx"
                portMappings     = [
                    {
                        containerPort = 80
                        hostPort      = 80
                        protocol      = "tcp"
                    },
                ]
                volumesFrom      = []
            },
        ]
    )
    cpu                      = "512"
    execution_role_arn       = "arn:aws:iam::[AWSアカウントID]:role/MyEcsTaskExecutionRole"
    family                   = "nginx-task-definition"
    id                       = "nginx-task-definition"
    memory                   = "1024"
    network_mode             = "awsvpc"
    requires_compatibilities = [
        "FARGATE",
    ]
    revision                 = 37
    task_role_arn            = "arn:aws:iam::[AWSアカウントID]:role/MyEcsTaskRole"
}

JSONに書いていなかった部分も、デフォルト値が入っています。

ここまで確認したら、いったんterraform destroy。

$ terraform destroy

AWS Fargateのタスク定義を行う（nginx＋AWS for Fluent Bit）

続いて、今度はAWS FireLensを使ったログ出力を行います。

AWSのドキュメントを見ながら、

カスタムログルーティング

先ほどのタスク定義にAWS FireLensおよびAWS for Fluent Bitの設定を追加します。

resource "aws_ecs_task_definition" "nginx" {
  family                   = "nginx-task-definition"
  cpu                      = "512"
  memory                   = "1024"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  execution_role_arn       = aws_iam_role.ecs_task_execution_role.arn
  task_role_arn            = aws_iam_role.ecs_task_role.arn

  container_definitions = <<JSON
  [
    {
      "name": "log_router",
      "image": "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest",
      "essential": true,
      "firelensConfiguration": {
        "type": "fluentbit"
      },
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/fluentbit",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "fluentbit"
        }
      }
    },
    {
      "name": "nginx",
      "image": "nginx:1.19.4",
      "essential": true,
      "portMappings": [
        {
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "logConfiguration": {
        "logDriver": "awsfirelens",
        "options": {
          "Name": "cloudwatch",
          "region": "ap-northeast-1",
          "log_group_name": "/fargate/containers/nginx",
          "log_stream_prefix": "nginx",
          "auto_create_group": "false"
        }
      }
    }
  ]
  JSON
}

nginxのログは、AWS FireLensを介してAmazon CloudWatch Logsに出力するようにします。

Fluent Bit用のロググループも作成。

resource "aws_cloudwatch_log_group" "fluentbit" {
  name = "/fargate/containers/fluentbit"
}

タスクロールには、Fluent BitがAmazon CloudWatch Logsにログを出力できるように権限を追加しておきます。

data "aws_iam_policy_document" "ecs_task_role_policy_document" {
  statement {
    effect = "Allow"

    actions = [
      "logs:DescribeLogStreams",
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents"
    ]

    resources = ["*"]
  }
}

リソース構築。

$ terraform apply

動作確認。

$ curl -i [ALBのDNS名]
HTTP/1.1 200 OK
Date: Sun, 15 Nov 2020 03:41:51 GMT
Content-Type: text/html
Content-Length: 612
Connection: keep-alive
Server: nginx/1.19.4
Last-Modified: Tue, 27 Oct 2020 15:09:20 GMT
ETag: "5f983820-264"
Accept-Ranges: bytes

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

ここまではOKです。

で、この状態でterraform planを実行すると

$ terraform plan

なにも変更していませんが、差分が検出されます。

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # aws_ecs_service.nginx will be updated in-place
  ~ resource "aws_ecs_service" "nginx" {
        cluster                            = "arn:aws:ecs:ap-northeast-1:[AWSアカウントID]:cluster/nginx-cluster"
        deployment_maximum_percent         = 200
        deployment_minimum_healthy_percent = 50
        desired_count                      = 3
        enable_ecs_managed_tags            = false
        health_check_grace_period_seconds  = 0
        iam_role                           = "aws-service-role"
        id                                 = "arn:aws:ecs:ap-northeast-1:[AWSアカウントID]:service/nginx-cluster/nginx-service"
        launch_type                        = "FARGATE"
        name                               = "nginx-service"
        platform_version                   = "1.4.0"
        propagate_tags                     = "NONE"
        scheduling_strategy                = "REPLICA"
        tags                               = {}
      ~ task_definition                    = "arn:aws:ecs:ap-northeast-1:[AWSアカウントID]:task-definition/nginx-task-definition:38" -> (known after apply)
        wait_for_steady_state              = false

        deployment_controller {
            type = "ECS"
        }

        load_balancer {
            container_name   = "nginx"
            container_port   = 80
            target_group_arn = "arn:aws:elasticloadbalancing:ap-northeast-1:[AWSアカウントID]:targetgroup/tf-20201115034033903500000005/31aa063424fafa30"
        }

        network_configuration {
            assign_public_ip = false
            security_groups  = [
                "sg-0bdd096a03b3467ab",
            ]
            subnets          = [
                "subnet-07ca48860443af5f7",
                "subnet-0aa3ac7b0fcf82b5e",
            ]
        }
    }

  # aws_ecs_task_definition.nginx must be replaced
-/+ resource "aws_ecs_task_definition" "nginx" {
      ~ arn                      = "arn:aws:ecs:ap-northeast-1:[AWSアカウントID]:task-definition/nginx-task-definition:38" -> (known after apply)
      ~ container_definitions    = jsonencode(
          ~ [ # forces replacement
              ~ {
                  - cpu                   = 0 -> null
                  - environment           = [] -> null
                    essential             = true
                    firelensConfiguration = {
                        type = "fluentbit"
                    }
                    image                 = "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest"
                    logConfiguration      = {
                        logDriver = "awslogs"
                        options   = {
                            awslogs-group         = "/fargate/containers/fluentbit"
                            awslogs-region        = "ap-northeast-1"
                            awslogs-stream-prefix = "fluentbit"
                        }
                    }
                  - mountPoints           = [] -> null
                    name                  = "log_router"
                  - portMappings          = [] -> null
                  - user                  = "0" -> null
                  - volumesFrom           = [] -> null
                } # forces replacement,
              ~ {
                  - cpu              = 0 -> null
                  - environment      = [] -> null
                    essential        = true
                    image            = "nginx:1.19.4"
                    logConfiguration = {
                        logDriver = "awsfirelens"
                        options   = {
                            Name              = "cloudwatch"
                            auto_create_group = "false"
                            log_group_name    = "/fargate/containers/nginx"
                            log_stream_prefix = "nginx"
                            region            = "ap-northeast-1"
                        }
                    }
                  - mountPoints      = [] -> null
                    name             = "nginx"
                  ~ portMappings     = [
                      ~ {
                            containerPort = 80
                          - hostPort      = 80 -> null
                            protocol      = "tcp"
                        },
                    ]
                  - volumesFrom      = [] -> null
                } # forces replacement,
            ]
        )
        cpu                      = "512"
        execution_role_arn       = "arn:aws:iam::[AWSアカウントID]:role/MyEcsTaskExecutionRole"
        family                   = "nginx-task-definition"
      ~ id                       = "nginx-task-definition" -> (known after apply)
        memory                   = "1024"
        network_mode             = "awsvpc"
        requires_compatibilities = [
            "FARGATE",
        ]
      ~ revision                 = 38 -> (known after apply)
      - tags                     = {} -> null
        task_role_arn            = "arn:aws:iam::[AWSアカウントID]:role/MyEcsTaskRole"
    }

Plan: 1 to add, 1 to change, 1 to destroy.

------------------------------------------------------------------------

コンテナ定義の、デフォルト値まわりが軒並み指摘されているように見えます。

      ~ container_definitions    = jsonencode(
          ~ [ # forces replacement
              ~ {
                  - cpu                   = 0 -> null
                  - environment           = [] -> null
                    essential             = true
                    firelensConfiguration = {
                        type = "fluentbit"
                    }
                    image                 = "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest"
                    logConfiguration      = {
                        logDriver = "awslogs"
                        options   = {
                            awslogs-group         = "/fargate/containers/fluentbit"
                            awslogs-region        = "ap-northeast-1"
                            awslogs-stream-prefix = "fluentbit"
                        }
                    }
                  - mountPoints           = [] -> null
                    name                  = "log_router"
                  - portMappings          = [] -> null
                  - user                  = "0" -> null
                  - volumesFrom           = [] -> null
                } # forces replacement,
              ~ {
                  - cpu              = 0 -> null
                  - environment      = [] -> null
                    essential        = true
                    image            = "nginx:1.19.4"
                    logConfiguration = {
                        logDriver = "awsfirelens"
                        options   = {
                            Name              = "cloudwatch"
                            auto_create_group = "false"
                            log_group_name    = "/fargate/containers/nginx"
                            log_stream_prefix = "nginx"
                            region            = "ap-northeast-1"
                        }
                    }
                  - mountPoints      = [] -> null
                    name             = "nginx"
                  ~ portMappings     = [
                      ~ {
                            containerPort = 80
                          - hostPort      = 80 -> null
                            protocol      = "tcp"
                        },
                    ]
                  - volumesFrom      = [] -> null
                } # forces replacement,
            ]
        )

terraform showで見てみても

$ terraform show

nginx単体の時とそんなに変わった気はしないんですけどねぇ…。

# aws_ecs_task_definition.nginx:
resource "aws_ecs_task_definition" "nginx" {
    arn                      = "arn:aws:ecs:ap-northeast-1:[AWSアカウントID]:task-definition/nginx-task-definition:38"
    container_definitions    = jsonencode(
        [
            {
                cpu                   = 0
                environment           = []
                essential             = true
                firelensConfiguration = {
                    type = "fluentbit"
                }
                image                 = "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest"
                logConfiguration      = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/fargate/containers/fluentbit"
                        awslogs-region        = "ap-northeast-1"
                        awslogs-stream-prefix = "fluentbit"
                    }
                }
                mountPoints           = []
                name                  = "log_router"
                portMappings          = []
                user                  = "0"
                volumesFrom           = []
            },
            {
                cpu              = 0
                environment      = []
                essential        = true
                image            = "nginx:1.19.4"
                logConfiguration = {
                    logDriver = "awsfirelens"
                    options   = {
                        Name              = "cloudwatch"
                        auto_create_group = "false"
                        log_group_name    = "/fargate/containers/nginx"
                        log_stream_prefix = "nginx"
                        region            = "ap-northeast-1"
                    }
                }
                mountPoints      = []
                name             = "nginx"
                portMappings     = [
                    {
                        containerPort = 80
                        hostPort      = 80
                        protocol      = "tcp"
                    },
                ]
                volumesFrom      = []
            },
        ]
    )
    cpu                      = "512"
    execution_role_arn       = "arn:aws:iam::[AWSアカウントID]:role/MyEcsTaskExecutionRole"
    family                   = "nginx-task-definition"
    id                       = "nginx-task-definition"
    memory                   = "1024"
    network_mode             = "awsvpc"
    requires_compatibilities = [
        "FARGATE",
    ]
    revision                 = 38
    task_role_arn            = "arn:aws:iam::[AWSアカウントID]:role/MyEcsTaskRole"
}

仕方がないので、指摘されていたデフォルト値を全部埋めてあげると

  container_definitions = <<JSON
  [
    {
      "name": "log_router",
      "image": "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest",
      "essential": true,
      "firelensConfiguration": {
        "type": "fluentbit"
      },
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/fluentbit",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "fluentbit"
        }
      },
      "cpu": 0,
      "environment": [],
      "mountPoints": [],
      "volumesFrom": [],
      "portMappings": [],
      "user": "0"
    },
    {
      "name": "nginx",
      "image": "nginx:1.19.4",
      "essential": true,
      "portMappings": [
        {
          "protocol": "tcp",
          "containerPort": 80,
          "hostPort": 80
        }
      ],
      "logConfiguration": {
        "logDriver": "awsfirelens",
        "options": {
          "Name": "cloudwatch",
          "region": "ap-northeast-1",
          "log_group_name": "/fargate/containers/nginx",
          "log_stream_prefix": "nginx",
          "auto_create_group": "false"
        }
      },
      "cpu": 0,
      "environment": [],
      "mountPoints": [],
      "volumesFrom": []
    }
  ]
  JSON

planで差分が出なくなります。

$ terraform plan

どうなってるんでしょうね？

No changes. Infrastructure is up-to-date.

ちなみに、（まったく意味がないですが）Fluent Bitをタスク定義に入れてもAWS FireLensとして構成していない場合は差分が検出されません。

  container_definitions = <<JSON
  [
    {
      "name": "log_router",
      "image": "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest",
      "essential": true,
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/fluentbit",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "fluentbit"
        }
      }
    },
    {
      "name": "nginx",
      "image": "nginx:1.19.4",
      "essential": true,
      "portMappings": [
        {
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/nginx",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "nginx"
        }
      }
    }
  ]
  JSON

terraform showで、タスク定義の状態を見てみます。

# aws_ecs_task_definition.nginx:
resource "aws_ecs_task_definition" "nginx" {
    arn                      = "arn:aws:ecs:ap-northeast-1:[AWSアカウントID]:task-definition/nginx-task-definition:39"
    container_definitions    = jsonencode(
        [
            {
                cpu              = 0
                environment      = []
                essential        = true
                image            = "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest"
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/fargate/containers/fluentbit"
                        awslogs-region        = "ap-northeast-1"
                        awslogs-stream-prefix = "fluentbit"
                    }
                }
                mountPoints      = []
                name             = "log_router"
                portMappings     = []
                volumesFrom      = []
            },
            {
                cpu              = 0
                environment      = []
                essential        = true
                image            = "nginx:1.19.4"
                logConfiguration = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = "/fargate/containers/nginx"
                        awslogs-region        = "ap-northeast-1"
                        awslogs-stream-prefix = "nginx"
                    }
                }
                mountPoints      = []
                name             = "nginx"
                portMappings     = [
                    {
                        containerPort = 80
                        hostPort      = 80
                        protocol      = "tcp"
                    },
                ]
                volumesFrom      = []
            },
        ]
    )
    cpu                      = "512"
    execution_role_arn       = "arn:aws:iam::[AWSアカウントID]:role/MyEcsTaskExecutionRole"
    family                   = "nginx-task-definition"
    id                       = "nginx-task-definition"
    memory                   = "1024"
    network_mode             = "awsvpc"
    requires_compatibilities = [
        "FARGATE",
    ]
    revision                 = 39
    task_role_arn            = "arn:aws:iam::[AWSアカウントID]:role/MyEcsTaskRole"
}

AWS FireLensとして構成した時と比べて、firelensConfigurationとかログ出力定義とかに差分があるのはもちろんなのですが。

AWS FireLensとして構成した場合は、Fluent Bitコンテナに以下の値が追加されています。

      "user": "0"

なにか、差分になる理由があるんでしょうね…。

userを0に指定しているということは、UIDを指定していることになると思うのですが。

タスク定義パラメータ

AWS FireLensが関係しそうだということはわかったのですが、今回はこれ以上追わないことにします。

リソース定義全体

最後に、ここまでのリソース定義全体を載せておきます。

VPCからALB、AWS Fargateのクラスターやサービス定義などを含め、ここまで登場したタスク定義のJSONについてはコメントで残しています。

有効にして残しているのは、AWS FireLensを使いつつplanで差分が出ないパターンですね。

main.tf

terraform {
  required_version = "0.13.5"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "3.15.0"
    }
  }
}

provider "aws" {
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "2.64.0"

  name = "my-vpc"
  cidr = "10.0.0.0/16"

  enable_dns_hostnames = true
  enable_dns_support   = true

  azs             = ["ap-northeast-1a", "ap-northeast-1c"]
  public_subnets  = ["10.0.10.0/24", "10.0.20.0/24"]
  private_subnets = ["10.0.30.0/24", "10.0.40.0/24"]

  map_public_ip_on_launch = true

  enable_nat_gateway     = true
  single_nat_gateway     = false
  one_nat_gateway_per_az = true
}

module "load_balancer_sg" {
  source  = "terraform-aws-modules/security-group/aws//modules/http-80"
  version = "3.16.0"

  name   = "load-balancer-sg"
  vpc_id = module.vpc.vpc_id

  ingress_cidr_blocks = ["0.0.0.0/0"]
}

module "nginx_service_sg" {
  source  = "terraform-aws-modules/security-group/aws"
  version = "3.16.0"

  name   = "nginx-service-sg"
  vpc_id = module.vpc.vpc_id

  ingress_with_cidr_blocks = [
    {
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      description = "nginx-service inbound ports"
      cidr_blocks = "10.0.10.0/24"
    },
    {
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      description = "nginx-service inbound ports"
      cidr_blocks = "10.0.20.0/24"
    }
  ]

  egress_with_cidr_blocks = [
    {
      from_port   = 0
      to_port     = 0
      protocol    = "-1"
      description = "nginx-service outbound ports"
      cidr_blocks = "0.0.0.0/0"
    }
  ]
}

module "load_balancer" {
  source  = "terraform-aws-modules/alb/aws"
  version = "5.9.0"

  name = "nginx"

  vpc_id             = module.vpc.vpc_id
  load_balancer_type = "application"
  internal           = false

  subnets         = module.vpc.public_subnets
  security_groups = [module.load_balancer_sg.this_security_group_id]

  target_groups = [
    {
      backend_protocol = "HTTP"
      backend_port     = 80
      target_type      = "ip"

      health_check = {
        interval = 20
      }
    }
  ]

  http_tcp_listeners = [
    {
      port     = 80
      protocol = "HTTP"
    }
  ]
}

locals {
  vpc_id = module.vpc.vpc_id

  private_subnets                = module.vpc.private_subnets
  nginx_service_security_groups  = [module.nginx_service_sg.this_security_group_id]
  load_balancer_target_group_arn = module.load_balancer.target_group_arns[0]
}

data "aws_iam_policy_document" "assume_role" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["ecs-tasks.amazonaws.com"]
    }
  }
}

data "aws_iam_policy" "ecs_task_execution_role_policy" {
  arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

resource "aws_iam_role" "ecs_task_execution_role" {
  name               = "MyEcsTaskExecutionRole"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

resource "aws_iam_role_policy_attachment" "ecs_task_execution_role_policy_attachment" {
  role       = aws_iam_role.ecs_task_execution_role.name
  policy_arn = data.aws_iam_policy.ecs_task_execution_role_policy.arn
}

data "aws_iam_policy_document" "ecs_task_role_policy_document" {
  statement {
    effect = "Allow"

    actions = [
      "logs:DescribeLogStreams",
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents"
    ]

    resources = ["*"]
  }
}

resource "aws_iam_policy" "ecs_task_role_policy" {
  name   = "MyEcsTaskPolicy"
  policy = data.aws_iam_policy_document.ecs_task_role_policy_document.json
}

resource "aws_iam_role" "ecs_task_role" {
  name               = "MyEcsTaskRole"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

resource "aws_iam_role_policy_attachment" "ecs_task_role_policy_attachment" {
  role       = aws_iam_role.ecs_task_role.name
  policy_arn = aws_iam_policy.ecs_task_role_policy.arn
}

resource "aws_cloudwatch_log_group" "nginx" {
  name = "/fargate/containers/nginx"
}

resource "aws_cloudwatch_log_group" "fluentbit" {
  name = "/fargate/containers/fluentbit"
}

resource "aws_ecs_cluster" "nginx" {
  name = "nginx-cluster"
}

resource "aws_ecs_task_definition" "nginx" {
  family                   = "nginx-task-definition"
  cpu                      = "512"
  memory                   = "1024"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  execution_role_arn       = aws_iam_role.ecs_task_execution_role.arn
  task_role_arn            = aws_iam_role.ecs_task_role.arn

  container_definitions = <<JSON
  [
    {
      "name": "log_router",
      "image": "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest",
      "essential": true,
      "firelensConfiguration": {
        "type": "fluentbit"
      },
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/fluentbit",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "fluentbit"
        }
      },
      "cpu": 0,
      "environment": [],
      "mountPoints": [],
      "volumesFrom": [],
      "portMappings": [],
      "user": "0"
    },
    {
      "name": "nginx",
      "image": "nginx:1.19.4",
      "essential": true,
      "portMappings": [
        {
          "protocol": "tcp",
          "containerPort": 80,
          "hostPort": 80
        }
      ],
      "logConfiguration": {
        "logDriver": "awsfirelens",
        "options": {
          "Name": "cloudwatch",
          "region": "ap-northeast-1",
          "log_group_name": "/fargate/containers/nginx",
          "log_stream_prefix": "nginx",
          "auto_create_group": "false"
        }
      },
      "cpu": 0,
      "environment": [],
      "mountPoints": [],
      "volumesFrom": []
    }
  ]
  JSON

  /*
  container_definitions = <<JSON
  [
    {
      "name": "log_router",
      "image": "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest",
      "essential": true,
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/fluentbit",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "fluentbit"
        }
      }
    },
    {
      "name": "nginx",
      "image": "nginx:1.19.4",
      "essential": true,
      "portMappings": [
        {
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/nginx",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "nginx"
        }
      }
    }
  ]
  JSON
  */

  /*
  container_definitions = <<JSON
  [
    {
      "name": "log_router",
      "image": "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest",
      "essential": true,
      "firelensConfiguration": {
        "type": "fluentbit"
      },
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/fluentbit",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "fluentbit"
        }
      }
    },
    {
      "name": "nginx",
      "image": "nginx:1.19.4",
      "essential": true,
      "portMappings": [
        {
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "logConfiguration": {
        "logDriver": "awsfirelens",
        "options": {
          "Name": "cloudwatch",
          "region": "ap-northeast-1",
          "log_group_name": "/fargate/containers/nginx",
          "log_stream_prefix": "nginx",
          "auto_create_group": "false"
        }
      }
    }
  ]
  JSON
  */

  /*
  container_definitions = <<JSON
  [
    {
      "name": "nginx",
      "image": "nginx:1.19.4",
      "essential": true,
      "portMappings": [
        {
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/nginx",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "nginx"
        }
      }
    }
  ]
  JSON
  */
}

resource "aws_ecs_service" "nginx" {
  name             = "nginx-service"
  cluster          = aws_ecs_cluster.nginx.arn
  task_definition  = aws_ecs_task_definition.nginx.arn
  desired_count    = 3
  launch_type      = "FARGATE"
  platform_version = "1.4.0"

  deployment_minimum_healthy_percent = 50

  network_configuration {
    assign_public_ip = false
    security_groups  = local.nginx_service_security_groups
    subnets          = local.private_subnets
  }

  load_balancer {
    target_group_arn = local.load_balancer_target_group_arn
    container_name   = "nginx"
    container_port   = 80
  }
}