What's？

AWS FireLensを使用すると、Amazon ECSやAWS Fargate上で動作するコンテナのログルーターとして機能させることができます。

Firelens の発表 – コンテナログの新たな管理方法

それはいいのですが、AWS FireLensを介した各コンテナのログの出力先は、それぞれのコンテナのlogConfigurationとして設定します。

FireLens構成を使用するタスク定義の作成

ここから、Fluent BitやFluentdの設定が生成されるというわけです。

ですが、この方法だとコンテナごとにログ出力設定が個別に埋め込まれることになり、ログの送信先の設定が同じような場合は管理が煩雑になります。

Fluent Bitでは、設定を環境変数から参照できるようなので、これを利用してみようかなと。

追記

書いた後で気づきましたが、secretOptionsを使えばよさそうですね…。

ログに機密データを注入する構成

お題

AWS Fargateに、nginxのコンテナを使ったクラスターを構築します。

nginxのログは、AWS FireLensおよびAWS for Fluent Bitを介してAmazon CloudWatch Logsに送信します。

この時、nginxのログ出力設定を、AWS for Fluent Bitに設定した環境変数でコントロールできるか？というのを見てみたいと思います。

環境変数の注入元は、AWS Systems Manager パラメーターストアを使用することにします。

環境は、Terraformで構築します。

環境

$ terraform version
Terraform v0.13.3
+ provider registry.terraform.io/hashicorp/aws v3.8.0


$ aws --version
aws-cli/2.0.52 Python/3.7.3 Linux/4.15.0-112-generic exe/x86_64.ubuntu.20

AWSのクレデンシャルは、環境変数で設定しているものとします。

$ export AWS_ACCESS_KEY_ID=...
$ export AWS_SECRET_ACCESS_KEY=...
$ export AWS_DEFAULT_REGION=ap-northeast-1

AWS Fargateクラスターを構築する

それでは、AWS Fargateクラスターを構築します。

定義はTerraformで書き、VPCやALB、セキュリティグループ等はあとでまとめて載せます。

main.tf

locals {
  vpc_id = module.vpc.vpc_id

  private_subnets                = module.vpc.private_subnets
  nginx_service_security_groups  = [module.nginx_service_sg.this_security_group_id]
  load_balancer_target_group_arn = module.load_balancer.target_group_arns[0]
}

まずは、Amazon ECSサービスに割り当てる、IAMロールを作成します。

タスク実行用のIAMロールと、タスク用のIAMロールの2つを作成します。

Assume Role。

data "aws_iam_policy_document" "assume_role" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["ecs-tasks.amazonaws.com"]
    }
  }
}

タスク実行用のIAMロール。

data "aws_iam_policy" "ecs_task_execution_role_policy" {
  arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

data "aws_iam_policy_document" "ecs_task_execution_role_policy_document" {
  source_json = data.aws_iam_policy.ecs_task_execution_role_policy.policy

  statement {
    effect = "Allow"

    actions = [
      "ssm:GetParameters",
      "kms:Decrypt"
    ]

    resources = ["*"]
  }
}

resource "aws_iam_role" "ecs_task_execution_role" {
  name               = "MyEcsTaskExecutionRole"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

resource "aws_iam_policy" "ecs_task_execution_policy" {
  name   = "MyEcsTaskExecutionPolicy"
  policy = data.aws_iam_policy_document.ecs_task_execution_role_policy_document.json
}

resource "aws_iam_role_policy_attachment" "ecs_task_execution_role_policy_attachment" {
  role       = aws_iam_role.ecs_task_execution_role.name
  policy_arn = aws_iam_policy.ecs_task_execution_policy.arn
}

Amazon Systems Manager パラメーターストアから値を取得するので、AmazonECSTaskExecutionRolePolicyをベースにして権限を追加しています。

data "aws_iam_policy_document" "ecs_task_execution_role_policy_document" {
  source_json = data.aws_iam_policy.ecs_task_execution_role_policy.policy

  statement {
    effect = "Allow"

    actions = [
      "ssm:GetParameters",
      "kms:Decrypt"
    ]

    resources = ["*"]
  }
}

タスクに割り当てるIAMロール。AWS for Fluent BitからAmazon CloudWatch Logsにログを送信するための権限を割り当てています。

data "aws_iam_policy_document" "ecs_task_role_policy_document" {
  statement {
    effect = "Allow"

    actions = [
      "logs:DescribeLogStreams",
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents"
    ]

    resources = ["*"]
  }
}

resource "aws_iam_policy" "ecs_task_role_policy" {
  name   = "MyEcsTaskPolicy"
  policy = data.aws_iam_policy_document.ecs_task_role_policy_document.json
}

resource "aws_iam_role" "ecs_task_role" {
  name               = "MyEcsTaskRole"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

resource "aws_iam_role_policy_attachment" "ecs_task_role_policy_attachment" {
  role       = aws_iam_role.ecs_task_role.name
  policy_arn = aws_iam_policy.ecs_task_role_policy.arn
}

コンテナのログを保存するAmazon CloudWatch Logsのロググループも作成。

resource "aws_cloudwatch_log_group" "nginx_container_log_group" {
  name = "/fargate/containers/nginx"
}

resource "aws_cloudwatch_log_group" "firelens_container_log_group" {
  name = "/fargate/containers/firelens"
}

AWS FireLensで動かしているFluent Bitのログも、Amazon CloudWatch Logsに送ることにします。

nginx用のコンテナのログ出力先は、Amazon Systems Manager パラメーターストアのSecureStringとして作成しておきましょう。

resource "aws_ssm_parameter" "nginx_log_group" {
  name  = "/config/fargate/containers/nginx/log_group"
  type  = "SecureString"
  value = aws_cloudwatch_log_group.nginx_container_log_group.name
}

resource "aws_ssm_parameter" "nginx_log_stream_prefix" {
  name  = "/config/fargate/containers/nginx/log_stream_prefix"
  type  = "SecureString"
  value = "nginx-"
}

大した情報ではないんですが、今回はSecureStringで。

resource "aws_ecs_cluster" "nginx" {
  name = "nginx-cluster"
}

resource "aws_ecs_task_definition" "nginx" {
  family                   = "nginx-task-definition"
  cpu                      = "512"
  memory                   = "1024"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  execution_role_arn       = aws_iam_role.ecs_task_execution_role.arn
  task_role_arn            = aws_iam_role.ecs_task_role.arn

  container_definitions = <<JSON
  [
    {
      "name": "nginx",
      "image": "nginx:1.19.2",
      "essential": true,
      "portMappings": [
        {
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "cpu": 256,
      "memory": 512,
      "logConfiguration": {
        "logDriver": "awsfirelens",
        "options": {
          "Name": "cloudwatch",
          "region": "ap-northeast-1",
          "log_group_name": "$${NGINX_LOG_GROUP}",
          "log_stream_prefix": "$${NGINX_LOG_STREAM_PREFIX}",
          "auto_create_group": "false"
        }
      }
    },
    {
      "name": "log_router",
      "image": "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest",
      "essential": true,
      "cpu": 256,
      "memory": 512,
      "firelensConfiguration": {
        "type": "fluentbit"
      },
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/firelens",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "firelens"
        }
      },
      "secrets": [
        {
          "name": "NGINX_LOG_GROUP",
          "valueFrom": "/config/fargate/containers/nginx/log_group"
        },
        {
          "name": "NGINX_LOG_STREAM_PREFIX",
          "valueFrom": "/config/fargate/containers/nginx/log_stream_prefix"
        }
      ]
    }
  ]
  JSON
}

resource "aws_ecs_service" "nginx" {
  name             = "nginx-service"
  cluster          = aws_ecs_cluster.nginx.arn
  task_definition  = aws_ecs_task_definition.nginx.arn
  desired_count    = 3
  launch_type      = "FARGATE"
  platform_version = "1.4.0"

  deployment_minimum_healthy_percent = 50

  network_configuration {
    assign_public_ip = false
    security_groups  = local.nginx_service_security_groups
    subnets          = local.private_subnets
  }

  load_balancer {
    target_group_arn = local.load_balancer_target_group_arn
    container_name   = "nginx"
    container_port   = 80
  }
}

先程定義したIAMロールは、Amazon ECSサービスに割り当てます。

resource "aws_ecs_service" "nginx" {
  name             = "nginx-service"
  cluster          = aws_ecs_cluster.nginx.arn
  task_definition  = aws_ecs_task_definition.nginx.arn
  desired_count    = 3
  launch_type      = "FARGATE"
  platform_version = "1.4.0"

タスク定義の方ですが、nginxとAWS for Fluent Bitの定義をそれぞれ見ていきます。

nginx側。

    {
      "name": "nginx",
      "image": "nginx:1.19.2",
      "essential": true,
      "portMappings": [
        {
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "cpu": 256,
      "memory": 512,
      "logConfiguration": {
        "logDriver": "awsfirelens",
        "options": {
          "Name": "cloudwatch",
          "region": "ap-northeast-1",
          "log_group_name": "$${NGINX_LOG_GROUP}",
          "log_stream_prefix": "$${NGINX_LOG_STREAM_PREFIX}",
          "auto_create_group": "false"
        }
      }
    },

AWS FireLensを介してAmazon CloudWatch Logsにログを送信するわけですが、この送信先ロググループおよびログストリームは環境変数で取得することにします。

      "logConfiguration": {
        "logDriver": "awsfirelens",
        "options": {
          "Name": "cloudwatch",
          "region": "ap-northeast-1",
          "log_group_name": "$${NGINX_LOG_GROUP}",
          "log_stream_prefix": "$${NGINX_LOG_STREAM_PREFIX}",
          "auto_create_group": "false"
        }
      }

ちなみに、$$となっているのはTerraformでエスケープを行うのが理由なので、Fluent Bit上での環境変数の参照は${NGINX_LOG_GROUP}や${NGINX_LOG_STREAM_PREFIX}とするのが正解です。

Variables

つまり、Fluent Bitの定義としては、こういう定義が生成されるはずです。

[OUTPUT]
    Name   cloudwatch
    Match  nginx-firelens*
    region ap-northeast-1
    log_group_name ${NGINX_LOG_GROUP}
    log_stream_prefix ${NGINX_LOG_STREAM_PREFIX}
    auto_create_group false

AWS FireLensで使う、AWS for Fluent Bit側。

    {
      "name": "log_router",
      "image": "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest",
      "essential": true,
      "cpu": 256,
      "memory": 512,
      "firelensConfiguration": {
        "type": "fluentbit"
      },
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/firelens",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "firelens"
        }
      },
      "secrets": [
        {
          "name": "NGINX_LOG_GROUP",
          "valueFrom": "/config/fargate/containers/nginx/log_group"
        },
        {
          "name": "NGINX_LOG_STREAM_PREFIX",
          "valueFrom": "/config/fargate/containers/nginx/log_stream_prefix"
        }
      ]
    }

環境変数は、AWS for Fluent Bit側に注入します。

      "secrets": [
        {
          "name": "NGINX_LOG_GROUP",
          "valueFrom": "/config/fargate/containers/nginx/log_group"
        },
        {
          "name": "NGINX_LOG_STREAM_PREFIX",
          "valueFrom": "/config/fargate/containers/nginx/log_stream_prefix"
        }
      ]

これで、nginx側で定義したlogConfigurationで環境変数を使いつつ、その値はAWS for Fluent Bit側に設定した環境変数のものが利用されればOKです。

確認

では、環境を構築して

$ terraform apply

確認してみましょう。

AWS for Fluent Bitのログをtailしてみます。

$ aws logs tail --follow /fargate/containers/firelens

しばらく待っていると、起動時のログが出力されます。

2020-09-26T13:45:03.806000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a AWS for Fluent Bit Container Image Version 2.7.0
2020-09-26T13:45:04.464000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a Fluent Bit v1.5.6
2020-09-26T13:45:04.464000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a * Copyright (C) 2019-2020 The Fluent Bit Authors
2020-09-26T13:45:04.464000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a * Copyright (C) 2015-2018 Treasure Data
2020-09-26T13:45:04.464000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
2020-09-26T13:45:04.464000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a * https://fluentbit.io
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a [2020/09/26 13:45:04] [ info] [engine] started (pid=1)
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a [2020/09/26 13:45:04] [ info] [storage] version=1.0.5, initializing...
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a [2020/09/26 13:45:04] [ info] [storage] in-memory
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a [2020/09/26 13:45:04] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a [2020/09/26 13:45:04] [ info] [input:tcp:tcp.0] listening on 127.0.0.1:8877
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a [2020/09/26 13:45:04] [ info] [input:forward:forward.1] listening on unix:///var/run/fluent.sock
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a [2020/09/26 13:45:04] [ info] [input:forward:forward.2] listening on 127.0.0.1:24224
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a time="2020-09-26T13:45:04Z" level=info msg="[cloudwatch 0] plugin parameter log_group = '/fargate/containers/nginx'"
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a time="2020-09-26T13:45:04Z" level=info msg="[cloudwatch 0] plugin parameter log_stream_prefix = 'nginx-'"
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a time="2020-09-26T13:45:04Z" level=info msg="[cloudwatch 0] plugin parameter log_stream_name = ''"
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a time="2020-09-26T13:45:04Z" level=info msg="[cloudwatch 0] plugin parameter region = 'ap-northeast-1'"
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a time="2020-09-26T13:45:04Z" level=info msg="[cloudwatch 0] plugin parameter log_key = ''"
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a time="2020-09-26T13:45:04Z" level=info msg="[cloudwatch 0] plugin parameter role_arn = ''"
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a time="2020-09-26T13:45:04Z" level=info msg="[cloudwatch 0] plugin parameter new_log_group_tags = ''"
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a time="2020-09-26T13:45:04Z" level=info msg="[cloudwatch 0] plugin parameter log_retention_days = '0'"
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a time="2020-09-26T13:45:04Z" level=info msg="[cloudwatch 0] plugin parameter endpoint = ''"
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a time="2020-09-26T13:45:04Z" level=info msg="[cloudwatch 0] plugin parameter sts_endpoint = ''"
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a time="2020-09-26T13:45:04Z" level=info msg="[cloudwatch 0] plugin parameter credentials_endpoint = "
2020-09-26T13:45:04.468000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a time="2020-09-26T13:45:04Z" level=info msg="[cloudwatch 0] plugin parameter log_format = ''"
2020-09-26T13:45:04.469000+00:00 firelens/log_router/e609bfc9-0347-45cf-b348-a2e296595a1a [2020/09/26 13:45:04] [ info] [sp] stream processor started

AWS for Fluent Bitの環境変数として設定した値が、nginx側のlogConfiguration設定とうまく合わせられたようです。

msg="[cloudwatch 0] plugin parameter log_group = '/fargate/containers/nginx'"
msg="[cloudwatch 0] plugin parameter log_stream_prefix = 'nginx-'"

nginxに、curlでアクセス。

$ curl -I nginx-1423100276.ap-northeast-1.elb.amazonaws.com
HTTP/1.1 200 OK
Date: Sat, 26 Sep 2020 13:47:39 GMT
Content-Type: text/html
Content-Length: 612
Connection: keep-alive
Server: nginx/1.19.2
Last-Modified: Tue, 11 Aug 2020 14:50:35 GMT
ETag: "5f32b03b-264"
Accept-Ranges: bytes

nginxのログを見てみます。

$ aws logs tail --follow /fargate/containers/nginx

curlのログを含め、nginxのログがAmazon CloudWatch Logsに出力されていることが確認できました。

2020-09-26T13:47:39+00:00 nginx-nginx-firelens-e609bfc9-0347-45cf-b348-a2e296595a1a {"container_id":"e609bfc9-0347-45cf-b348-a2e296595a1a-2531612879","container_name":"nginx","ecs_cluster":"nginx-cluster","ecs_task_arn":"arn:aws:ecs:ap-northeast-1:xxxxxxxxxxxx:task/e609bfc9-0347-45cf-b348-a2e296595a1a","ecs_task_definition":"nginx-task-definition:20","log":"10.0.20.204 - - [26/Sep/2020:13:47:39 +0000] \"HEAD / HTTP/1.1\" 200 0 \"-\" \"curl/7.68.0\" \"124.211.189.238\"","source":"stdout"}
2020-09-26T13:47:43+00:00 nginx-nginx-firelens-e609bfc9-0347-45cf-b348-a2e296595a1a {"container_id":"e609bfc9-0347-45cf-b348-a2e296595a1a-2531612879","container_name":"nginx","ecs_cluster":"nginx-cluster","ecs_task_arn":"arn:aws:ecs:ap-northeast-1:xxxxxxxxxxxx:task/e609bfc9-0347-45cf-b348-a2e296595a1a","ecs_task_definition":"nginx-task-definition:20","log":"10.0.10.249 - - [26/Sep/2020:13:47:43 +0000] \"GET / HTTP/1.1\" 200 612 \"-\" \"ELB-HealthChecker/2.0\" \"-\"","source":"stdout"}
2020-09-26T13:47:43+00:00 nginx-nginx-firelens-0ae0c358-3bf8-4d2f-bb00-3426f64d76cc {"container_id":"0ae0c358-3bf8-4d2f-bb00-3426f64d76cc-2531612879","container_name":"nginx","ecs_cluster":"nginx-cluster","ecs_task_arn":"arn:aws:ecs:ap-northeast-1:xxxxxxxxxxxx:task/0ae0c358-3bf8-4d2f-bb00-3426f64d76cc","ecs_task_definition":"nginx-task-definition:20","log":"10.0.10.249 - - [26/Sep/2020:13:47:43 +0000] \"GET / HTTP/1.1\" 200 612 \"-\" \"ELB-HealthChecker/2.0\" \"-\"","source":"stdout"}
2020-09-26T13:47:43+00:00 nginx-nginx-firelens-fe02df02-981e-49ec-b320-4e9f3a42e553 {"container_id":"fe02df02-981e-49ec-b320-4e9f3a42e553-2531612879","container_name":"nginx","ecs_cluster":"nginx-cluster","ecs_task_arn":"arn:aws:ecs:ap-northeast-1:xxxxxxxxxxxx:task/fe02df02-981e-49ec-b320-4e9f3a42e553","ecs_task_definition":"nginx-task-definition:20","log":"10.0.10.249 - - [26/Sep/2020:13:47:43 +0000] \"GET / HTTP/1.1\" 200 612 \"-\" \"ELB-HealthChecker/2.0\" \"-\"","source":"stdout"}

これで、以下の設定ができたことになります。

AWS FireLensを使用し、サイドカーとして動かすAWS for Fluent Bitに環境変数を設定
通常のコンテナは、AWS for Fluent Bitに設定された環境変数をルックアップするようにlogConfigurationを設定

今回確認したかったこととしては、これでOKです。

知識として覚えておきましょう。

VPC〜ALBまで（〜AWS Fargateも）

最後に、省略していたVPCからALBまでの定義を含めた、全体のTerraform定義を載せておきます。

main.tf

terraform {
  required_version = "0.13.3"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "3.8.0"
    }
  }
}

provider "aws" {
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "2.54.0"

  name = "my-vpc"
  cidr = "10.0.0.0/16"

  enable_dns_hostnames = true
  enable_dns_support   = true

  azs             = ["ap-northeast-1a", "ap-northeast-1c"]
  public_subnets  = ["10.0.10.0/24", "10.0.20.0/24"]
  private_subnets = ["10.0.30.0/24", "10.0.40.0/24"]

  map_public_ip_on_launch = true

  enable_nat_gateway     = true
  single_nat_gateway     = false
  one_nat_gateway_per_az = true
}

module "load_balancer_sg" {
  source  = "terraform-aws-modules/security-group/aws//modules/http-80"
  version = "3.16.0"

  name   = "load-balancer-sg"
  vpc_id = module.vpc.vpc_id

  ingress_cidr_blocks = ["0.0.0.0/0"]
}

module "nginx_service_sg" {
  source  = "terraform-aws-modules/security-group/aws"
  version = "3.16.0"

  name   = "nginx-service-sg"
  vpc_id = module.vpc.vpc_id

  ingress_with_cidr_blocks = [
    {
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      description = "nginx-service inbound ports"
      cidr_blocks = "10.0.10.0/24"
    },
    {
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      description = "nginx-service inbound ports"
      cidr_blocks = "10.0.20.0/24"
    }
  ]

  egress_with_cidr_blocks = [
    {
      from_port   = 0
      to_port     = 0
      protocol    = "-1"
      description = "nginx-service outbound ports"
      cidr_blocks = "0.0.0.0/0"
    }
  ]
}

module "load_balancer" {
  source  = "terraform-aws-modules/alb/aws"
  version = "5.9.0"

  name = "nginx"

  vpc_id             = module.vpc.vpc_id
  load_balancer_type = "application"
  internal           = false

  subnets         = module.vpc.public_subnets
  security_groups = [module.load_balancer_sg.this_security_group_id]

  target_groups = [
    {
      backend_protocol = "HTTP"
      backend_port     = 80
      target_type      = "ip"

      health_check = {
        interval = 20
      }
    }
  ]

  http_tcp_listeners = [
    {
      port     = 80
      protocol = "HTTP"
    }
  ]
}

locals {
  vpc_id = module.vpc.vpc_id

  private_subnets                = module.vpc.private_subnets
  nginx_service_security_groups  = [module.nginx_service_sg.this_security_group_id]
  load_balancer_target_group_arn = module.load_balancer.target_group_arns[0]
}

data "aws_iam_policy_document" "assume_role" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["ecs-tasks.amazonaws.com"]
    }
  }
}

data "aws_iam_policy" "ecs_task_execution_role_policy" {
  arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

data "aws_iam_policy_document" "ecs_task_execution_role_policy_document" {
  source_json = data.aws_iam_policy.ecs_task_execution_role_policy.policy

  statement {
    effect = "Allow"

    actions = [
      "ssm:GetParameters",
      "kms:Decrypt"
    ]

    resources = ["*"]
  }
}

resource "aws_iam_role" "ecs_task_execution_role" {
  name               = "MyEcsTaskExecutionRole"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

resource "aws_iam_policy" "ecs_task_execution_policy" {
  name   = "MyEcsTaskExecutionPolicy"
  policy = data.aws_iam_policy_document.ecs_task_execution_role_policy_document.json
}

resource "aws_iam_role_policy_attachment" "ecs_task_execution_role_policy_attachment" {
  role       = aws_iam_role.ecs_task_execution_role.name
  policy_arn = aws_iam_policy.ecs_task_execution_policy.arn
}

data "aws_iam_policy_document" "ecs_task_role_policy_document" {
  statement {
    effect = "Allow"

    actions = [
      "logs:DescribeLogStreams",
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents"
    ]

    resources = ["*"]
  }
}

resource "aws_iam_policy" "ecs_task_role_policy" {
  name   = "MyEcsTaskPolicy"
  policy = data.aws_iam_policy_document.ecs_task_role_policy_document.json
}

resource "aws_iam_role" "ecs_task_role" {
  name               = "MyEcsTaskRole"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

resource "aws_iam_role_policy_attachment" "ecs_task_role_policy_attachment" {
  role       = aws_iam_role.ecs_task_role.name
  policy_arn = aws_iam_policy.ecs_task_role_policy.arn
}

resource "aws_cloudwatch_log_group" "nginx_container_log_group" {
  name = "/fargate/containers/nginx"
}

resource "aws_cloudwatch_log_group" "firelens_container_log_group" {
  name = "/fargate/containers/firelens"
}

resource "aws_ssm_parameter" "nginx_log_group" {
  name  = "/config/fargate/containers/nginx/log_group"
  type  = "SecureString"
  value = aws_cloudwatch_log_group.nginx_container_log_group.name
}

resource "aws_ssm_parameter" "nginx_log_stream_prefix" {
  name  = "/config/fargate/containers/nginx/log_stream_prefix"
  type  = "SecureString"
  value = "nginx-"
}

resource "aws_ecs_cluster" "nginx" {
  name = "nginx-cluster"
}

resource "aws_ecs_task_definition" "nginx" {
  family                   = "nginx-task-definition"
  cpu                      = "512"
  memory                   = "1024"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  execution_role_arn       = aws_iam_role.ecs_task_execution_role.arn
  task_role_arn            = aws_iam_role.ecs_task_role.arn

  container_definitions = <<JSON
  [
    {
      "name": "nginx",
      "image": "nginx:1.19.2",
      "essential": true,
      "portMappings": [
        {
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "cpu": 256,
      "memory": 512,
      "logConfiguration": {
        "logDriver": "awsfirelens",
        "options": {
          "Name": "cloudwatch",
          "region": "ap-northeast-1",
          "log_group_name": "$${NGINX_LOG_GROUP}",
          "log_stream_prefix": "$${NGINX_LOG_STREAM_PREFIX}",
          "auto_create_group": "false"
        }
      }
    },
    {
      "name": "log_router",
      "image": "906394416424.dkr.ecr.ap-northeast-1.amazonaws.com/aws-for-fluent-bit:latest",
      "essential": true,
      "cpu": 256,
      "memory": 512,
      "firelensConfiguration": {
        "type": "fluentbit"
      },
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/fargate/containers/firelens",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "firelens"
        }
      },
      "secrets": [
        {
          "name": "NGINX_LOG_GROUP",
          "valueFrom": "/config/fargate/containers/nginx/log_group"
        },
        {
          "name": "NGINX_LOG_STREAM_PREFIX",
          "valueFrom": "/config/fargate/containers/nginx/log_stream_prefix"
        }
      ]
    }
  ]
  JSON
}

resource "aws_ecs_service" "nginx" {
  name             = "nginx-service"
  cluster          = aws_ecs_cluster.nginx.arn
  task_definition  = aws_ecs_task_definition.nginx.arn
  desired_count    = 3
  launch_type      = "FARGATE"
  platform_version = "1.4.0"

  deployment_minimum_healthy_percent = 50

  network_configuration {
    assign_public_ip = false
    security_groups  = local.nginx_service_security_groups
    subnets          = local.private_subnets
  }

  load_balancer {
    target_group_arn = local.load_balancer_target_group_arn
    container_name   = "nginx"
    container_port   = 80
  }
}

AWS FireLensで動かしているFluent Bitに設定した環境変数を使って、他のコンテナのログ設定を行う

What's？

お題

環境

AWS Fargateクラスターを構築する

確認

VPC〜ALBまで（〜AWS Fargateも）