Go to Qiita Advent Calendar 2024 Top
LoginSignup
12
9

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

@joe-king-sh(Joe King)

Fargateで起動するPythonの定期バッチをAWS-CDKで構築する

Last updated at Posted at 2019-12-31

Fargateで起動するPythonの定期バッチをAWS-CDK(Python)で構築する

Lambdaの15分制限を突破するために、Fargateを使用することにした。
CDKのPythonで書かれたサンプルコードをあまり見ないので、備忘録がわりに投稿してみる。

AWS-CDKを触ったことがない方はこちらのワークショップを一通り終わらせるとイメージが付きやすいと思います。
この記事のソースはGitHubのこちらのリポジトリにあります。

これを作るよー
image.png

開発環境構築

今回はCloud9で構築したが、CDKとPythonとDockerを入れればMacでもWindowsでも動く、はず。

AWS CDK

Cloud9には現時点で最初からCDKが入っていないので、事前にインストールすること。

$ npm install -g aws-cdk

$ cdk --version
1.19.0 (build 5597bbe)

Python実行環境

cloud9の場合現時点でpipがPython2系なので3系に変更する。

$ sudo update-alternatives --config python

There are 2 programs which provide 'python'.

  Selection    Command
-----------------------------------------------
*+ 1           /usr/bin/python2.7
   2           /usr/bin/python3.6

Enter to keep the current selection[+], or type selection number: 2

pipenvを使うと仮想環境と必要なライブラリを同時に管理できて便利。

$ sudo pip install pipenv

その他、入っていなければDockerをインストールすること。

プロジェクトの初期化

  1. CDKの初期化

    # 適当なディレクトリを作成
$ mkdir aws-cdk-fargate-batch && cd aws-cdk-fargate-batch
# 下記コマンドでCDKプロジェクトを初期化
$ cdk init app --language=python

    下記の様なディレクトリが構築される。
    (※自動で作られるrequirements.txt等はpipenvで管理したいので削除しました)

    aws-cdk-fargate-batch
├── app.py
├── aws_cdk_fargate_batch
│   ├── aws_cdk_fargate_batch_stack.py
│   └── __init__.py
├── cdk.json

  2. python仮想環境の初期化

    pythonの仮想環境を初期化し、必要なライブラリをインストールする。

    # Python3系でpipenvを初期化
$ pipenv --python 3
# 仮想環境へ入る
$ pipenv shell
# 必要なライブラリをインストール
$ pipenv install aws_cdk.aws_ecr aws_cdk.aws_ec2 aws_cdk.aws_logs aws_cdk.aws_events aws_cdk.aws_events_targets

これで準備が整ったのでaws_cdk_fargate_batch_stack.pyにリソースを定義していく。

ECRの作成

  1. リソースの定義

    まずは、Docker Imageを保管するリポジトリを作成する。
    ECRを下記のように定義する。

    aws_cdk_fargate_batch_stack.py
    from aws_cdk import core
from aws_cdk import aws_ecr

class AwsCdkFargateBatchStack(core.Stack):

    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        # ====================================
        # ECR
        # ====================================
        ecr_repository = ecr_aws.Repository(
            self,
            id='ecr_repository',
            repository_name='sample_repository'
        )

  2. 生成されるテンプレートの確認

    $ cdk synth
    aws-cdk-fargate-batch.template.json
    {
  "Resources": {
    "ecrrepository8944E775": {
      "Type": "AWS::ECR::Repository",
      "Properties": {
        "RepositoryName": "sample_repository"
      },
      "UpdateReplacePolicy": "Retain",
      "DeletionPolicy": "Retain",
      "Metadata": {
        "aws:cdk:path": "aws-cdk-fargate-batch/ecr_repository/Resource"
      }
    }
  }
}

    CloudFormationのjsonが生成されていることが確認できる。

  3. デプロイ

    試しにデプロイしてみる。

    $ cdk deploy
aws-cdk-fargate-batch: deploying...
aws-cdk-fargate-batch: creating CloudFormation changeset...
 0/3 | 2:11:42 PM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata   | CDKMetadata 
 0/3 | 2:11:42 PM | CREATE_IN_PROGRESS   | AWS::ECR::Repository | ecr_repository (ecrrepository8944E775) 
 0/3 | 2:11:43 PM | CREATE_IN_PROGRESS   | AWS::ECR::Repository | ecr_repository (ecrrepository8944E775)                             Resource creation Initiated
 1/3 | 2:11:43 PM | CREATE_COMPLETE      | AWS::ECR::Repository | ecr_repository (ecrrepository8944E775) 
 1/3 | 2:11:44 PM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata   | CDKMetadata Resource creation Initiated
 2/3 | 2:11:44 PM | CREATE_COMPLETE      | AWS::CDK::Metadata   | CDKMetadata 
 3/3 | 2:11:46 PM | CREATE_COMPLETE      | AWS::CloudFormation::Stack | aws-cdk-fargate-batch

    ECRが作成されました :tada:
    image.png

テスト用のバッチをECRに登録

  1. バッチのソースを管理するディレクトリとテスト用スクリプトの作成

    $ mkdir batch && cd ./batch && echo "print('Hello Python from Fargate...')" > ./hello.py

  2. DockerFileの作成

    Dockerfile
    FROM python:3.6.8-alpine3.8

COPY ./hello.py .

CMD [ "python", "./hello.py" ]

  3. ローカル環境でテスト

    # イメージのビルド
$ docker build -t test_image . 
# コンテナの起動
$ docker run -it test_image
Hello Python from Fargate...

    先ほど作成したスクリプトが実行されていることを確認。

  4. ECRへPush

    REPOSITORY_NAME=sample_repository
ACCOUNT=<使用しているAWSアカウント>
AWS_REGION=<使用しているリージョン>

# Docker ログイン
$(aws ecr get-login --no-include-email --region ${AWS_REGION})

# Docker ビルド & プッシュ
docker build -t ${REPOSITORY_NAME} .
docker tag ${REPOSITORY_NAME}:latest ${ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com/${REPOSITORY_NAME}:latest
docker push ${ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com/${REPOSITORY_NAME}:latest


イメージがプッシュされました :tada:
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/541733/46d7b505-aa3d-93ad-60d9-4bb83c14ff68.png)


## ECS(Fargate)の定義
Fargateと配置するVPCの定義を追加する。

1. リソースの定義

    ```python:aws_cdk_fargate_batch_stack.py
    from aws_cdk import core
    from aws_cdk import aws_ecr
    from aws_cdk import aws_ec2
    from aws_cdk import aws_ecs
    from aws_cdk import aws_logs
    from aws_cdk import aws_events
    from aws_cdk import aws_events_targets
    
    
    class AwsCdkFargateBatchStack(core.Stack):
    
        def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
            super().__init__(scope, id, **kwargs)
    
            # ====================================
            # ECR
            # ====================================
            ecr_repository = aws_ecr.Repository(
                self,
                id='ecr_repository',
                repository_name='sample_repository'
            )
            
            # ====================================
            # VPC
            # ====================================
            vpc = aws_ec2.Vpc(self,
                id='vpc',
                cidr='10.0.0.0/16',
                max_azs=2,
                nat_gateways=1,
                vpn_gateway=False
            )
    
    
            # ====================================
            # ECS
            # ====================================
            # Create ecs cluester.
            ecs_cluster = aws_ecs.Cluster(
                self,
                id='ecs_cluster',
                cluster_name='sample_fargate_batch_cluster',
                vpc=vpc
            )
    
            # Create fargate task definition.
            fargate_task_definition = aws_ecs.FargateTaskDefinition(
                self,
                id='fargate-task-definition',
                cpu=256,
                memory_limit_mib=512,
                family='fargate-task-definition'
            )
    
            # Add container to task definition.
            fargate_task_definition.add_container(
                id='container',
                image=aws_ecs.ContainerImage.from_ecr_repository(ecr_repository),
                logging=aws_ecs.LogDriver.aws_logs(
                    stream_prefix='ecs',
                    log_group=aws_logs.LogGroup(
                        self,
                        id='log-group',
                        log_group_name='/ecs/fargate/fargate-batch'
                    )
                )
            )
    
            # Create cloud watch event rule.
            rule = aws_events.Rule(
                self,
                id='rule',
                rule_name='execute-task-rule',
                description='Event rule to execute ecs task.',
                schedule=aws_events.Schedule.cron(
                    day=None,
                    hour=None,
                    minute='*/5', # execute by every 5 minutes.
                    month=None,
                    week_day=None,
                    year=None
                )
            )
    
            rule.add_target(
                target=aws_events_targets.EcsTask(
                    cluster=ecs_cluster,
                    task_definition=fargate_task_definition,
                    task_count=1
                )
            )
    ```

2. 生成されるテンプレートの確認

    ```bash
    $ cdk synth
    ```

    <details><summary>下記Cfnテンプレートが生成される(長いので折りたたみ)</summary><div>

    ```python
    {
      "Resources": {
        "ecrrepository8944E775": {
          "Type": "AWS::ECR::Repository",
          "Properties": {
            "RepositoryName": "sample_repository"
          },
          "UpdateReplacePolicy": "Retain",
          "DeletionPolicy": "Retain",
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/ecr_repository/Resource"
          }
        },
        "vpcA2121C38": {
          "Type": "AWS::EC2::VPC",
          "Properties": {
            "CidrBlock": "10.0.0.0/16",
            "EnableDnsHostnames": true,
            "EnableDnsSupport": true,
            "InstanceTenancy": "default",
            "Tags": [
              {
                "Key": "Name",
                "Value": "aws-cdk-fargate-batch/vpc"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/Resource"
          }
        },
        "vpcPublicSubnet1Subnet2E65531E": {
          "Type": "AWS::EC2::Subnet",
          "Properties": {
            "CidrBlock": "10.0.0.0/18",
            "VpcId": {
              "Ref": "vpcA2121C38"
            },
            "AvailabilityZone": {
              "Fn::Select": [
                0,
                {
                  "Fn::GetAZs": ""
                }
              ]
            },
            "MapPublicIpOnLaunch": true,
            "Tags": [
              {
                "Key": "Name",
                "Value": "aws-cdk-fargate-batch/vpc/PublicSubnet1"
              },
              {
                "Key": "aws-cdk:subnet-name",
                "Value": "Public"
              },
              {
                "Key": "aws-cdk:subnet-type",
                "Value": "Public"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet1/Subnet"
          }
        },
        "vpcPublicSubnet1RouteTable48A2DF9B": {
          "Type": "AWS::EC2::RouteTable",
          "Properties": {
            "VpcId": {
              "Ref": "vpcA2121C38"
            },
            "Tags": [
              {
                "Key": "Name",
                "Value": "aws-cdk-fargate-batch/vpc/PublicSubnet1"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet1/RouteTable"
          }
        },
        "vpcPublicSubnet1RouteTableAssociation5D3F4579": {
          "Type": "AWS::EC2::SubnetRouteTableAssociation",
          "Properties": {
            "RouteTableId": {
              "Ref": "vpcPublicSubnet1RouteTable48A2DF9B"
            },
            "SubnetId": {
              "Ref": "vpcPublicSubnet1Subnet2E65531E"
            }
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet1/RouteTableAssociation"
          }
        },
        "vpcPublicSubnet1DefaultRoute10708846": {
          "Type": "AWS::EC2::Route",
          "Properties": {
            "RouteTableId": {
              "Ref": "vpcPublicSubnet1RouteTable48A2DF9B"
            },
            "DestinationCidrBlock": "0.0.0.0/0",
            "GatewayId": {
              "Ref": "vpcIGWE57CBDCA"
            }
          },
          "DependsOn": [
            "vpcVPCGW7984C166"
          ],
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet1/DefaultRoute"
          }
        },
        "vpcPublicSubnet1EIPDA49DCBE": {
          "Type": "AWS::EC2::EIP",
          "Properties": {
            "Domain": "vpc",
            "Tags": [
              {
                "Key": "Name",
                "Value": "aws-cdk-fargate-batch/vpc/PublicSubnet1"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet1/EIP"
          }
        },
        "vpcPublicSubnet1NATGateway9C16659E": {
          "Type": "AWS::EC2::NatGateway",
          "Properties": {
            "AllocationId": {
              "Fn::GetAtt": [
                "vpcPublicSubnet1EIPDA49DCBE",
                "AllocationId"
              ]
            },
            "SubnetId": {
              "Ref": "vpcPublicSubnet1Subnet2E65531E"
            },
            "Tags": [
              {
                "Key": "Name",
                "Value": "aws-cdk-fargate-batch/vpc/PublicSubnet1"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet1/NATGateway"
          }
        },
        "vpcPublicSubnet2Subnet009B674F": {
          "Type": "AWS::EC2::Subnet",
          "Properties": {
            "CidrBlock": "10.0.64.0/18",
            "VpcId": {
              "Ref": "vpcA2121C38"
            },
            "AvailabilityZone": {
              "Fn::Select": [
                1,
                {
                  "Fn::GetAZs": ""
                }
              ]
            },
            "MapPublicIpOnLaunch": true,
            "Tags": [
              {
                "Key": "Name",
                "Value": "aws-cdk-fargate-batch/vpc/PublicSubnet2"
              },
              {
                "Key": "aws-cdk:subnet-name",
                "Value": "Public"
              },
              {
                "Key": "aws-cdk:subnet-type",
                "Value": "Public"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet2/Subnet"
          }
        },
        "vpcPublicSubnet2RouteTableEB40D4CB": {
          "Type": "AWS::EC2::RouteTable",
          "Properties": {
            "VpcId": {
              "Ref": "vpcA2121C38"
            },
            "Tags": [
              {
                "Key": "Name",
                "Value": "aws-cdk-fargate-batch/vpc/PublicSubnet2"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet2/RouteTable"
          }
        },
        "vpcPublicSubnet2RouteTableAssociation21F81B59": {
          "Type": "AWS::EC2::SubnetRouteTableAssociation",
          "Properties": {
            "RouteTableId": {
              "Ref": "vpcPublicSubnet2RouteTableEB40D4CB"
            },
            "SubnetId": {
              "Ref": "vpcPublicSubnet2Subnet009B674F"
            }
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet2/RouteTableAssociation"
          }
        },
        "vpcPublicSubnet2DefaultRouteA1EC0F60": {
          "Type": "AWS::EC2::Route",
          "Properties": {
            "RouteTableId": {
              "Ref": "vpcPublicSubnet2RouteTableEB40D4CB"
            },
            "DestinationCidrBlock": "0.0.0.0/0",
            "GatewayId": {
              "Ref": "vpcIGWE57CBDCA"
            }
          },
          "DependsOn": [
            "vpcVPCGW7984C166"
          ],
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet2/DefaultRoute"
          }
        },
        "vpcPrivateSubnet1Subnet934893E8": {
          "Type": "AWS::EC2::Subnet",
          "Properties": {
            "CidrBlock": "10.0.128.0/18",
            "VpcId": {
              "Ref": "vpcA2121C38"
            },
            "AvailabilityZone": {
              "Fn::Select": [
                0,
                {
                  "Fn::GetAZs": ""
                }
              ]
            },
            "MapPublicIpOnLaunch": false,
            "Tags": [
              {
                "Key": "Name",
                "Value": "aws-cdk-fargate-batch/vpc/PrivateSubnet1"
              },
              {
                "Key": "aws-cdk:subnet-name",
                "Value": "Private"
              },
              {
                "Key": "aws-cdk:subnet-type",
                "Value": "Private"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet1/Subnet"
          }
        },
        "vpcPrivateSubnet1RouteTableB41A48CC": {
          "Type": "AWS::EC2::RouteTable",
          "Properties": {
            "VpcId": {
              "Ref": "vpcA2121C38"
            },
            "Tags": [
              {
                "Key": "Name",
                "Value": "aws-cdk-fargate-batch/vpc/PrivateSubnet1"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet1/RouteTable"
          }
        },
        "vpcPrivateSubnet1RouteTableAssociation67945127": {
          "Type": "AWS::EC2::SubnetRouteTableAssociation",
          "Properties": {
            "RouteTableId": {
              "Ref": "vpcPrivateSubnet1RouteTableB41A48CC"
            },
            "SubnetId": {
              "Ref": "vpcPrivateSubnet1Subnet934893E8"
            }
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet1/RouteTableAssociation"
          }
        },
        "vpcPrivateSubnet1DefaultRoute1AA8E2E5": {
          "Type": "AWS::EC2::Route",
          "Properties": {
            "RouteTableId": {
              "Ref": "vpcPrivateSubnet1RouteTableB41A48CC"
            },
            "DestinationCidrBlock": "0.0.0.0/0",
            "NatGatewayId": {
              "Ref": "vpcPublicSubnet1NATGateway9C16659E"
            }
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet1/DefaultRoute"
          }
        },
        "vpcPrivateSubnet2Subnet7031C2BA": {
          "Type": "AWS::EC2::Subnet",
          "Properties": {
            "CidrBlock": "10.0.192.0/18",
            "VpcId": {
              "Ref": "vpcA2121C38"
            },
            "AvailabilityZone": {
              "Fn::Select": [
                1,
                {
                  "Fn::GetAZs": ""
                }
              ]
            },
            "MapPublicIpOnLaunch": false,
            "Tags": [
              {
                "Key": "Name",
                "Value": "aws-cdk-fargate-batch/vpc/PrivateSubnet2"
              },
              {
                "Key": "aws-cdk:subnet-name",
                "Value": "Private"
              },
              {
                "Key": "aws-cdk:subnet-type",
                "Value": "Private"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet2/Subnet"
          }
        },
        "vpcPrivateSubnet2RouteTable7280F23E": {
          "Type": "AWS::EC2::RouteTable",
          "Properties": {
            "VpcId": {
              "Ref": "vpcA2121C38"
            },
            "Tags": [
              {
                "Key": "Name",
                "Value": "aws-cdk-fargate-batch/vpc/PrivateSubnet2"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet2/RouteTable"
          }
        },
        "vpcPrivateSubnet2RouteTableAssociation007E94D3": {
          "Type": "AWS::EC2::SubnetRouteTableAssociation",
          "Properties": {
            "RouteTableId": {
              "Ref": "vpcPrivateSubnet2RouteTable7280F23E"
            },
            "SubnetId": {
              "Ref": "vpcPrivateSubnet2Subnet7031C2BA"
            }
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet2/RouteTableAssociation"
          }
        },
        "vpcPrivateSubnet2DefaultRouteB0E07F99": {
          "Type": "AWS::EC2::Route",
          "Properties": {
            "RouteTableId": {
              "Ref": "vpcPrivateSubnet2RouteTable7280F23E"
            },
            "DestinationCidrBlock": "0.0.0.0/0",
            "NatGatewayId": {
              "Ref": "vpcPublicSubnet1NATGateway9C16659E"
            }
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet2/DefaultRoute"
          }
        },
        "vpcIGWE57CBDCA": {
          "Type": "AWS::EC2::InternetGateway",
          "Properties": {
            "Tags": [
              {
                "Key": "Name",
                "Value": "aws-cdk-fargate-batch/vpc"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/IGW"
          }
        },
        "vpcVPCGW7984C166": {
          "Type": "AWS::EC2::VPCGatewayAttachment",
          "Properties": {
            "VpcId": {
              "Ref": "vpcA2121C38"
            },
            "InternetGatewayId": {
              "Ref": "vpcIGWE57CBDCA"
            }
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/vpc/VPCGW"
          }
        },
        "ecscluster8A6F0609": {
          "Type": "AWS::ECS::Cluster",
          "Properties": {
            "ClusterName": "sample_fargate_batch_cluster"
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/ecs_cluster/Resource"
          }
        },
        "fargatetaskdefinitionTaskRole33DCBDA3": {
          "Type": "AWS::IAM::Role",
          "Properties": {
            "AssumeRolePolicyDocument": {
              "Statement": [
                {
                  "Action": "sts:AssumeRole",
                  "Effect": "Allow",
                  "Principal": {
                    "Service": "ecs-tasks.amazonaws.com"
                  }
                }
              ],
              "Version": "2012-10-17"
            }
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/TaskRole/Resource"
          }
        },
        "fargatetaskdefinitionE6399C98": {
          "Type": "AWS::ECS::TaskDefinition",
          "Properties": {
            "ContainerDefinitions": [
              {
                "Essential": true,
                "Image": {
                  "Fn::Join": [
                    "",
                    [
                      {
                        "Fn::Select": [
                          4,
                          {
                            "Fn::Split": [
                              ":",
                              {
                                "Fn::GetAtt": [
                                  "ecrrepository8944E775",
                                  "Arn"
                                ]
                              }
                            ]
                          }
                        ]
                      },
                      ".dkr.ecr.",
                      {
                        "Fn::Select": [
                          3,
                          {
                            "Fn::Split": [
                              ":",
                              {
                                "Fn::GetAtt": [
                                  "ecrrepository8944E775",
                                  "Arn"
                                ]
                              }
                            ]
                          }
                        ]
                      },
                      ".",
                      {
                        "Ref": "AWS::URLSuffix"
                      },
                      "/",
                      {
                        "Ref": "ecrrepository8944E775"
                      },
                      ":latest"
                    ]
                  ]
                },
                "LogConfiguration": {
                  "LogDriver": "awslogs",
                  "Options": {
                    "awslogs-group": {
                      "Ref": "loggroupB02AAEB1"
                    },
                    "awslogs-stream-prefix": "ecs",
                    "awslogs-region": {
                      "Ref": "AWS::Region"
                    }
                  }
                },
                "Name": "container"
              }
            ],
            "Cpu": "256",
            "ExecutionRoleArn": {
              "Fn::GetAtt": [
                "fargatetaskdefinitionExecutionRoleAB4EA2DB",
                "Arn"
              ]
            },
            "Family": "fargate-task-definition",
            "Memory": "512",
            "NetworkMode": "awsvpc",
            "RequiresCompatibilities": [
              "FARGATE"
            ],
            "TaskRoleArn": {
              "Fn::GetAtt": [
                "fargatetaskdefinitionTaskRole33DCBDA3",
                "Arn"
              ]
            }
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/Resource"
          }
        },
        "fargatetaskdefinitionExecutionRoleAB4EA2DB": {
          "Type": "AWS::IAM::Role",
          "Properties": {
            "AssumeRolePolicyDocument": {
              "Statement": [
                {
                  "Action": "sts:AssumeRole",
                  "Effect": "Allow",
                  "Principal": {
                    "Service": "ecs-tasks.amazonaws.com"
                  }
                }
              ],
              "Version": "2012-10-17"
            }
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/ExecutionRole/Resource"
          }
        },
        "fargatetaskdefinitionExecutionRoleDefaultPolicyF98473BD": {
          "Type": "AWS::IAM::Policy",
          "Properties": {
            "PolicyDocument": {
              "Statement": [
                {
                  "Action": [
                    "ecr:BatchCheckLayerAvailability",
                    "ecr:GetDownloadUrlForLayer",
                    "ecr:BatchGetImage"
                  ],
                  "Effect": "Allow",
                  "Resource": {
                    "Fn::GetAtt": [
                      "ecrrepository8944E775",
                      "Arn"
                    ]
                  }
                },
                {
                  "Action": "ecr:GetAuthorizationToken",
                  "Effect": "Allow",
                  "Resource": "*"
                },
                {
                  "Action": [
                    "logs:CreateLogStream",
                    "logs:PutLogEvents"
                  ],
                  "Effect": "Allow",
                  "Resource": {
                    "Fn::GetAtt": [
                      "loggroupB02AAEB1",
                      "Arn"
                    ]
                  }
                }
              ],
              "Version": "2012-10-17"
            },
            "PolicyName": "fargatetaskdefinitionExecutionRoleDefaultPolicyF98473BD",
            "Roles": [
              {
                "Ref": "fargatetaskdefinitionExecutionRoleAB4EA2DB"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/ExecutionRole/DefaultPolicy/Resource"
          }
        },
        "fargatetaskdefinitionSecurityGroup1C5AEBC9": {
          "Type": "AWS::EC2::SecurityGroup",
          "Properties": {
            "GroupDescription": "aws-cdk-fargate-batch/fargate-task-definition/SecurityGroup",
            "SecurityGroupEgress": [
              {
                "CidrIp": "0.0.0.0/0",
                "Description": "Allow all outbound traffic by default",
                "IpProtocol": "-1"
              }
            ],
            "VpcId": {
              "Ref": "vpcA2121C38"
            }
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/SecurityGroup/Resource"
          }
        },
        "fargatetaskdefinitionEventsRole53D525F6": {
          "Type": "AWS::IAM::Role",
          "Properties": {
            "AssumeRolePolicyDocument": {
              "Statement": [
                {
                  "Action": "sts:AssumeRole",
                  "Effect": "Allow",
                  "Principal": {
                    "Service": "events.amazonaws.com"
                  }
                }
              ],
              "Version": "2012-10-17"
            }
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/EventsRole/Resource"
          }
        },
        "fargatetaskdefinitionEventsRoleDefaultPolicy04BCE974": {
          "Type": "AWS::IAM::Policy",
          "Properties": {
            "PolicyDocument": {
              "Statement": [
                {
                  "Action": "ecs:RunTask",
                  "Condition": {
                    "ArnEquals": {
                      "ecs:cluster": {
                        "Fn::GetAtt": [
                          "ecscluster8A6F0609",
                          "Arn"
                        ]
                      }
                    }
                  },
                  "Effect": "Allow",
                  "Resource": {
                    "Ref": "fargatetaskdefinitionE6399C98"
                  }
                },
                {
                  "Action": "iam:PassRole",
                  "Effect": "Allow",
                  "Resource": {
                    "Fn::GetAtt": [
                      "fargatetaskdefinitionExecutionRoleAB4EA2DB",
                      "Arn"
                    ]
                  }
                },
                {
                  "Action": "iam:PassRole",
                  "Effect": "Allow",
                  "Resource": {
                    "Fn::GetAtt": [
                      "fargatetaskdefinitionTaskRole33DCBDA3",
                      "Arn"
                    ]
                  }
                }
              ],
              "Version": "2012-10-17"
            },
            "PolicyName": "fargatetaskdefinitionEventsRoleDefaultPolicy04BCE974",
            "Roles": [
              {
                "Ref": "fargatetaskdefinitionEventsRole53D525F6"
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/EventsRole/DefaultPolicy/Resource"
          }
        },
        "loggroupB02AAEB1": {
          "Type": "AWS::Logs::LogGroup",
          "Properties": {
            "LogGroupName": "/ecs/fargate/fargate-batch",
            "RetentionInDays": 731
          },
          "UpdateReplacePolicy": "Retain",
          "DeletionPolicy": "Retain",
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/log-group/Resource"
          }
        },
        "ruleF2C1DCDC": {
          "Type": "AWS::Events::Rule",
          "Properties": {
            "Description": "Event rule to execute ecs task.",
            "Name": "execute-task-rule",
            "ScheduleExpression": "cron(*/5 * * * ? *)",
            "State": "ENABLED",
            "Targets": [
              {
                "Arn": {
                  "Fn::GetAtt": [
                    "ecscluster8A6F0609",
                    "Arn"
                  ]
                },
                "EcsParameters": {
                  "LaunchType": "FARGATE",
                  "NetworkConfiguration": {
                    "AwsVpcConfiguration": {
                      "AssignPublicIp": "DISABLED",
                      "SecurityGroups": [
                        {
                          "Fn::GetAtt": [
                            "fargatetaskdefinitionSecurityGroup1C5AEBC9",
                            "GroupId"
                          ]
                        }
                      ],
                      "Subnets": [
                        {
                          "Ref": "vpcPrivateSubnet1Subnet934893E8"
                        },
                        {
                          "Ref": "vpcPrivateSubnet2Subnet7031C2BA"
                        }
                      ]
                    }
                  },
                  "TaskCount": 1,
                  "TaskDefinitionArn": {
                    "Ref": "fargatetaskdefinitionE6399C98"
                  }
                },
                "Id": "Target0",
                "Input": "{}",
                "RoleArn": {
                  "Fn::GetAtt": [
                    "fargatetaskdefinitionEventsRole53D525F6",
                    "Arn"
                  ]
                }
              }
            ]
          },
          "Metadata": {
            "aws:cdk:path": "aws-cdk-fargate-batch/rule/Resource"
          }
        }
      }
    }
    ```
    </div></details>

    １００行足らずのPythonコードで、
    800行弱のテンプレートを書かなくて良くなると思うとCDKの強みが感じられますね。

    また、ディフォルトのVpcは、多少オーバースペックなものが出来上がるので、azやnat_gateways等の数に制限をかけています。

3. デプロイ

    下記コマンドでデプロイすると実際に各リソースが作成される。

    ```bash
    $ cdk deploy
    ```

3-1. VPC
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/541733/ad188f09-50ea-7a67-5f11-9c38e6fac7fe.png)

3-2. ECS Cluster
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/541733/c191e7e3-a039-e92f-a3f5-35c2be668968.png)

3-3. ECS タスク定義
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/541733/42da815b-81b2-e6b9-9e77-e39d71b44625.png)

3-4. CloudWatch Event Rule
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/541733/1feab0ba-a6ee-dfaa-4119-24e06179bd7c.png)


## バッチ実行結果確認

CloudWatch Logsを見ると定期実行されていることが確認できる。
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/541733/46032a30-bf8f-67dc-4f45-890ed4d419d9.png)

ログを見ると、hello.pyから出力されています :tada:
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/541733/2d8a5ab3-d93c-3ed2-88b0-15b894b93c39.png)

## お掃除
下記コマンドで作成したリソースを削除する。

```bash
$ cdk ls
aws-cdk-fargate-batch
$ cdk destroy
cdk destroy aws-cdk-fargate-batch
Are you sure you want to delete: aws-cdk-fargate-batch (y/n)? y

※ECRとCloudWatchLogGroupは自動では消えません。手動で削除する必要あるので注意。

まとめ

これまでFargate自体触ったことがなかったが、
一度手動でマネジメントコンソールから構築後、同じイメージでCDKで定義できた。

CDKを使うとRole周りを良いようにしてくれるので、テンプレートの記述量が圧倒的に減って嬉しい。

これをCodePipelineを使用してCI/CDに乗せる方法を書きました！
AWS CDKをデプロイするCodepipelineをCDK(Python)で構築する

12
9
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
12
9

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?