Fargateで起動するPythonの定期バッチをAWS-CDK(Python)で構築する
Lambdaの15分制限を突破するために、Fargateを使用することにした。
CDKのPythonで書かれたサンプルコードをあまり見ないので、備忘録がわりに投稿してみる。
AWS-CDKを触ったことがない方はこちらのワークショップを一通り終わらせるとイメージが付きやすいと思います。
この記事のソースはGitHubのこちらのリポジトリにあります。
これを作るよー
開発環境構築
今回はCloud9で構築したが、CDKとPythonとDockerを入れればMacでもWindowsでも動く、はず。
AWS CDK
Cloud9には現時点で最初からCDKが入っていないので、事前にインストールすること。
$ npm install -g aws-cdk
$ cdk --version
1.19.0 (build 5597bbe)
Python実行環境
cloud9の場合現時点でpipがPython2系なので3系に変更する。
$ sudo update-alternatives --config python
There are 2 programs which provide 'python'.
Selection Command
-----------------------------------------------
*+ 1 /usr/bin/python2.7
2 /usr/bin/python3.6
Enter to keep the current selection[+], or type selection number: 2
pipenvを使うと仮想環境と必要なライブラリを同時に管理できて便利。
$ sudo pip install pipenv
その他、入っていなければDockerをインストールすること。
プロジェクトの初期化
-
CDKの初期化
# 適当なディレクトリを作成 $ mkdir aws-cdk-fargate-batch && cd aws-cdk-fargate-batch # 下記コマンドでCDKプロジェクトを初期化 $ cdk init app --language=python下記の様なディレクトリが構築される。
(※自動で作られるrequirements.txt等はpipenvで管理したいので削除しました)aws-cdk-fargate-batch ├── app.py ├── aws_cdk_fargate_batch │ ├── aws_cdk_fargate_batch_stack.py │ └── __init__.py ├── cdk.json -
python仮想環境の初期化
pythonの仮想環境を初期化し、必要なライブラリをインストールする。
# Python3系でpipenvを初期化 $ pipenv --python 3 # 仮想環境へ入る $ pipenv shell # 必要なライブラリをインストール $ pipenv install aws_cdk.aws_ecr aws_cdk.aws_ec2 aws_cdk.aws_logs aws_cdk.aws_events aws_cdk.aws_events_targets
これで準備が整ったのでaws_cdk_fargate_batch_stack.pyにリソースを定義していく。
ECRの作成
-
リソースの定義
まずは、Docker Imageを保管するリポジトリを作成する。
ECRを下記のように定義する。aws_cdk_fargate_batch_stack.pyfrom aws_cdk import core from aws_cdk import aws_ecr class AwsCdkFargateBatchStack(core.Stack): def __init__(self, scope: core.Construct, id: str, **kwargs) -> None: super().__init__(scope, id, **kwargs) # ==================================== # ECR # ==================================== ecr_repository = ecr_aws.Repository( self, id='ecr_repository', repository_name='sample_repository' ) -
生成されるテンプレートの確認
$ cdk synthaws-cdk-fargate-batch.template.json{ "Resources": { "ecrrepository8944E775": { "Type": "AWS::ECR::Repository", "Properties": { "RepositoryName": "sample_repository" }, "UpdateReplacePolicy": "Retain", "DeletionPolicy": "Retain", "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/ecr_repository/Resource" } } } }CloudFormationのjsonが生成されていることが確認できる。
-
デプロイ
試しにデプロイしてみる。
$ cdk deploy aws-cdk-fargate-batch: deploying... aws-cdk-fargate-batch: creating CloudFormation changeset... 0/3 | 2:11:42 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata 0/3 | 2:11:42 PM | CREATE_IN_PROGRESS | AWS::ECR::Repository | ecr_repository (ecrrepository8944E775) 0/3 | 2:11:43 PM | CREATE_IN_PROGRESS | AWS::ECR::Repository | ecr_repository (ecrrepository8944E775) Resource creation Initiated 1/3 | 2:11:43 PM | CREATE_COMPLETE | AWS::ECR::Repository | ecr_repository (ecrrepository8944E775) 1/3 | 2:11:44 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata Resource creation Initiated 2/3 | 2:11:44 PM | CREATE_COMPLETE | AWS::CDK::Metadata | CDKMetadata 3/3 | 2:11:46 PM | CREATE_COMPLETE | AWS::CloudFormation::Stack | aws-cdk-fargate-batchECRが作成されました
テスト用のバッチをECRに登録
-
バッチのソースを管理するディレクトリとテスト用スクリプトの作成
$ mkdir batch && cd ./batch && echo "print('Hello Python from Fargate...')" > ./hello.py -
DockerFileの作成
DockerfileFROM python:3.6.8-alpine3.8 COPY ./hello.py . CMD [ "python", "./hello.py" ] -
ローカル環境でテスト
# イメージのビルド $ docker build -t test_image . # コンテナの起動 $ docker run -it test_image Hello Python from Fargate...先ほど作成したスクリプトが実行されていることを確認。
-
ECRへPush
REPOSITORY_NAME=sample_repository ACCOUNT=<使用しているAWSアカウント> AWS_REGION=<使用しているリージョン> # Docker ログイン $(aws ecr get-login --no-include-email --region ${AWS_REGION}) # Docker ビルド & プッシュ docker build -t ${REPOSITORY_NAME} . docker tag ${REPOSITORY_NAME}:latest ${ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com/${REPOSITORY_NAME}:latest docker push ${ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com/${REPOSITORY_NAME}:latest
イメージがプッシュされました
ECS(Fargate)の定義
Fargateと配置するVPCの定義を追加する。
-
リソースの定義
aws_cdk_fargate_batch_stack.pyfrom aws_cdk import core from aws_cdk import aws_ecr from aws_cdk import aws_ec2 from aws_cdk import aws_ecs from aws_cdk import aws_logs from aws_cdk import aws_events from aws_cdk import aws_events_targets class AwsCdkFargateBatchStack(core.Stack): def __init__(self, scope: core.Construct, id: str, **kwargs) -> None: super().__init__(scope, id, **kwargs) # ==================================== # ECR # ==================================== ecr_repository = aws_ecr.Repository( self, id='ecr_repository', repository_name='sample_repository' ) # ==================================== # VPC # ==================================== vpc = aws_ec2.Vpc(self, id='vpc', cidr='10.0.0.0/16', max_azs=2, nat_gateways=1, vpn_gateway=False ) # ==================================== # ECS # ==================================== # Create ecs cluester. ecs_cluster = aws_ecs.Cluster( self, id='ecs_cluster', cluster_name='sample_fargate_batch_cluster', vpc=vpc ) # Create fargate task definition. fargate_task_definition = aws_ecs.FargateTaskDefinition( self, id='fargate-task-definition', cpu=256, memory_limit_mib=512, family='fargate-task-definition' ) # Add container to task definition. fargate_task_definition.add_container( id='container', image=aws_ecs.ContainerImage.from_ecr_repository(ecr_repository), logging=aws_ecs.LogDriver.aws_logs( stream_prefix='ecs', log_group=aws_logs.LogGroup( self, id='log-group', log_group_name='/ecs/fargate/fargate-batch' ) ) ) # Create cloud watch event rule. rule = aws_events.Rule( self, id='rule', rule_name='execute-task-rule', description='Event rule to execute ecs task.', schedule=aws_events.Schedule.cron( day=None, hour=None, minute='*/5', # execute by every 5 minutes. month=None, week_day=None, year=None ) ) rule.add_target( target=aws_events_targets.EcsTask( cluster=ecs_cluster, task_definition=fargate_task_definition, task_count=1 ) ) -
生成されるテンプレートの確認
$ cdk synth下記Cfnテンプレートが生成される(長いので折りたたみ)
{ "Resources": { "ecrrepository8944E775": { "Type": "AWS::ECR::Repository", "Properties": { "RepositoryName": "sample_repository" }, "UpdateReplacePolicy": "Retain", "DeletionPolicy": "Retain", "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/ecr_repository/Resource" } }, "vpcA2121C38": { "Type": "AWS::EC2::VPC", "Properties": { "CidrBlock": "10.0.0.0/16", "EnableDnsHostnames": true, "EnableDnsSupport": true, "InstanceTenancy": "default", "Tags": [ { "Key": "Name", "Value": "aws-cdk-fargate-batch/vpc" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/Resource" } }, "vpcPublicSubnet1Subnet2E65531E": { "Type": "AWS::EC2::Subnet", "Properties": { "CidrBlock": "10.0.0.0/18", "VpcId": { "Ref": "vpcA2121C38" }, "AvailabilityZone": { "Fn::Select": [ 0, { "Fn::GetAZs": "" } ] }, "MapPublicIpOnLaunch": true, "Tags": [ { "Key": "Name", "Value": "aws-cdk-fargate-batch/vpc/PublicSubnet1" }, { "Key": "aws-cdk:subnet-name", "Value": "Public" }, { "Key": "aws-cdk:subnet-type", "Value": "Public" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet1/Subnet" } }, "vpcPublicSubnet1RouteTable48A2DF9B": { "Type": "AWS::EC2::RouteTable", "Properties": { "VpcId": { "Ref": "vpcA2121C38" }, "Tags": [ { "Key": "Name", "Value": "aws-cdk-fargate-batch/vpc/PublicSubnet1" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet1/RouteTable" } }, "vpcPublicSubnet1RouteTableAssociation5D3F4579": { "Type": "AWS::EC2::SubnetRouteTableAssociation", "Properties": { "RouteTableId": { "Ref": "vpcPublicSubnet1RouteTable48A2DF9B" }, "SubnetId": { "Ref": "vpcPublicSubnet1Subnet2E65531E" } }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet1/RouteTableAssociation" } }, "vpcPublicSubnet1DefaultRoute10708846": { "Type": "AWS::EC2::Route", "Properties": { "RouteTableId": { "Ref": "vpcPublicSubnet1RouteTable48A2DF9B" }, "DestinationCidrBlock": "0.0.0.0/0", "GatewayId": { "Ref": "vpcIGWE57CBDCA" } }, "DependsOn": [ "vpcVPCGW7984C166" ], "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet1/DefaultRoute" } }, "vpcPublicSubnet1EIPDA49DCBE": { "Type": "AWS::EC2::EIP", "Properties": { "Domain": "vpc", "Tags": [ { "Key": "Name", "Value": "aws-cdk-fargate-batch/vpc/PublicSubnet1" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet1/EIP" } }, "vpcPublicSubnet1NATGateway9C16659E": { "Type": "AWS::EC2::NatGateway", "Properties": { "AllocationId": { "Fn::GetAtt": [ "vpcPublicSubnet1EIPDA49DCBE", "AllocationId" ] }, "SubnetId": { "Ref": "vpcPublicSubnet1Subnet2E65531E" }, "Tags": [ { "Key": "Name", "Value": "aws-cdk-fargate-batch/vpc/PublicSubnet1" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet1/NATGateway" } }, "vpcPublicSubnet2Subnet009B674F": { "Type": "AWS::EC2::Subnet", "Properties": { "CidrBlock": "10.0.64.0/18", "VpcId": { "Ref": "vpcA2121C38" }, "AvailabilityZone": { "Fn::Select": [ 1, { "Fn::GetAZs": "" } ] }, "MapPublicIpOnLaunch": true, "Tags": [ { "Key": "Name", "Value": "aws-cdk-fargate-batch/vpc/PublicSubnet2" }, { "Key": "aws-cdk:subnet-name", "Value": "Public" }, { "Key": "aws-cdk:subnet-type", "Value": "Public" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet2/Subnet" } }, "vpcPublicSubnet2RouteTableEB40D4CB": { "Type": "AWS::EC2::RouteTable", "Properties": { "VpcId": { "Ref": "vpcA2121C38" }, "Tags": [ { "Key": "Name", "Value": "aws-cdk-fargate-batch/vpc/PublicSubnet2" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet2/RouteTable" } }, "vpcPublicSubnet2RouteTableAssociation21F81B59": { "Type": "AWS::EC2::SubnetRouteTableAssociation", "Properties": { "RouteTableId": { "Ref": "vpcPublicSubnet2RouteTableEB40D4CB" }, "SubnetId": { "Ref": "vpcPublicSubnet2Subnet009B674F" } }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet2/RouteTableAssociation" } }, "vpcPublicSubnet2DefaultRouteA1EC0F60": { "Type": "AWS::EC2::Route", "Properties": { "RouteTableId": { "Ref": "vpcPublicSubnet2RouteTableEB40D4CB" }, "DestinationCidrBlock": "0.0.0.0/0", "GatewayId": { "Ref": "vpcIGWE57CBDCA" } }, "DependsOn": [ "vpcVPCGW7984C166" ], "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PublicSubnet2/DefaultRoute" } }, "vpcPrivateSubnet1Subnet934893E8": { "Type": "AWS::EC2::Subnet", "Properties": { "CidrBlock": "10.0.128.0/18", "VpcId": { "Ref": "vpcA2121C38" }, "AvailabilityZone": { "Fn::Select": [ 0, { "Fn::GetAZs": "" } ] }, "MapPublicIpOnLaunch": false, "Tags": [ { "Key": "Name", "Value": "aws-cdk-fargate-batch/vpc/PrivateSubnet1" }, { "Key": "aws-cdk:subnet-name", "Value": "Private" }, { "Key": "aws-cdk:subnet-type", "Value": "Private" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet1/Subnet" } }, "vpcPrivateSubnet1RouteTableB41A48CC": { "Type": "AWS::EC2::RouteTable", "Properties": { "VpcId": { "Ref": "vpcA2121C38" }, "Tags": [ { "Key": "Name", "Value": "aws-cdk-fargate-batch/vpc/PrivateSubnet1" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet1/RouteTable" } }, "vpcPrivateSubnet1RouteTableAssociation67945127": { "Type": "AWS::EC2::SubnetRouteTableAssociation", "Properties": { "RouteTableId": { "Ref": "vpcPrivateSubnet1RouteTableB41A48CC" }, "SubnetId": { "Ref": "vpcPrivateSubnet1Subnet934893E8" } }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet1/RouteTableAssociation" } }, "vpcPrivateSubnet1DefaultRoute1AA8E2E5": { "Type": "AWS::EC2::Route", "Properties": { "RouteTableId": { "Ref": "vpcPrivateSubnet1RouteTableB41A48CC" }, "DestinationCidrBlock": "0.0.0.0/0", "NatGatewayId": { "Ref": "vpcPublicSubnet1NATGateway9C16659E" } }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet1/DefaultRoute" } }, "vpcPrivateSubnet2Subnet7031C2BA": { "Type": "AWS::EC2::Subnet", "Properties": { "CidrBlock": "10.0.192.0/18", "VpcId": { "Ref": "vpcA2121C38" }, "AvailabilityZone": { "Fn::Select": [ 1, { "Fn::GetAZs": "" } ] }, "MapPublicIpOnLaunch": false, "Tags": [ { "Key": "Name", "Value": "aws-cdk-fargate-batch/vpc/PrivateSubnet2" }, { "Key": "aws-cdk:subnet-name", "Value": "Private" }, { "Key": "aws-cdk:subnet-type", "Value": "Private" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet2/Subnet" } }, "vpcPrivateSubnet2RouteTable7280F23E": { "Type": "AWS::EC2::RouteTable", "Properties": { "VpcId": { "Ref": "vpcA2121C38" }, "Tags": [ { "Key": "Name", "Value": "aws-cdk-fargate-batch/vpc/PrivateSubnet2" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet2/RouteTable" } }, "vpcPrivateSubnet2RouteTableAssociation007E94D3": { "Type": "AWS::EC2::SubnetRouteTableAssociation", "Properties": { "RouteTableId": { "Ref": "vpcPrivateSubnet2RouteTable7280F23E" }, "SubnetId": { "Ref": "vpcPrivateSubnet2Subnet7031C2BA" } }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet2/RouteTableAssociation" } }, "vpcPrivateSubnet2DefaultRouteB0E07F99": { "Type": "AWS::EC2::Route", "Properties": { "RouteTableId": { "Ref": "vpcPrivateSubnet2RouteTable7280F23E" }, "DestinationCidrBlock": "0.0.0.0/0", "NatGatewayId": { "Ref": "vpcPublicSubnet1NATGateway9C16659E" } }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/PrivateSubnet2/DefaultRoute" } }, "vpcIGWE57CBDCA": { "Type": "AWS::EC2::InternetGateway", "Properties": { "Tags": [ { "Key": "Name", "Value": "aws-cdk-fargate-batch/vpc" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/IGW" } }, "vpcVPCGW7984C166": { "Type": "AWS::EC2::VPCGatewayAttachment", "Properties": { "VpcId": { "Ref": "vpcA2121C38" }, "InternetGatewayId": { "Ref": "vpcIGWE57CBDCA" } }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/vpc/VPCGW" } }, "ecscluster8A6F0609": { "Type": "AWS::ECS::Cluster", "Properties": { "ClusterName": "sample_fargate_batch_cluster" }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/ecs_cluster/Resource" } }, "fargatetaskdefinitionTaskRole33DCBDA3": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": "ecs-tasks.amazonaws.com" } } ], "Version": "2012-10-17" } }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/TaskRole/Resource" } }, "fargatetaskdefinitionE6399C98": { "Type": "AWS::ECS::TaskDefinition", "Properties": { "ContainerDefinitions": [ { "Essential": true, "Image": { "Fn::Join": [ "", [ { "Fn::Select": [ 4, { "Fn::Split": [ ":", { "Fn::GetAtt": [ "ecrrepository8944E775", "Arn" ] } ] } ] }, ".dkr.ecr.", { "Fn::Select": [ 3, { "Fn::Split": [ ":", { "Fn::GetAtt": [ "ecrrepository8944E775", "Arn" ] } ] } ] }, ".", { "Ref": "AWS::URLSuffix" }, "/", { "Ref": "ecrrepository8944E775" }, ":latest" ] ] }, "LogConfiguration": { "LogDriver": "awslogs", "Options": { "awslogs-group": { "Ref": "loggroupB02AAEB1" }, "awslogs-stream-prefix": "ecs", "awslogs-region": { "Ref": "AWS::Region" } } }, "Name": "container" } ], "Cpu": "256", "ExecutionRoleArn": { "Fn::GetAtt": [ "fargatetaskdefinitionExecutionRoleAB4EA2DB", "Arn" ] }, "Family": "fargate-task-definition", "Memory": "512", "NetworkMode": "awsvpc", "RequiresCompatibilities": [ "FARGATE" ], "TaskRoleArn": { "Fn::GetAtt": [ "fargatetaskdefinitionTaskRole33DCBDA3", "Arn" ] } }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/Resource" } }, "fargatetaskdefinitionExecutionRoleAB4EA2DB": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": "ecs-tasks.amazonaws.com" } } ], "Version": "2012-10-17" } }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/ExecutionRole/Resource" } }, "fargatetaskdefinitionExecutionRoleDefaultPolicyF98473BD": { "Type": "AWS::IAM::Policy", "Properties": { "PolicyDocument": { "Statement": [ { "Action": [ "ecr:BatchCheckLayerAvailability", "ecr:GetDownloadUrlForLayer", "ecr:BatchGetImage" ], "Effect": "Allow", "Resource": { "Fn::GetAtt": [ "ecrrepository8944E775", "Arn" ] } }, { "Action": "ecr:GetAuthorizationToken", "Effect": "Allow", "Resource": "*" }, { "Action": [ "logs:CreateLogStream", "logs:PutLogEvents" ], "Effect": "Allow", "Resource": { "Fn::GetAtt": [ "loggroupB02AAEB1", "Arn" ] } } ], "Version": "2012-10-17" }, "PolicyName": "fargatetaskdefinitionExecutionRoleDefaultPolicyF98473BD", "Roles": [ { "Ref": "fargatetaskdefinitionExecutionRoleAB4EA2DB" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/ExecutionRole/DefaultPolicy/Resource" } }, "fargatetaskdefinitionSecurityGroup1C5AEBC9": { "Type": "AWS::EC2::SecurityGroup", "Properties": { "GroupDescription": "aws-cdk-fargate-batch/fargate-task-definition/SecurityGroup", "SecurityGroupEgress": [ { "CidrIp": "0.0.0.0/0", "Description": "Allow all outbound traffic by default", "IpProtocol": "-1" } ], "VpcId": { "Ref": "vpcA2121C38" } }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/SecurityGroup/Resource" } }, "fargatetaskdefinitionEventsRole53D525F6": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": "events.amazonaws.com" } } ], "Version": "2012-10-17" } }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/EventsRole/Resource" } }, "fargatetaskdefinitionEventsRoleDefaultPolicy04BCE974": { "Type": "AWS::IAM::Policy", "Properties": { "PolicyDocument": { "Statement": [ { "Action": "ecs:RunTask", "Condition": { "ArnEquals": { "ecs:cluster": { "Fn::GetAtt": [ "ecscluster8A6F0609", "Arn" ] } } }, "Effect": "Allow", "Resource": { "Ref": "fargatetaskdefinitionE6399C98" } }, { "Action": "iam:PassRole", "Effect": "Allow", "Resource": { "Fn::GetAtt": [ "fargatetaskdefinitionExecutionRoleAB4EA2DB", "Arn" ] } }, { "Action": "iam:PassRole", "Effect": "Allow", "Resource": { "Fn::GetAtt": [ "fargatetaskdefinitionTaskRole33DCBDA3", "Arn" ] } } ], "Version": "2012-10-17" }, "PolicyName": "fargatetaskdefinitionEventsRoleDefaultPolicy04BCE974", "Roles": [ { "Ref": "fargatetaskdefinitionEventsRole53D525F6" } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/fargate-task-definition/EventsRole/DefaultPolicy/Resource" } }, "loggroupB02AAEB1": { "Type": "AWS::Logs::LogGroup", "Properties": { "LogGroupName": "/ecs/fargate/fargate-batch", "RetentionInDays": 731 }, "UpdateReplacePolicy": "Retain", "DeletionPolicy": "Retain", "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/log-group/Resource" } }, "ruleF2C1DCDC": { "Type": "AWS::Events::Rule", "Properties": { "Description": "Event rule to execute ecs task.", "Name": "execute-task-rule", "ScheduleExpression": "cron(*/5 * * * ? *)", "State": "ENABLED", "Targets": [ { "Arn": { "Fn::GetAtt": [ "ecscluster8A6F0609", "Arn" ] }, "EcsParameters": { "LaunchType": "FARGATE", "NetworkConfiguration": { "AwsVpcConfiguration": { "AssignPublicIp": "DISABLED", "SecurityGroups": [ { "Fn::GetAtt": [ "fargatetaskdefinitionSecurityGroup1C5AEBC9", "GroupId" ] } ], "Subnets": [ { "Ref": "vpcPrivateSubnet1Subnet934893E8" }, { "Ref": "vpcPrivateSubnet2Subnet7031C2BA" } ] } }, "TaskCount": 1, "TaskDefinitionArn": { "Ref": "fargatetaskdefinitionE6399C98" } }, "Id": "Target0", "Input": "{}", "RoleArn": { "Fn::GetAtt": [ "fargatetaskdefinitionEventsRole53D525F6", "Arn" ] } } ] }, "Metadata": { "aws:cdk:path": "aws-cdk-fargate-batch/rule/Resource" } } } }１００行足らずのPythonコードで、
800行弱のテンプレートを書かなくて良くなると思うとCDKの強みが感じられますね。また、ディフォルトのVpcは、多少オーバースペックなものが出来上がるので、azやnat_gateways等の数に制限をかけています。
-
デプロイ
下記コマンドでデプロイすると実際に各リソースが作成される。
$ cdk deploy
3-1. VPC
3-2. ECS Cluster
3-3. ECS タスク定義
3-4. CloudWatch Event Rule
バッチ実行結果確認
CloudWatch Logsを見ると定期実行されていることが確認できる。
ログを見ると、hello.pyから出力されています
お掃除
下記コマンドで作成したリソースを削除する。
$ cdk ls
aws-cdk-fargate-batch
$ cdk destroy
cdk destroy aws-cdk-fargate-batch
Are you sure you want to delete: aws-cdk-fargate-batch (y/n)? y
※ECRとCloudWatchLogGroupは自動では消えません。手動で削除する必要あるので注意。
まとめ
これまでFargate自体触ったことがなかったが、
一度手動でマネジメントコンソールから構築後、同じイメージでCDKで定義できた。
CDKを使うとRole周りを良いようにしてくれるので、テンプレートの記述量が圧倒的に減って嬉しい。
これをCodePipelineを使用してCI/CDに乗せる方法を書きました！
AWS CDKをデプロイするCodepipelineをCDK(Python)で構築する