#1. 概要
AWS CodePipelineのCodeCommitによるCICDのBlue/Greenを実装するには、Deployステージに渡されたアーティファクトのサイズ制限があり、これを超えると以下のエラーが発生し、デプロイが失敗する。
「Exception while trying to read the task definition artifact file from: appxxx」
AWSの正式なドキュメントによると、「CodeDeployToECSアクションを使用してアプリケーションをデプロイしている場合、アーティファクトの最大サイズは常に3MBです。」
「Exception: If you are using the CodeDeployToECS action to deploy applications, the maximum artifact size is always 3 MB.」
AWS CodePipelineのCodeCommitによるCICDのBlue/Greenを実装するには、Deployステージに渡されたアーティファクトのサイズ制限があり、これを超えると以下のエラーが発生し、デプロイが失敗する。
「Exception while trying to read the task definition artifact file from: appxxx」
AWSの正式なドキュメントによると、「CodeDeployToECSアクションを使用してアプリケーションをデプロイしている場合、アーティファクトの最大サイズは常に3MBです。」
「Exception: If you are using the CodeDeployToECS action to deploy applications, the maximum artifact size is always 3 MB.」
Maximum size of artifacts in a source stage | Artifacts stored in Amazon S3 buckets: 5 GB |
---|---|
Artifacts stored in CodeCommit or GitHub repositories: 1 GB | |
Exception: If you are using AWS Elastic Beanstalk to deploy applications, the maximum artifact size is always 512 MB. | |
Exception: If you are using AWS CloudFormation to deploy applications, the maximum artifact size is always 256 MB. | |
Exception: If you are using the CodeDeployToECS action to deploy Artifacts stored in CodeCommit or GitHub repositories: 1 GB | |
Exception: If you are using AWS Elastic Beanstalk to deploy applications, the maximum artifact size is always 512 MB. | |
Exception: If you are using AWS CloudFormation to deploy applications, the maximum artifact size is always 256 MB. | |
Exception: If you are using the CodeDeployToECS action to deploy applications, the maximum artifact size is always 3 MB. |
本文、CodeDeployアーティファクトサイズリミットの対応仕方を紹介する。
#2. 対処法
対処法1(S3 Pipeline):
CodePipelineのSourceステージがCodeCommitではなく、S3 に変更する。要するに、開発のソースコードなどはCodeCommitのリポジトリを利用するではなく圧縮してS3のバーケットに保存する。勿論、この場合事前にS3バーケットの作成、アクセスロールの付与など必要である。これを利用すれば、デプロイステージに渡すアーティファクトのサイズは5GBになるので、殆どのアプリ開発には対応できると思う。
詳細な実施プロシージャは以下のリンクをご参照ください。
https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-cloudformation-s3.html
S3 Pipelineで実装するCloudFormationの例を本文の後ろ部分をご参照ください。
対処法2(CodeCommit Pipeline):
Step1:
別のCodeCommitリポジトリを作成し、なかはappspec.yml、taskdef.jsonだけを格納する。CloudWatchからこのリポジトリのファイル更新や作成のイベントを捉えてCodeDeployを起動させるように、他の実際CICDに参照されないファイルを格納しても良い。筆者の場合は、Dockerfileを入れた(参照する目的だけ)。
Step2:
CodeBuild Projectのbuildspec.ymlは、DockerImageの作成やECRへのPushをスキップし(別途で実現する)、”latest”タグだけを付与するようにする。これによりCodeDeployをする際、ECR内の“latest”タグを付けたDockerImageを利用してBlue/Green実装になる(CodeDeployのInstallステージ)。
例:
# CodeBuild
CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
ServiceRole: !Ref CodeBuildServiceRole
Artifacts:
Type: CODEPIPELINE
Source:
Type: CODEPIPELINE
BuildSpec: |
version: 0.2
env:
parameter-store:
DOCKER_USER: dockerhub-user
DOCKER_TOKEN: dockerhub-token
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...
- $(aws ecr get-login --no-include-email)
- IMAGE_TAG=latest
- echo Logging in to Docker Hub...
- echo ${DOCKER_TOKEN} | docker login -u ${DOCKER_USER} --password-stdin
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker images...
- echo Writing imageDetail json...
- echo "{\"name\":\"${ContainerName}\",\"ImageURI\":\"${REPOSITORY_URI}:${IMAGE_TAG}\"}" > imageDetail.json
artifacts:
files: imageDetail.json
Environment:
PrivilegedMode: true
ComputeType: BUILD_GENERAL1_SMALL
Image: aws/codebuild/standard:4.0
Type: LINUX_CONTAINER
EnvironmentVariables:
- Name: AWS_DEFAULT_REGION
Value: !Ref AWS::Region
- Name: REPOSITORY_URI
Value: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${ECRName}
- Name: ContainerName
Value: !Ref ContainerName
- Name: DOCKER_BUILDKIT
Value: '1'
Name: !Sub ${Service}-${Env}-phld-project1
Step3:
Step1で作成されたCodeCommitリポジトリのファイルを更新または作成する際、CodeDeployを起動される。
#3. 対処用必要な環境
ECRリポジトリ
CodeCommitリポジトリ
ECSサービスとタスク
DockerImageのPush
#4. 完成したCloudFormationYamlファイル
参照するため、完成したCodePipelineを作成するCloudFormationのYamlファイルを添付する。
AWSTemplateFormatVersion: 2010-09-09
Description: CodePipeline For ECS Fargate Blue/Green Deploy with PlaceHolder
# ------------------------------------------------------------#
# Parameters
# ------------------------------------------------------------#
Parameters:
AwsTokyoRegion:
Type: String
Default: ap-xxxxxxxxx-1
Service:
Type: String
Default: xxxx
Env:
Type: String
Default: dev
ECRName:
Type: String
Default: xxx-dev-xxxx-app-ecr
ContainerName:
Type: String
Default: xxx-dev-xxxx-app-container
CodeCommitRepositoryName:
Type: String
Default: xxx-dev-xxxx-codecommit0
CodeDeployAppName:
Type: String
Default: xxx-dev-xxxx-deploy-app1
CodeDeployGrpName:
Type: String
Default: xxx-dev-xxxx-deploy-grp1
NameTagPrefix:
Type: String
Default: test
Description: Prefix of Name tags.
ServiceName:
Type: String
Default: xxxx
Description: Prefix of Service tags.
# ------------------------------------------------------------#
# Parameters
# ------------------------------------------------------------#
Resources:
# ------------------------------------------------------------#
# IAM Roles
# ------------------------------------------------------------#
# CodeWatchEventを実行できるIAMRole
CloudwatchEventRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub cwe-xxx-${Service}-${Env}-phld-role
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
- PolicyName: CloudWatchEventsPipelineExecution
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Sub arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}
# CodeBuildに適用するIAMRole
CodeBuildServiceRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub codebuild-xxx-${Service}-${Env}-phld-service-role
Path: /
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: codebuild.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: SampleCodeBuildAccess
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Resource: '*'
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
- Effect: Allow
Resource: !Sub arn:aws:s3:::${ArtifactBucket}/*
Action:
- s3:PutObject
- s3:GetObject
- s3:GetObjectVersion
- s3:GetBucketAcl
- s3:GetBucketLocation
- Effect: Allow
Action:
- codebuild:CreateReportGroup
- codebuild:CreateReport
- codebuild:UpdateReport
- codebuild:BatchPutTestCases
- codebuild:BatchPutCodeCoverages
Resource: '*'
- Effect: Allow
Action:
- ecr:GetAuthorizationToken
- ecr:BatchCheckLayerAvailability
- ecr:GetDownloadUrlForLayer
- ecr:GetRepositoryPolicy
- ecr:DescribeRepositories
- ecr:ListImages
- ecr:DescribeImages
- ecr:BatchGetImage
- ecr:InitiateLayerUpload
- ecr:UploadLayerPart
- ecr:CompleteLayerUpload
- ecr:PutImage
Resource: '*'
# add policy to get parameter from SSM parameter store
- Effect: Allow
Action:
- ssm:DescribeParameters
- ssm:GetParameters
Resource: '*'
# CodePipelineに適用するIAMRole
CodePipelineServiceRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub codepipeline-xxx-${Service}-${Env}-phld-role
Path: /
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: codepipeline.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: SamplePipeline
PolicyDocument:
Version: 2012-10-17
Statement:
- Action:
- iam:PassRole
Resource: '*'
Effect: Allow
Condition:
StringEqualsIfExists:
iam:PassedToService:
- ecs-tasks.amazonaws.com
- Resource:
- !Sub arn:aws:s3:::${ArtifactBucket}/*
Effect: Allow
Action:
- s3:PutObject
- s3:GetObject
- s3:GetObjectVersion
- s3:GetBucketVersioning
- Action:
- codecommit:CancelUploadArchive
- codecommit:GetBranch
- codecommit:GetCommit
- codecommit:GetRepository
- codecommit:GetUploadArchiveStatus
- codecommit:UploadArchive
Resource: '*'
Effect: Allow
- Action:
- codedeploy:CreateDeployment
- codedeploy:GetApplication
- codedeploy:GetApplicationRevision
- codedeploy:GetDeployment
- codedeploy:GetDeploymentConfig
- codedeploy:RegisterApplicationRevision
- codedeploy:*
Resource: '*'
Effect: Allow
- Action:
- elasticbeanstalk:*
- ec2:*
- elasticloadbalancing:*
- autoscaling:*
- cloudwatch:*
- sns:*
- cloudformation:*
- rds:*
- sqs:*
- ecs:*
Resource: '*'
Effect: Allow
- Action:
- codebuild:BatchGetBuilds
- codebuild:StartBuild
- codebuild:BatchGetBuildBatches
- codebuild:StartBuildBatch
Resource: '*'
Effect: Allow
# CodeDeployに適用するIAMRole
CodeDeployRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: 'Allow'
Principal:
Service:
- 'codedeploy.amazonaws.com'
Action:
- 'sts:AssumeRole'
Path: '/'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AWSCodeDeployRoleForECS
- arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPowerUser
RoleName: !Sub codedeploy-xxx-${Service}-${Env}-phld-role
# S3Bucket
ArtifactBucket:
Type: AWS::S3::Bucket
Properties:
PublicAccessBlockConfiguration:
BlockPublicAcls: True
BlockPublicPolicy: True
IgnorePublicAcls: True
RestrictPublicBuckets: True
# CloudWatchEventの実行ルール
AmazonCloudWatchEventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.codecommit
detail-type:
- CodeCommit Repository State Change
resources:
- Fn::Join:
- ''
- - 'arn:aws:codecommit:'
- !Ref 'AWS::Region'
- ':'
- !Ref 'AWS::AccountId'
- ':'
- !Ref CodeCommitRepositoryName
detail:
event:
- referenceCreated
- referenceUpdated
referenceType:
- branch
referenceName:
- master
Targets:
- Arn: !Sub arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}
RoleArn: !GetAtt CloudwatchEventRole.Arn
Id: codepipeline-AppPipeline
# CodeBuild
CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
ServiceRole: !Ref CodeBuildServiceRole
Artifacts:
Type: CODEPIPELINE
Source:
Type: CODEPIPELINE
BuildSpec: |
version: 0.2
env:
parameter-store:
DOCKER_USER: dockerhub-user
DOCKER_TOKEN: dockerhub-token
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...
- $(aws ecr get-login --no-include-email)
- IMAGE_TAG=latest
- echo Logging in to Docker Hub...
- echo ${DOCKER_TOKEN} | docker login -u ${DOCKER_USER} --password-stdin
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker images...
- echo Writing imageDetail json...
- echo "{\"name\":\"${ContainerName}\",\"ImageURI\":\"${REPOSITORY_URI}:${IMAGE_TAG}\"}" > imageDetail.json
artifacts:
files: imageDetail.json
Environment:
PrivilegedMode: true
ComputeType: BUILD_GENERAL1_SMALL
Image: aws/codebuild/standard:4.0
Type: LINUX_CONTAINER
EnvironmentVariables:
- Name: AWS_DEFAULT_REGION
Value: !Ref AWS::Region
- Name: REPOSITORY_URI
Value: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${ECRName}
- Name: ContainerName
Value: !Ref ContainerName
- Name: DOCKER_BUILDKIT
Value: '1'
Name: !Sub ${Service}-${Env}-phld-project1
# Name: !Ref AWS::StackName
# Following was original from pre_build
# - IMAGE_TAG=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
# Following 2 lines was the last part in build
# - docker build -t $REPOSITORY_URI:$IMAGE_TAG .
# - docker tag $REPOSITORY_URI:$IMAGE_TAG $REPOSITORY_URI:$IMAGE_TAG
# Following 1 line was the last part in post_build
# - docker push $REPOSITORY_URI:$IMAGE_TAG
# Create Code deploy application
# CodeDeployApplication:
# Type: AWS::CodeDeploy::Application
# Properties:
# ApplicationName: !Ref CodeDeployAppName
# ComputePlatform: ECS
# CodeDeployDeploymentGroup:
# Type: AWS::CodeDeploy::DeploymentGroup
# Properties:
# ApplicationName: !Ref CodeDeployAppName
# DeploymentGroupName: !Ref CodeDeployGrpName
# DeploymentConfigName: CodeDeployDefault.ECSAllAtOnce
# DeploymentStyle:
# DeploymentType: BLUE_GREEN
# DeploymentOption: WITH_TRAFFIC_CONTROL
# ServiceRoleArn: !GetAtt CodeDeployRole.Arn
# LoadBalancerInfo:
# ElbInfoList:
# - Name: !ImportValue DevXxxxAppNLB
# TargetGroupInfoList:
# - Name: !ImportValue DevXxxxAppTG1
# - Name: !ImportValue DevXxxxAppTG2
# ------------------------------------------------------------#
# CodePipeline
# ------------------------------------------------------------#
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
RoleArn: !GetAtt CodePipelineServiceRole.Arn
Name: !Sub ${ServiceName}-phld-pipeline1
ArtifactStore:
Type: S3
Location: !Ref ArtifactBucket
Stages:
- Name: Source
Actions:
- Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: '1'
Provider: CodeCommit
Configuration:
RepositoryName: !Ref CodeCommitRepositoryName
PollForSourceChanges: false
BranchName: master
RunOrder: 1
OutputArtifacts:
- Name: App
- Name: Build
Actions:
- Name: Build
ActionTypeId:
Category: Build
Owner: AWS
Version: '1'
Provider: CodeBuild
Configuration:
ProjectName: !Ref CodeBuildProject
RunOrder: 1
InputArtifacts:
- Name: App
OutputArtifacts:
- Name: BuildOutput
# - Name: Approval
# Actions:
# - Name: Manual_Approval
# ActionTypeId:
# Category: Approval
# Owner: AWS
# Version: '1'
# Provider: Manual
# Configuration:
# CustomData: !Sub '${ServiceName} will be updated. Do you want to deploy it?'
# NotificationArn: arn:aws:sns:ap-xx-xxxx-2:xxxxxxxx:hogehoge
- Name: Deploy
Actions:
- Name: Deploy
ActionTypeId:
Category: Deploy
Owner: AWS
Version: '1'
Provider: CodeDeployToECS
Configuration:
AppSpecTemplateArtifact: App
AppSpecTemplatePath: appspec.yml
TaskDefinitionTemplateArtifact: App
TaskDefinitionTemplatePath: taskdef.json
ApplicationName: !Ref CodeDeployAppName
DeploymentGroupName: !Ref CodeDeployGrpName
Image1ArtifactName: BuildOutput
Image1ContainerName: IMAGE1_NAME
RunOrder: 1
InputArtifacts:
- Name: App
- Name: BuildOutput
Region: !Ref AWS::Region
# ------------------------------------------------------------#
# Outputs
# ------------------------------------------------------------#
Outputs:
PipelineDev:
Description: Dev Xxxx Pipeline
Value: !Ref Pipeline
Export:
Name: DevPhldAppPipeline
#5. まとめ
CodePipelineのCodeCommitによるCICDのBlue/Greenを実装するには、現状のDeployステージに渡されたアーティファクトの3MBサイズ制限によるデプロイは失敗したケース少なくはないと思う。特に、Javaなど商用サービスのアプリは開発されたJarファイルなどを入れると、この制限は遥かに超える。
本文のS3 PipelineとCodeCommit Pipeline対処法はCICDを実装する際に役に立てば幸いである。
――――――――
S3 Pipelineで実装するCloudFormationの例。
##############################################################################
# Prerequisites:
# - CodeDeploy deploy exists. Update ApplicationName and BetaFleet as needed
##############################################################################
Parameters:
SourceObjectKey:
Description: 'S3 source artifact'
Type: String
Default: SampleApp_Linux.zip
ApplicationName:
Description: 'CodeDeploy application name'
Type: String
Default: DemoApplication
BetaFleet:
Description: 'Fleet configured in CodeDeploy'
Type: String
Default: DemoFleet
Resources:
SourceBucket:
Type: AWS::S3::Bucket
Properties:
VersioningConfiguration:
Status: Enabled
CodePipelineArtifactStoreBucket:
Type: AWS::S3::Bucket
CodePipelineArtifactStoreBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref CodePipelineArtifactStoreBucket
PolicyDocument:
Version: 2012-10-17
Statement:
-
Sid: DenyUnEncryptedObjectUploads
Effect: Deny
Principal: '*'
Action: s3:PutObject
Resource: !Join [ '', [ !GetAtt CodePipelineArtifactStoreBucket.Arn, '/*' ] ]
Condition:
StringNotEquals:
s3:x-amz-server-side-encryption: aws:kms
-
Sid: DenyInsecureConnections
Effect: Deny
Principal: '*'
Action: s3:*
Resource: !Join [ '', [ !GetAtt CodePipelineArtifactStoreBucket.Arn, '/*' ] ]
Condition:
Bool:
aws:SecureTransport: false
CodePipelineServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: AWS-CodePipeline-Service-3
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action:
- codecommit:CancelUploadArchive
- codecommit:GetBranch
- codecommit:GetCommit
- codecommit:GetUploadArchiveStatus
- codecommit:UploadArchive
Resource: '*'
-
Effect: Allow
Action:
- codedeploy:CreateDeployment
- codedeploy:GetApplicationRevision
- codedeploy:GetDeployment
- codedeploy:GetDeploymentConfig
- codedeploy:RegisterApplicationRevision
Resource: '*'
-
Effect: Allow
Action:
- codebuild:BatchGetBuilds
- codebuild:StartBuild
Resource: '*'
-
Effect: Allow
Action:
- devicefarm:ListProjects
- devicefarm:ListDevicePools
- devicefarm:GetRun
- devicefarm:GetUpload
- devicefarm:CreateUpload
- devicefarm:ScheduleRun
Resource: '*'
-
Effect: Allow
Action:
- lambda:InvokeFunction
- lambda:ListFunctions
Resource: '*'
-
Effect: Allow
Action:
- iam:PassRole
Resource: '*'
-
Effect: Allow
Action:
- elasticbeanstalk:*
- ec2:*
- elasticloadbalancing:*
- autoscaling:*
- cloudwatch:*
- s3:*
- sns:*
- cloudformation:*
- rds:*
- sqs:*
- ecs:*
Resource: '*'
AppPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
Name: s3-events-pipeline
RoleArn:
!GetAtt CodePipelineServiceRole.Arn
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: S3
OutputArtifacts:
- Name: SourceOutput
Configuration:
S3Bucket: !Ref SourceBucket
S3ObjectKey: !Ref SourceObjectKey
PollForSourceChanges: false
RunOrder: 1
-
Name: Beta
Actions:
-
Name: BetaAction
InputArtifacts:
- Name: SourceOutput
ActionTypeId:
Category: Deploy
Owner: AWS
Version: 1
Provider: CodeDeploy
Configuration:
ApplicationName: !Ref ApplicationName
DeploymentGroupName: !Ref BetaFleet
RunOrder: 1
ArtifactStore:
Type: S3
Location: !Ref CodePipelineArtifactStoreBucket
AmazonCloudWatchEventRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: cwe-pipeline-execution
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
AmazonCloudWatchEventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.s3
detail-type:
- 'AWS API Call via CloudTrail'
detail:
eventSource:
- s3.amazonaws.com
eventName:
- PutObject
- CompleteMultipartUpload
resources:
ARN:
- !Join [ '', [ !GetAtt SourceBucket.Arn, '/', !Ref SourceObjectKey ] ]
Targets:
-
Arn:
!Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
RoleArn: !GetAtt AmazonCloudWatchEventRole.Arn
Id: codepipeline-AppPipeline
Outputs:
SourceBucketARN:
Description: "S3 bucket ARN that Cloudtrail will use"
Value: !GetAtt SourceBucket.Arn
Export:
Name: SourceBucketARN