APIGateway Lambda + Codebuild Codepipeline CI/CD

環境

• Go 1.10
• github Enterprise(以下GHEと記載)

AWS側
- CodeBuild
- CodePipline
- CloudFormation
- AWS Lambda
- APIGateway

やりたいこと

• GHEのPushトリガーによる自動テスト
• 特定ブランチ（staging/master）へPushした場合に自動デプロイ
• 本番デプロイは完全自動化せず承認フロー(Approve)をいれる
• 本番デプロイ時、テスト通過後に承認フローへ行くようにする
• 各インフラのコード化
• 問題発生時にロールバック(cloudformation rollback / Lambda Version switch)ができる
• リリースリソースのバージョン管理(Lambda prod:version/stage:version)
• Blue/Green デプロイ

CI/CD Flow イメージ

Feature Branch を GHE に push した際の動作：

1. GHEへPush
2. Webhook で CodeBuild が起動
3. Codebuild より UnitTest を実行
4. UnitTest の結果を GHE に返す

特定（staging/production）branch へ merge した際の動作：

1. GHEへPush
2. Webhook で CodeBuild が起動
3. Codebuild より UnitTest を実行
4. S3に、ビルドファイルと CloudFormation 用の template ファイルを配置
5. S3トリガーから CodePipeline を起動
6. CloudFormation から Lambda/API Gateway の Stage ステージにデプロイ
7. 承認しPipelineを継続 (productionのみ)
8. lambda:$latest からの新しいバージョンを切り出し、Prod エイリアスに紐付ける (productionのみ)

staging

production：

全体Flow：

lambda

-- staging
   # versionやalias情報：
   -- lambda:$latest 

-- production
   # versionやalias情報：
   -- lambda:$latest
   -- lambda:Prod (alias: Prod)

ApiGateway

-- staging
  -- stage
     -- lambda：$latest を指している
  -- prod
     -- lambda：$latest を指している

-- production
  -- stage
     -- lambda：$latest を指している
  -- prod
     -- lambda:Prod を指している

• lambda：$latest: lambdaの最新version
• lambda:Prod: lambdaのProd alias

CI/CDディレクトリの構成

--deploy
  ## codebuild 実行yaml
  --buildspec.yml

  ## codebuild 実行yaml
  --codepipeline.yaml

  ## apigateway lambda cloudformation yaml
  --template.yaml

  ## lambda version cloudformation yaml
  --update_version.yaml

template.yaml

AWSTemplateFormatVersion: "2010-09-09"

Globals:
  Function:
    Timeout: 30

Mappings:
  EnvMap:
    staging:
      APIName: stage-xxx-api
      Alias: /invocations
      Permission: stage-xxx-api
    production:
      APIName: xxx-api
      Alias: :Prod/invocations
      Permission: xxx-api:Prod

Parameters:
  # ...
  Goenv:
    Type: String
    Default: staging
    AllowedValues:
      - staging
      - production

Resources:
  xxxAPIFunction:
    DeletionPolicy: Retain
    Properties:
      # filename template.zip change by codebuild sed command
      CodeUri: s3://stage-xxx-deploy/api/api_template.zip
      # ...

  xxxAPIApi:
    DeletionPolicy: Retain
    Properties:
      DefinitionBody:
        info:
          title: !FindInMap [EnvMap, !Ref Goenv, APIName]
        paths:
          /authorize:
            get:
              x-amazon-apigateway-integration:
                httpMethod: POST
                type: aws_proxy
                uri: !Join
                  - ""
                  - - !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${xxxAPIFunction.Arn}
                    - !FindInMap [EnvMap, !Ref Goenv, Alias]
      # ...
    Type: AWS::Serverless::Api

  ATSAPILambdaProdPermission:
    DeletionPolicy: Retain
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !Join
        - ":"
        - - !Sub arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function
          - !FindInMap [EnvMap, !Ref Goenv, Permission]
      Principal: apigateway.amazonaws.com
    Type: AWS::Lambda::Permission

Transform: AWS::Serverless-2016-10-31

buildspec.yml

version: 0.2

env:
  variables:
    # ...
    StageCodepipeline: arn:aws:cloudformation:ap-northeast-1:xxxxxxxxx-stage
    ProdCodepipeline: arn:aws:cloudformation:ap-northeast-1:xxxxxxxxx-prod
    S3: s3://stage-xxxxx/api/

phases:
  install:
    runtime-versions:
      golang: 1.11
    commands:
      # ...


  pre_build:
    commands:
      # ...
      - go test ./...

  build:
    commands:
      - |
        if [ "X${BRANCH_NAME}" = "Xstaging" ]; then
          # ...
          # change codepipeline trigger filename 
          aws cloudformation update-stack --stack-name ${StageCodepipeline} --use-previous-template --parameters ParameterKey=S3BucketKey,ParameterValue=api/${CODEBUILD_START_TIME}.zip
          # ...
        fi

      - |
        if [ "X${BRANCH_NAME}" = "Xmaster" ]; then 
          # ...
          # change codepipeline trigger filename 例： 2019091111221.zipが探知できたら起動
          aws cloudformation update-stack --stack-name ${ProdCodepipeline} --use-previous-template --parameters ParameterKey=S3BucketKey,ParameterValue=api/${CODEBUILD_START_TIME}.zip ParameterKey=ENV,ParameterValue=production
          # ...
        fi

  post_build:
    commands:
      - |
        if [ "X${BRANCH_NAME}" = "Xmaster" -o "X${BRANCH_NAME}" = "Xstaging" ]; then 
          [ X${BRANCH_NAME} = "Xmaster" ] && S3=$(echo ${S3} | sed -e "s/stage-//")
          # ...

          # artifacts filename 変更: sed  例： 2019091111221.zipに
          sed -i -e 's/api_template/'${CODEBUILD_START_TIME}'/g' ./deploy/template.yaml
          cp ./deploy/template.yaml ./build/bin/template.yaml
          cp ./deploy/version_update.yaml ./build/bin/version_update.yaml

　　　　　 # artifacts zip 作成
          cd ./build/bin/ && zip -r ${CODEBUILD_START_TIME}.zip api template.yaml version_update.yaml

          # 本番のs3に投げる
          aws s3 cp ${CODEBUILD_START_TIME}.zip $S3 --acl bucket-owner-full-control
          # ...
        fi

version_update.yaml

Mappings:
  EnvMap:
    staging:
      FunctionName: stage-xxx-api
    production:
      FunctionName: xxx-api

Parameters:
  ENV:
    Type: String
    AllowedValues:
      - staging
      - production

Resources:
  V1:
    Type: AWS::Lambda::Version
    Properties:
      FunctionName: !FindInMap [EnvMap, !Ref ENV, FunctionName]

  CreateOrUpdateAlias:
    Type: AWS::Lambda::Alias
    Properties:
      FunctionName: !FindInMap [EnvMap, !Ref ENV, FunctionName]
      FunctionVersion:
        Fn::GetAtt: V1.Version
      Name: Prod

codepipeline.yaml

Mappings:
  EnvMap:
    staging:
      StackName: stage-xxx-api
      S3Bucket: stage-xxx-deploy
      PipelineName: stage-pipeline-xxx-api
    production:
      StackName: xxx-api
      S3Bucket: xxx-deploy
      PipelineName: pipeline-xxx-api

Conditions:
  ConditionName: !Equals [!Ref ENV, production]

Parameters:
  ENV:
    Type: String
    Default: staging
    AllowedValues:
      - staging
      - production
  S3BucketKey:
    Type: String
    Default: api/artifacts.zip

Resources:
  ATSLambdaAPIPipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      ArtifactStore:
        Type: S3
        Location: !FindInMap [EnvMap, !Ref ENV, S3Bucket]
      RoleArn: !Sub arn:aws:iam::${AWS::AccountId}:role/service-role/PipelineRole
      Name: !FindInMap [EnvMap, !Ref ENV, PipelineName]
      Stages:
        - Name: Source
          Actions:
            - Name: Trigger
              ActionTypeId:
                Category: Source
                Owner: AWS
                Version: 1
                Provider: S3
              Configuration:
                S3Bucket: !FindInMap [EnvMap, !Ref ENV, S3Bucket]
                S3ObjectKey: !Ref S3BucketKey
              OutputArtifacts:
                - Name: SourceArtifact
              RunOrder: "1"
        - Name: Release
          Actions:
            - Name: Deploy
              ActionTypeId:
                Category: Deploy
                Owner: AWS
                Provider: CloudFormation
                Version: "1"
              InputArtifacts:
                - Name: SourceArtifact
              Configuration:
                ActionMode: CREATE_UPDATE
                RoleArn: !Sub arn:aws:iam::${AWS::AccountId}:role/CloudFormationRole
                StackName: !FindInMap [EnvMap, !Ref ENV, StackName]
                Capabilities: CAPABILITY_IAM,CAPABILITY_AUTO_EXPAND
                TemplatePath: SourceArtifact::template.yaml
                ParameterOverrides: !Sub |
                  {
                    "Goenv": "${ENV}"
                  }
              RunOrder: 1
            - !If
              - ConditionName
              - Name: Approval
                ActionTypeId:
                  Category: Approval
                  Owner: AWS
                  Provider: Manual
                  Version: "1"
                Configuration:
                  NotificationArn: !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:xxx_sns_topic
                  CustomData: !Sub "A new change set was created for the xxx-api stack. Do you want to implement the changes?"
                RunOrder: 2
              - !Ref AWS::NoValue
        - Name: VersionUP
          Actions:
            - Name: ProdLambdaVersionUP
              ActionTypeId:
                Category: Deploy
                Owner: AWS
                Provider: CloudFormation
                Version: "1"
              InputArtifacts:
                - Name: SourceArtifact
              Configuration:
                ActionMode: CREATE_UPDATE
                RoleArn: !Sub arn:aws:iam::${AWS::AccountId}:role/CloudFormationRole
                StackName: xxx-api-version-up
                Capabilities: CAPABILITY_IAM,CAPABILITY_AUTO_EXPAND
                TemplatePath: !Sub SourceArtifact::version_update.yaml
                ParameterOverrides: !Sub |
                  {
                    "ENV": "${ENV}"
                  }
              RunOrder: 1

まとめ

• codebuildの中では、違うaws count に　
  • update-stack をcliで掛けてるので、違うアカウントのRoleや信頼関係の設定追加が必要。
  • 違う S3 に artifacts 投げてるので S3の信頼関係の設定追加が必要。
• version_update.yaml毎回新しいversion追加が必要となります。（以下の例）

version_update.yaml

# ...
# ...
Resources:
  V1:
    Type: AWS::Lambda::Version
    Properties:
      FunctionName: !FindInMap [EnvMap, !Ref ENV, FunctionName]
  # Add V2
  V2:
    Type: AWS::Lambda::Version
    Properties:
      FunctionName: !FindInMap [EnvMap, !Ref ENV, FunctionName]

  CreateOrUpdateAlias:
    Type: AWS::Lambda::Alias
    Properties:
      FunctionName: !FindInMap [EnvMap, !Ref ENV, FunctionName]
      FunctionVersion:
        # V2
        Fn::GetAtt: V2.Version
      Name: Prod