【AWS-CDK】CodePipelineでGithubにpush⇨ビルド⇨ECSにデプロイを自動化する

はじめに

前回記事「お名前.comで購入したドメインを使ってALBでECSにホストベースルーティングする」を拡張してデプロイパイプラインまで作ります。

全体図

前提

-　お名前.comでドメイン(example.com)を購入済み
-　Route53でexample.comのホストゾーンを作成し、そのNSレコードをお名前.comで設定
-　ECRのプライベートリポジトリを作成後、アプリのイメージをpush済
-　(今回追加)GithubのトークンなどをSecretManagerに保存済

実装

ディレクトリ構成

前回の内容に下記を追加

• buildspec.ts
• codepipeline.ts

.
├── bin
│   └── app.ts
├── cdk.json
├── jest.config.js
├── lib
│   ├── app-stack.ts
│   ├── getENV.ts
│   └── resources
│       ├── buildspec.ts
│       ├── codePipeline.ts
│       ├── fargateService.ts
│       ├── fargateServiceWithAlb.ts
│       └── vpc.ts
├── package-lock.json
├── package.json
├── test
│   └── app.test.ts
├── tsconfig.json
└── yarn.lock

環境変数の設定

前回同様

lib/getENV.ts

const ENV_NAMES = ['stg', 'prd'] as const;
type EnvName = typeof ENV_NAMES[number];
type EnvValues = {
  envName: string;
  domainName: string;
  subDomainName: string;
  HostedZoneId: string;
};

export function getEnv(): EnvValues {
  const envName = (process.env.ENV_NAME) as EnvName;
  switch (envName) {
    case 'stg':
      return {
        envName,
        domainName: 'example.com', // お名前.comでRoute53のNSレコードを事前設定済
        subDomainName: 'www.example.com',
        HostedZoneId: 'Zxxxxxxxxxxxx' // Route53でexample.comのホストゾーンを事前設定済
      };
    case 'prd':
      return {
        envName,
        domainName: 'xxxxx.com',
        subDomainName: 'xxxx.xxxxx.com',
        HostedZoneId: 'Zxxxxxxxxxxxx'
      };
    default:
      const _: never = envName;
      throw new Error(
        `Invalid environment variable ENV_NAME has given. ${envName}`
      );
  }
}

package.json

{
  ...
  "scripts": {
    "build": "tsc",
    "watch": "tsc -w",
    "test": "jest",
    "cdk": "cdk",
    "synth:stg": "ENV_NAME=stg npx cdk synth",
    "synth:prd": "ENV_NAME=prd npx cdk synth",
    "diff:stg": "ENV_NAME=stg npx cdk diff",
    "diff:prd": "ENV_NAME=prd npx cdk diff",
    "deploy:stg": "ENV_NAME=stg npx cdk deploy",
    "deploy:prd": "ENV_NAME=prd npx cdk deploy"
  },
  ...
}

App

前回同様

bin/app.ts

#!/usr/bin/env node
import 'source-map-support/register';
import { App } from 'aws-cdk-lib';
import { AppStack } from '../lib/app-stack';

const app = new App();
new AppStack(app, 'AppStack', {});

Stack

前回にCodePipelineを追加

lib/app-stack.ts

import { Stack, StackProps } from 'aws-cdk-lib';
import { Cluster } from 'aws-cdk-lib/aws-ecs';
import { Construct } from 'constructs';
import {
  Certificate,
  CertificateValidation
} from 'aws-cdk-lib/aws-certificatemanager';
import { HostedZone } from 'aws-cdk-lib/aws-route53';
import { Bucket } from 'aws-cdk-lib/aws-s3';
import { RemovalPolicy } from 'aws-cdk-lib';
import { SecretValue } from 'aws-cdk-lib';
import { BuildEnvironmentVariableType } from 'aws-cdk-lib/aws-codebuild';
import { Role, ServicePrincipal } from 'aws-cdk-lib/aws-iam';

import { CustomVpc } from './resources/vpc';
import { FargateServiceWithSslAlb } from './resources/fargateServiceWithAlb';
import { HostBasedRoutingFargateService } from './resources/fargateService';
import { CustomCodePipeline } from './resources/codepipeline';
import { getEnv } from './getENV';
import { buildSpec } from './resources/buildspec';

export class AppStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);
    
    // VPC
    const VpcCdk = new CustomVpc(this, `${getEnv().envName}-cdk-VPC`);

    // ECS Cluster
    const EcsClusterCdk = new Cluster(
      this,
      `${getEnv().envName}-cdk-EcsCluster`,
      {
        clusterName: `${getEnv().envName}-cdk-EcsCluster`,
        vpc: VpcCdk.vpc
      }
    );

    // 作成済のホストゾーン（example.com）の読み込み
    const HostedZoneCdk = HostedZone.fromHostedZoneAttributes(
      this,
      `${getEnv().envName}-cdk-HostedZone`,
      { hostedZoneId: getEnv().HostedZoneId, zoneName: getEnv().domainName }
    );

    // SSL証明書
    const CertificateCdk = new Certificate(
      this,
      `${getEnv().envName}-cdk-Cert`,
      {
        domainName: getEnv().domainName,
        subjectAlternativeNames: [getEnv().subDomainName],
        validation: CertificateValidation.fromDns(HostedZoneCdk)
      }
    );
 
    // ALB + Service1
    const serviceName1 = 'sample-service-1';
    const ServiceWithAlb = new FargateServiceWithSslAlb(
      this,
      `${getEnv().envName}-cdk-Service1`,
      {
        loadBalancerName: `${getEnv().envName}-cdk-ALB`,
        domainName: getEnv().domainName,
        domainZone: HostedZoneCdk,
        certificate: CertificateCdk,
        serviceName: serviceName1,
        cluster: EcsClusterCdk,
        taskSubnets: { subnets: VpcCdk.vpc.privateSubnets },
        containerPort: 3000,　// Fargateにバインドするコンテナのポート番号（アプリが動いているポート番号）
        ecrRepoName: serviceName1,　// ECRリポジトリ名
        imageTag: 'xxxxx' // ECRイメージのタグ
        awsAccountNumber: this.account
      }
    );

    // ECR repo of Serviec1
    const ecrRepo1 = ServiceWithAlb.ecrRepo;

    // Service2
    const serviceName2 = 'sample-service-1';
    const AnotherService = new HostBasedRoutingFargateService(
      this,
      `${getEnv().envName}-cdk-Service2`,
      {
        domainName: getEnv().subDomainName,
        domainZone: HostedZoneCdk,
        serviceName: serviceName2,
        cluster: EcsClusterCdk,
        vpc: VpcCdk.vpc,
        vpcSubnets: { subnets: VpcCdk.vpc.privateSubnets },
        alb: ServiceWithAlb.loadBalancer,
        listener: ServiceWithAlb.listener,
        containerPort: 3000,
        ecrRepoName: serviceName2,
        imageTag: 'yyyyy',
        listenerPriority: 2,
        awsAccountNumber: this.account
      }
    );

    // ECR repo of Serviec2
    const ecrRepo2 = AnotherService.ecrRepo;

    // Artifactバケットを追加
    const artifactBucket = new Bucket(
      this,
      `${getEnv().envName}-artifact-bucket`,
      {
        bucketName: `${getEnv().envName}-artifact-bucket`,
        autoDeleteObjects: true,
        removalPolicy: RemovalPolicy.DESTROY
      }
    );

    // Secret ManagerからGithubのpersonal access tokenを読み込む
    // 本来ならCodeStarConnectionsを使うべきだが、CDKではまだ対応していない模様。

    const githubToken = SecretValue.secretsManager(
      'arn:aws:secretsmanager:ap-northeast-1:12345678910:secret:your-secrets-xxxxxx',
      { jsonField: 'GITHUB_PAT' }
    );

    // CodeBuildがVPC内にENIを設置するためにroleが必要。今回は面倒くさがってアドミン権限を与える。
    const codeBuildRole = new Role(
      this,
      `${getEnv().envName}-codebuild-role`,
      {
        assumedBy: new ServicePrincipal('codebuild.amazonaws.com'),
        managedPolicies: [
          {
            managedPolicyArn: 'arn:aws:iam::aws:policy/AdministratorAccess'
          }
        ]
      }
    );

    // CodePipelineの設定。とりあえずService1のみ。Service2の設定も同様。
    const artifactFileName = 'my-custom-artifact-file.json';
    const CodePipeline = new CustomCodePipeline(
      this,
      `${getEnv().envName}-codepipeline`,
      {
        gitHubAccountName: 'your-account',
        githubBranchName: 'develop',
        githubRepoName: 'your-repo',
        githubToken,
        serviceName: serviceName1,
        artifactBucket,
        vpc: VpcCdk.vpc,
        ecrRepo: ecrRepo1,
        buildSpec,
        codeBuildRole,
        artifactFileName,
        ecsService: ServiceWithAlb.service,
        codebuildEnvironmentVariables: {
          AWS_DEFAULT_REGION: { value: this.region },
          AWS_ACCOUNT_ID: { value: this.account },
          CONTAINER_NAME: { value: ecrRepo1.repositoryName },
          REPOSITORY_URI: { value: ecrRepo1.repositoryUri },
          ARTIFACT_FILE_NAME: { value: artifactFileName },
          SOME_SECRET: {
            type: BuildEnvironmentVariableType.SECRETS_MANAGER,
            value:
              'arn:aws:secretsmanager:ap-northeast-1:12345678910:secret:your-secrets-xxxxxx:SOME_SECRET::'
          }
        }
      }
    );

  }
}

Construct

CodePipeline

今回追加。
CodeBuildはVPCを指定してプライベートサブネットの中で実行する。
VPCを指定しないとDocker Hub の Rate Limitに引っかかります。

lib/resources/codePipeline.ts

import { Construct } from 'constructs';
import { FargateService } from 'aws-cdk-lib/aws-ecs';
import { Pipeline, Artifact } from 'aws-cdk-lib/aws-codepipeline';
import {
  GitHubSourceAction,
  CodeBuildAction,
  EcsDeployAction
} from 'aws-cdk-lib/aws-codepipeline-actions';
import {
  PipelineProject,
  ComputeType,
  LinuxBuildImage,
  BuildEnvironmentVariable,
  BuildSpec
} from 'aws-cdk-lib/aws-codebuild';
import { SecretValue } from 'aws-cdk-lib';
import { IBucket } from 'aws-cdk-lib/aws-s3';
import { IVpc } from 'aws-cdk-lib/aws-ec2';
import { IRepository } from 'aws-cdk-lib/aws-ecr';
import { IRole } from 'aws-cdk-lib/aws-iam';

import { getEnv } from '../getENV';

export interface CodePipelineProps {
  gitHubAccountName: string;
  githubRepoName: string;
  githubBranchName: string;
  githubToken: SecretValue;
  serviceName: string;
  ecsService: FargateService;
  artifactBucket: IBucket;
  buildSpec: {
    [key: string]: any;
  };
  ecrRepo: IRepository;
  artifactFileName: string;
  codeBuildRole?: IRole;
  vpc?: IVpc;
  codebuildEnvironmentVariables?: { [name: string]: BuildEnvironmentVariable };
}

export class CustomCodePipeline extends Construct {
  constructor(scope: Construct, id: string, props: CodePipelineProps) {
    super(scope, id);

    const pipelineName = `${getEnv().envName}-pl-${props.serviceName}`;
    const pjtName = `${getEnv().envName}-pjt-${props.serviceName}`;
    const pipeline = new Pipeline(this, pipelineName, {
      crossAccountKeys: false,
      pipelineName,
      artifactBucket: props.artifactBucket
    });

    const project = new PipelineProject(this, pjtName, {
      environment: {
        buildImage: LinuxBuildImage.STANDARD_5_0,
        computeType: ComputeType.SMALL,
        privileged: true,
        environmentVariables: props.codebuildEnvironmentVariables
      },
      buildSpec: BuildSpec.fromObject(props.buildSpec),
      vpc: props.vpc,
      role: props.codeBuildRole
    });
    props.ecrRepo.grantPullPush(project);

    // connect to Github
    const sourceStage = pipeline.addStage({
      stageName: 'Source'
    });
    const sourceOutput = new Artifact('sourceOutput');
    const sourceAction = new GitHubSourceAction({
      actionName: 'GitHub_Source',
      owner: props.gitHubAccountName,
      repo: props.githubRepoName,
      oauthToken: props.githubToken,
      output: sourceOutput,
      branch: props.githubBranchName
    });
    sourceStage.addAction(sourceAction);

    // build docker image and push to ECR
    const buildStage = pipeline.addStage({
      stageName: 'Build'
    });
    const buildOutput = new Artifact('buildOutput');
    const buildAction = new CodeBuildAction({
      actionName: 'Build_And_Push_To_ECR',
      project,
      input: sourceOutput,
      outputs: [buildOutput]
    });
    buildStage.addAction(buildAction);

    // deploy to ECS
    const deployStage = pipeline.addStage({
      stageName: 'Deploy'
    });
    const deployAction = new EcsDeployAction({
      actionName: 'Deploy_To_ECS',
      imageFile: buildOutput.atPath(props.artifactFileName),
      service: props.ecsService
    });
    deployStage.addAction(deployAction);
  }
}

ビルド設定はこちら↓に記載する。

lib/resources/buildspec.ts

export const buildSpec = {
  version: '0.2',
  phases: {
    pre_build: {
      commands: [
        'IMAGE_URI="${REPOSITORY_URI}:$(echo ${CODEBUILD_RESOLVED_SOURCE_VERSION} | head -c 7)"',
        'aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com'
      ]
    },
    build: {
      commands: [
        'echo Build started on $(date)',
        'docker build --tag ${IMAGE_URI} ./ --build-arg SOME_SECRET=${SOME_SECRET}',
        'docker push ${IMAGE_URI}'
      ]
    },
    post_build: {
      commands: [
        'echo Build completed on $(date)',
        'printf \'[{"name":"%s","imageUri":"%s"}]\' "${CONTAINER_NAME}" "$IMAGE_URI" > ${ARTIFACT_FILE_NAME}'
      ]
    }
  },
  artifacts: {
    files: '${ARTIFACT_FILE_NAME}'
  }
};

VPC

前回同様

lib/resources/vpc.ts

import { Construct } from 'constructs';
import { Vpc, SubnetType } from 'aws-cdk-lib/aws-ec2';
import { getEnv } from '../getENV';

export interface CustomVpcProps {}

export class CustomVpc extends Construct {
  public readonly vpc: Vpc;

  constructor(scope: Construct, id: string, _props: CustomVpcProps = {}) {
    super(scope, id);

    const vpc = new Vpc(this, `${getEnv().envName}-vpc`, {
      cidr: '10.0.0.0/16',
      natGateways: getEnv().envName === 'prd' ? 2 : 1,
      maxAzs: 2,
      subnetConfiguration: [
        {
          name: 'private-subnet',
          subnetType: SubnetType.PRIVATE_WITH_NAT,
          cidrMask: 24
        },
        {
          name: 'public-subnet',
          subnetType: SubnetType.PUBLIC,
          cidrMask: 24
        },
        // DB配置用のisolated-subnet(今回は使わない)
        // { 
        //   name: 'isolated-subnet',
        //   subnetType: SubnetType.PRIVATE_ISOLATED,
        //   cidrMask: 28
        // }
      ]
    });
 
    this.vpc = vpc;
  }
}

ALB(HTTPS) + Service1

前回同様

lib/resources/fargateServiceWithAlb.ts

import { Construct } from 'constructs';
import { ApplicationLoadBalancedFargateService } from 'aws-cdk-lib/aws-ecs-patterns';
import { ContainerImage, ICluster, FargateService } from 'aws-cdk-lib/aws-ecs';
import { Repository, IRepository } from 'aws-cdk-lib/aws-ecr';
import { ICertificate } from 'aws-cdk-lib/aws-certificatemanager';
import { IHostedZone } from 'aws-cdk-lib/aws-route53';
import {
  ApplicationLoadBalancer,
  ApplicationListener
} from 'aws-cdk-lib/aws-elasticloadbalancingv2';
import { SubnetSelection } from 'aws-cdk-lib/aws-ec2';

import { getEnv } from '../getENV';

export interface FargateServiceWithSslAlbProps {
  loadBalancerName: string;
  domainName: string;
  domainZone: IHostedZone;
  serviceName: string;
  cluster: ICluster;
  certificate: ICertificate;
  taskSubnets: SubnetSelection;
  containerPort: number;
  ecrRepoName: string;
  imageTag: string;
  autoScaleMaxCount?: number;
  awsAccountNumber: string;
}

export class FargateServiceWithSslAlb extends Construct {
  public readonly loadBalancer: ApplicationLoadBalancer;
  public readonly listener: ApplicationListener;
  public readonly ecrRepo: IRepository;
  public readonly service: FargateService;

  constructor(
    scope: Construct,
    id: string,
    props: FargateServiceWithSslAlbProps
  ) {
    super(scope, id);

    const ImageArn = Repository.arnForLocalRepository(
      props.ecrRepoName,
      this,
      props.awsAccountNumber
    );
    const Image = Repository.fromRepositoryAttributes(
      this,
      `${getEnv().envName}-image-${props.ecrRepoName}-${props.imageTag}`,
      { repositoryArn: ImageArn, repositoryName: props.ecrRepoName }
    );

    const loadBalancedFargateService =
      new ApplicationLoadBalancedFargateService(
        this,
        `${getEnv().envName}-fargate-alb-${props.serviceName}`,
        {
          loadBalancerName: props.loadBalancerName,
          domainName: props.domainName,
          domainZone: props.domainZone,
          serviceName: props.serviceName,
          certificate: props.certificate,
          redirectHTTP: true,
          cluster: props.cluster,
          taskSubnets: props.taskSubnets,
          taskImageOptions: {
            containerName: props.serviceName,
            image: ContainerImage.fromEcrRepository(ecrRepo, props.imageTag),
            containerPort: props.containerPort
          }
        }
      );

    // オートスケール設定
    const scalableTarget =
      loadBalancedFargateService.service.autoScaleTaskCount({
        minCapacity: 1,
        maxCapacity: props.autoScaleMaxCount ? props.autoScaleMaxCount : 10
      });

    scalableTarget.scaleOnCpuUtilization('CpuScaling', {
      targetUtilizationPercent: 50
    });

    scalableTarget.scaleOnMemoryUtilization('MemoryScaling', {
      targetUtilizationPercent: 50
    });

    this.loadBalancer = loadBalancedFargateService.loadBalancer;
    this.listener = loadBalancedFargateService.listener;
    this.service = loadBalancedFargateService.service;
    this.ecrRepo = ecrRepo;
  }
}

Service2

前回同様

lib/resources/fargateService.ts

import { Construct } from 'constructs';
import { FargateService, FargateTaskDefinition } from 'aws-cdk-lib/aws-ecs';
import {
  ApplicationLoadBalancer,
  ApplicationTargetGroup,
  ApplicationProtocol,
  ApplicationListener,
  ListenerCondition
} from 'aws-cdk-lib/aws-elasticloadbalancingv2';
import { ContainerImage, ICluster } from 'aws-cdk-lib/aws-ecs';
import { Repository, IRepository } from 'aws-cdk-lib/aws-ecr';
import { Port, Vpc } from 'aws-cdk-lib/aws-ec2';
import { SubnetSelection } from 'aws-cdk-lib/aws-ec2';
import {
  IHostedZone,
  RecordSet,
  RecordType,
  RecordTarget
} from 'aws-cdk-lib/aws-route53';

import { getEnv } from '../getENV';

export interface FargateServiceProps {
  domainName: string;
  domainZone: IHostedZone;
  serviceName: string;
  cluster: ICluster;
  vpc: Vpc;
  vpcSubnets: SubnetSelection;
  alb: ApplicationLoadBalancer;
  listener: ApplicationListener;
  containerPort: number;
  ecrRepoName: string;
  imageTag: string;
  autoScaleMaxCount?: number;
  listenerPriority: number;
  awsAccountNumber: string;
}

export class HostBasedRoutingFargateService extends Construct {
  public readonly service: FargateService;
  public readonly ecrRepo: IRepository;

  constructor(scope: Construct, id: string, props: FargateServiceProps) {
    super(scope, id);

    const ecrRepoArn = Repository.arnForLocalRepository(
      props.ecrRepoName,
      this,
      props.awsAccountNumber
    );
    const ecrRepo = Repository.fromRepositoryAttributes(
      this,
      `${getEnv().envName}-image-${props.ecrRepoName}-${props.imageTag}`,
      { repositoryArn: ecrRepoArn, repositoryName: props.ecrRepoName }
    );

    const taskDefinition = new FargateTaskDefinition(
      this,
      `${getEnv().envName}-taskDef-${props.serviceName}`
    );
    const container = taskDefinition.addContainer(
      `${props.serviceName}-contaier`,
      {
        image: ContainerImage.fromEcrRepository(ecrRepo, props.imageTag),
        containerName: props.serviceName,
        environment: { MY_ENVIRONMENT_VAR: 'FOO' },
        memoryLimitMiB: 512,
        cpu: 256
      }
    );
    container.addPortMappings({ containerPort: props.containerPort });

    const service = new FargateService(
      this,
      `${getEnv().envName}-${props.serviceName}`,
      {
        cluster: props.cluster,
        taskDefinition,
        vpcSubnets: props.vpcSubnets,
        serviceName: props.serviceName,
        desiredCount: 1
      }
    );
    service.connections.allowFrom(props.alb, Port.tcp(props.containerPort));

    const targetGroup = new ApplicationTargetGroup(
      this,
      `${props.serviceName}-Targetgroup`,
      {
        targets: [service],
        protocol: ApplicationProtocol.HTTP,
        port: props.containerPort,
        vpc: props.vpc
      }
    );

    props.listener.addTargetGroups('targetgroup', {
      targetGroups: [targetGroup],
      conditions: [ListenerCondition.hostHeaders([props.domainName])],
      priority: props.listenerPriority
    });

    const recordSet = new RecordSet(this, `${props.serviceName}-RecordSet`, {
      recordType: RecordType.CNAME,
      target: RecordTarget.fromValues(props.alb.loadBalancerDnsName),
      zone: props.domainZone,
      recordName: props.domainName
    });

    const scalableTarget = service.autoScaleTaskCount({
      minCapacity: 1,
      maxCapacity: props.autoScaleMaxCount ? props.autoScaleMaxCount : 10
    });

    scalableTarget.scaleOnCpuUtilization('CpuScaling', {
      targetUtilizationPercent: 50
    });

    scalableTarget.scaleOnMemoryUtilization('MemoryScaling', {
      targetUtilizationPercent: 50
    });

    this.service = service;
    this.ecrRepo = ecrRepo;
  }
}

所感

• CDKでCodePipelineを使うとGithub連携でCodeStarConnectionsが使えない？
• CDKだとRoleの設定が分かりづらい
• CDKだとリソース名の設定が分かりづらい
• CDK PipelineとCDK CodePipelineって全くの別物やないかい！
• でも型によるサジェストは最高