はじめましてCloudFormation

普段TerraformでAWSをいじっているのでほぼ触ってこなかったCloudFormationをちょっとやってみようということで、AWSのGithubにいっぱいsampleがあるので一つ試してみます。
本のECサイトをCloudFormationにて構築していきます。
AWS Bookstore Demo App

こんな感じらしい

Getting started

READMEに従ってコンソールで進めていきます。
まずはスタックの作成

ご丁寧に入力済みなのでご好意に甘えて進めていきます。

スタック名やらバケット名やらを設定していきます。（master-fullstack.yaml#L480）

確認画面にて表示されるIAMリソース作成についてもチェックをして作成をポチッとな

スタックの作成が完了しました。
In Progressなのでリソース郡の作成は始まっているみたいですね。

こんな感じで大量にイベントが出力されます。

describe-stack-events

aws --region us-east-1 cloudformation describe-stack-events --stack-name demoBookStoreApp --query 'StackEvents[]'

[
    {
        "StackId": "arn:aws:cloudformation:us-east-1:123456789012:stack/demoBookStoreApp/f9d5fa10-9590-11e9-8a60-1202dd259b0c",
        "EventId": "eb6581a0-9593-11e9-9ced-0e061c1f1416",
        "StackName": "demoBookStoreApp",
        "LogicalResourceId": "demoBookStoreApp",
        "PhysicalResourceId": "arn:aws:cloudformation:us-east-1:123456789012:stack/demoBookStoreApp/f9d5fa10-9590-11e9-8a60-1202dd259b0c",
        "ResourceType": "AWS::CloudFormation::Stack",
        "Timestamp": "2019-06-23T08:50:15.345Z",
        "ResourceStatus": "CREATE_COMPLETE",
        "ClientRequestToken": "Console-CreateStack-348cc951-69b5-28b1-7fe0-b1d338e6f3f8"
    },

=========================================================

    {
        "StackId": "arn:aws:cloudformation:us-east-1:123456789012:stack/demoBookStoreApp/f9d5fa10-9590-11e9-8a60-1202dd259b0c",
        "EventId": "f9d69650-9590-11e9-8a60-1202dd259b0c",
        "StackName": "demoBookStoreApp",
        "LogicalResourceId": "demoBookStoreApp",
        "PhysicalResourceId": "arn:aws:cloudformation:us-east-1:123456789012:stack/demoBookStoreApp/f9d5fa10-9590-11e9-8a60-1202dd259b0c",
        "ResourceType": "AWS::CloudFormation::Stack",
        "Timestamp": "2019-06-23T08:29:11.258Z",
        "ResourceStatus": "CREATE_IN_PROGRESS",
        "ResourceStatusReason": "User Initiated",
        "ClientRequestToken": "Console-CreateStack-348cc951-69b5-28b1-7fe0-b1d338e6f3f8"
    }
]

aws --region us-east-1 cloudformation describe-stack-events --stack-name demoBookStoreApp --query 'StackEvents[]' | wc -l
    4835

まあ大量にできます。

list_resources

$ aws --region us-east-1 cloudformation list-stack-resources --stack-name demoBookStoreApp --query 'StackResourceSummaries[].LogicalResourceId[]'
[
    "APIDeployment",
    "ApiAuthorizer",
    "AppApi",
    "AssetsBucket",
    "AssetsBucketOriginAccessIdentity",
    "AssetsBucketPolicy",
    "AssetsCDN",
    "AssetsCodePipeline",
    "AssetsCodeRepository",
    "BestsellersApiRequestGET",
    "BestsellersApiRequestOPTIONS",
    "BestsellersApiResource",
    "BookItemApiRequestGET",
    "BookItemApiRequestOPTIONS",
    "BookItemApiResource",
    "BooksApiRequestGET",
    "BooksApiRequestOPTIONS",
    "BooksApiResource",
    "BooksUploader",
    "CartApiRequestDELETE",
    "CartApiRequestGET",
    "CartApiRequestOPTIONS",
    "CartApiRequestPOST",
    "CartApiRequestPUT",
    "CartApiResource",
    "CartItemApiRequestGET",
    "CartItemApiRequestOPTIONS",
    "CartItemApiResource",
    "CodeBuildProject",
    "CodeBuildRole",
    "CodePipelineRole",
    "CognitoAuthorizedRole",
    "CognitoUnAuthorizedRole",
    "CreateESRole",
    "CreateESRoleFunction",
    "DataTableStream",
    "DynamoDbRole",
    "ESRoleCreator",
    "ESSearchRole",
    "ElastiCacheCluster",
    "ElasticsearchDomain",
    "FunctionAddToCart",
    "FunctionAddToCartPermissions",
    "FunctionCheckout",
    "FunctionCheckoutPermissions",
    "FunctionGetBestSellers",
    "FunctionGetBestSellersPermissions",
    "FunctionGetBook",
    "FunctionGetBookPermissions",
    "FunctionGetCartItem",
    "FunctionGetCartItemPermissions",
    "FunctionGetRecommendations",
    "FunctionGetRecommendationsByBook",
    "FunctionGetRecommendationsByBookPermissions",
    "FunctionGetRecommendationsPermissions",
    "FunctionListBooks",
    "FunctionListBooksPermissions",
    "FunctionListItemsInCart",
    "FunctionListItemsInCartPermissions",
    "FunctionListOrders",
    "FunctionListOrdersPermissions",
    "FunctionRemoveFromCart",
    "FunctionRemoveFromCartPermissions",
    "FunctionSearch",
    "FunctionSearchPermissions",
    "FunctionUpdateBestSellers",
    "FunctionUpdateCart",
    "FunctionUpdateCartPermissions",
    "FunctionUploadBooks",
    "IdentityPool",
    "IdentityPoolRoleMapping",
    "OrderTableStream",
    "OrdersApiRequestGET",
    "OrdersApiRequestOPTIONS",
    "OrdersApiRequestPOST",
    "OrdersApiResource",
    "PipelineArtifactsBucket",
    "RecomendationsApiRequestGET",
    "RecomendationsApiRequestOPTIONS",
    "RecomendationsByBookApiRequestGET",
    "RecomendationsByBookApiRequestOPTIONS",
    "RecommendationsApiResource",
    "RecommendationsByBookApiResource",
    "RecommendationsLambdaRole",
    "RedisRole",
    "RepositorySeeder",
    "RepositoryUpdater",
    "S3Endpoint",
    "SNSRole",
    "SearchApiRequestGET",
    "SearchApiRequestOPTIONS",
    "SearchApiResource",
    "SeederFunction",
    "SeederRole",
    "TBooks",
    "TCart",
    "TOrders",
    "UpdateConfigFunction",
    "UpdateSearchCluster",
    "UserPool",
    "UserPoolClient",
    "bookstoreCacheSecurityGroup",
    "bookstoreCacheSubnets",
    "bookstoreNeptuneCluster",
    "bookstoreNeptuneDB",
    "bookstoreNeptuneIAMAttach",
    "bookstoreNeptuneIAMAttachLambda",
    "bookstoreNeptuneIAMAttachLambdaRole",
    "bookstoreNeptuneIAMAttachLambdaRoleCloudWatchGroup",
    "bookstoreNeptuneIAMAttachLambdaRoleCloudWatchStream",
    "bookstoreNeptuneIAMAttachLambdaRoleEC2",
    "bookstoreNeptuneIAMAttachLambdaRoleRDS",
    "bookstoreNeptuneLoader",
    "bookstoreNeptuneLoaderLambda",
    "bookstoreNeptuneLoaderLambdaRole",
    "bookstoreNeptuneLoaderLambdaRoleCloudWatchGroup",
    "bookstoreNeptuneLoaderLambdaRoleCloudWatchStream",
    "bookstoreNeptuneLoaderLambdaRoleEC2",
    "bookstoreNeptuneLoaderS3ReadPolicy",
    "bookstoreNeptuneLoaderS3ReadRole",
    "bookstoreNeptuneSecurityGroup",
    "bookstoreNeptuneSubnetGroup",
    "bookstoreSecurityGroup",
    "bookstoreSubnet1",
    "bookstoreSubnet2",
    "bookstoreVPC",
    "bookstoreVPCRouteTable",
    "bookstoreVPCRouteTableAssociation",
    "bookstoreVPCRouteTableAssociationTwo",
    "redisLambdaSecurityGroup"
]

２０分くらいで完了しました。
出力タブを確認するとWebApplicationのCloudFrontのURLが表示されています。（master-fullstack.yaml#L3416）
こちらはS3にホスティングされているものをCloudFront経由でアクセス参照しており、S3のオリジンURLはスタック作成時に設定した http://demo-bookstore-app.s3-website-us-east-1.amazonaws.com になります。

こちらがデモアプリになるので早速確認してみましょう。

登録画面に遷移して、登録してみます。（master-fullstack.yaml#L2828）

入力後、メールが届きますので、コードを入力して

ログイン完了

検索してみたり、カートに入れてみたり、キャンセルしてみたりとしばし回遊しましたが快適に使えました。

Cleaning up

スタックを削除するだけで勝手に消してくれます。
コンソールだとこんな感じ。

不要なお試しスタック達がいたのでスクリプト作って放置しました。（消すのに結構時間かかる。。）

delete_stack_all

#!/bin/bash

list=$(aws --region us-east-1 cloudformation list-stacks --query 'StackSummaries[].StackName[]' --output text | tr "\t" "\n")

for target in list
  do
    aws --region us-east-1 cloudformation list-stack-resources --stack-name ${target} --query 'StackSummaries[].StackStatus[]' --output text
    aws --region us-east-1 cloudformation delete-stack --stack-name ${target}
    aws --region us-east-1 cloudformation list-stack-resources --stack-name ${target} --query 'StackSummaries[].StackStatus[]' --output text 
  done

と思ったら失敗してました。

result_failed

$ aws --region us-east-1 cloudformation list-stacks
{
    "StackSummaries": [
        {
            "StackId": "arn:aws:cloudformation:us-east-1:123456789012:stack/demoBookStoreApp/f9d5fa10-9590-11e9-8a60-1202dd259b0c",
            "StackName": "demoBookStoreApp",
            "CreationTime": "2019-06-23T08:29:11.258Z",
            "DeletionTime": "2019-06-23T12:20:49.292Z",
            "StackStatus": "DELETE_FAILED",
            "StackStatusReason": "The following resource(s) failed to delete: [PipelineArtifactsBucket, AssetsBucket]. ",
            "DriftInformation": {
                "StackDriftStatus": "NOT_CHECKED"
            }
        }
    ]
}

S3バケットを空にしてからじゃないとダメみたいですね。
スタックの削除の失敗

result_success

$ aws --region us-east-1 cloudformation list-stacks
{
    "StackSummaries": [
        {
            "StackId": "arn:aws:cloudformation:us-east-1:123456789012:stack/demoBookStoreApp/f9d5fa10-9590-11e9-8a60-1202dd259b0c",
            "StackName": "demoBookStoreApp",
            "CreationTime": "2019-06-23T08:29:11.258Z",
            "DeletionTime": "2019-06-23T12:41:27.108Z",
            "StackStatus": "DELETE_COMPLETE",
            "DriftInformation": {
                "StackDriftStatus": "NOT_CHECKED"
            }
        }
    ]
}

今度は成功しました。

Appendix

CloudFormationのeventsを確認するときにCLIとPythonで比較してみました。
やっぱりPythonの方が早いですね〜。
この少しの差が積み重なると大きくなるので、運用スクリプト等はPythonやGoで書いておきたいですね。

result

$ python_start_time=$(gdate +"%s.%3N")
$ python3 describe_stack_events.py | pbcopy
$ python_end_time=$(gdate +"%s.%3N")
$ echo "scale=1; $python_end_time - $python_start_time" | bc
2.855

$ cli_start_time=$(gdate +"%s.%3N")
$ aws --region us-east-1 cloudformation describe-stack-events --stack-name demoBookStoreApp --query 'StackEvents[]' | pbcopy
$ cli_end_time=$(gdate +"%s.%3N")
$ echo "scale=1; $cli_end_time - $cli_start_time" | bc
6.699

Pythonのスクリプトはこんな感じ

describe_stack_events.py

#!/usr/bin/env python
# coding: utf-8

import boto3
import pprint

pp = pprint.PrettyPrinter(indent=4)

client = boto3.client('cloudformation', region_name='us-east-1')

response = client.describe_stack_events(
    StackName='demoBookStoreApp'
)
result = response['StackEvents']

pp.pprint(result)

結果

result

[   {   'ClientRequestToken': 'Console-CreateStack-348cc951-69b5-28b1-7fe0-b1d338e6f3f8',
        'EventId': 'eb6581a0-9593-11e9-9ced-0e061c1f1416',
        'LogicalResourceId': 'demoBookStoreApp',
        'PhysicalResourceId': 'arn:aws:cloudformation:us-east-1:123456789012:stack/demoBookStoreApp/f9d5fa10-9590-11e9-8a60-1202dd259b0c',
        'ResourceStatus': 'CREATE_COMPLETE',
        'ResourceType': 'AWS::CloudFormation::Stack',
        'StackId': 'arn:aws:cloudformation:us-east-1:123456789012:stack/demoBookStoreApp/f9d5fa10-9590-11e9-8a60-1202dd259b0c',
        'StackName': 'demoBookStoreApp',
        'Timestamp': datetime.datetime(2019, 6, 23, 8, 50, 15, 345000, tzinfo=tzutc())},

====================================================

        'ResourceStatus': 'CREATE_COMPLETE',
        'ResourceType': 'AWS::ApiGateway::Method',
        'StackId': 'arn:aws:cloudformation:us-east-1:123456789012:stack/demoBookStoreApp/f9d5fa10-9590-11e9-8a60-1202dd259b0c',
        'StackName': 'demoBookStoreApp',
        'Timestamp': datetime.datetime(2019, 6, 23, 8, 30, 41, 98000, tzinfo=tzutc())}]

今度はCloudFormationを利用してCI/CDパイプラインでも作ってみたいですね。

Reference

Building a Modern Application with Purpose-Built AWS Databases
aws-bookstore-demo-app