Go to Qiita Advent Calendar 2024 Top
LoginSignup
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

@kai_kou(甲斐 甲)inアイレット株式会社

AWS CloudFormationのLambda-Backedカスタムリソースで`cfnresponse.send()` 実行時に`TypeError: Object of type datetime is not JSON serializable` エラーになった際の対応方法

Last updated at Posted at 2019-07-22

cloudpack あら便利カレンダー 2019 の記事となります。誕生秘話 はこちら。

AWS CloudFormation(CFn)でAmazon Managed Blockchainのリソースを管理しようとテンプレート作成してた際にハマったエラーです。

エラー原因

JSONに含まれる日付がdatetime.datetime() となっているため発生します。
cfnresponse.send でのjson.dumps 実行時にエラーとなります。

解決策

JSONに含まれる日付をdatetime.datetime() から文字列に変換して対応します。
下記が参考になりました。

[Python] dateやdatetimeをjson.dumpでエラーなく出力する - YoheiM .NET
https://www.yoheim.net/blog.php?q=20170703

エラー再現と対応

前提

  • AWSアカウントがある
  • AWS CLIが利用できる
  • AWS Lambda、CFnの作成権限がある

エラーとなるテンプレート定義

cfn-template.yaml
Resources:
  CustomResource:
    Type: Custom::CustomResource
    Properties:
      ServiceToken: !GetAtt CustomResourceFunction.Arn

  CustomResourceFunction:
    Type: AWS::Lambda::Function
    Properties:
      Handler: index.handler
      Role: !GetAtt FunctionExecutionRole.Arn
      Code:
        ZipFile: !Sub |
          import cfnresponse
          import datetime
          def handler(event, context):
            response = {}
            if event['RequestType'] == 'Create':
              response = {
                "Id": "hoge",
                "Datetime": datetime.datetime.today()
              }

            cfnresponse.send(event, context, cfnresponse.SUCCESS, response)
      Runtime: python3.7

  FunctionExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: root
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
              - logs:CreateLogGroup
              - logs:CreateLogStream
              - logs:PutLogEvents
            Resource: "arn:aws:logs:*:*:*"
Outputs:
  hoge:
    Value: !GetAtt CustomResource.Datetime

Lambda関数を抜粋してみるとcfnresponse.send のパラメータdata のJSONdatetime.datetime.today() で設定して値がdatetime.datetime() となるようにします。

cfn-template.yaml_抜粋
import cfnresponse
import datetime
def handler(event, context):
  response = {}
  if event['RequestType'] == 'Create':
    response = {
      "Id": "hoge",
      "Datetime": datetime.datetime.today()
    }

  cfnresponse.send(event, context, cfnresponse.SUCCESS, response)

スタック作成して確認する

aws cloudformation create-stack \
  --stack-name cfn-json-error \
  --template-body file://cfn-template.yaml \
  --capabilities CAPABILITY_IAM

{
    "StackId": "arn:aws:cloudformation:us-east-1:xxxxxxxxxxxx:stack/cfn-json-error/2371e5d0-97e6-11e9-8372-0abba895ce2c"
}

# エラーでロールバックするまでしばらくかかります...
aws cloudformation describe-stacks \
  --stack-name cfn-json-error

{
    "Stacks": [
        {
            "StackId": "arn:aws:cloudformation:us-east-1:xxxxxxxxxxxx:stack/cfn-json-error/2371e5d0-97e6-11e9-8372-0abba895ce2c",
            "StackName": "cfn-json-error",
            "CreationTime": "2019-06-26T07:43:50.455Z",
            "DeletionTime": "2019-06-26T08:44:33.971Z",
            "RollbackConfiguration": {},
            "StackStatus": "ROLLBACK_COMPLETE",
            "DisableRollback": false,
            "NotificationARNs": [],
            "Capabilities": [
                "CAPABILITY_IAM"
            ],
            "Tags": [],
            "EnableTerminationProtection": false,
            "DriftInformation": {
                "StackDriftStatus": "NOT_CHECKED"
            }
        }
    ]
}

Lambda関数のエラーログを一部抜粋します。

[ERROR] TypeError: Object of type datetime is not JSON serializable
Traceback (most recent call last):
  File "/var/task/index.py", line 11, in handler
    cfnresponse.send(event, context, cfnresponse.SUCCESS, data)
  File "/var/task/cfnresponse.py", line 29, in send
    json_responseBody = json.dumps(responseBody)
  File "/var/lang/lib/python3.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/var/lang/lib/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/var/lang/lib/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/var/lang/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '

エラー対応する

テンプレート定義

エラー対応したテンプレート定義です。

cfn-template.yaml_対応版
Resources:
  CustomResource:
    Type: Custom::CustomResource
    Properties:
      ServiceToken: !GetAtt CustomResourceFunction.Arn

  CustomResourceFunction:
    Type: AWS::Lambda::Function
    Properties:
      Handler: index.handler
      Role: !GetAtt FunctionExecutionRole.Arn
      Code:
        ZipFile: !Sub |
          import cfnresponse
          import json
          from datetime import date, datetime
          def json_serial(obj):
            if isinstance(obj, (datetime, date)):
              return obj.isoformat()
            raise TypeError ('Type %s not serializable' % type(obj))

          def handler(event, context):
            response = {}
            if event['RequestType'] == 'Create':
              response = {
                "Id": "hoge",
                "Datetime": datetime.today()
              }
              response = json.loads(json.dumps(response, default=json_serial))

            cfnresponse.send(event, context, cfnresponse.SUCCESS, response)
      Runtime: python3.7

  FunctionExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: root
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
              - logs:CreateLogGroup
              - logs:CreateLogStream
              - logs:PutLogEvents
            Resource: "arn:aws:logs:*:*:*"
Outputs:
  hoge:
    Value: !GetAtt CustomResource.Datetime

ポイント

json.dumps のdefault を指定して日付を文字列に変換する

実装そのままです。
json.dumps でJSONを文字列変換する際にdefault に指定したjson_serial メソッドで日付を文字列に変換します。
cfnresponse.send にはJSONを渡す必要があるのでjson.loads で文字列から再度JSONに戻しています。

cfn-template.yaml_対応版_抜粋
import cfnresponse
import json
from datetime import date, datetime
def json_serial(obj):
  if isinstance(obj, (datetime, date)):
    return obj.isoformat()
  raise TypeError ('Type %s not serializable' % type(obj))

def handler(event, context):
  response = {}
  if event['RequestType'] == 'Create':
    response = {
      "Id": "hoge",
      "Datetime": datetime.today()
    }
    response = json.loads(json.dumps(response, default=json_serial))

  cfnresponse.send(event, context, cfnresponse.SUCCESS, response)

再度スタック作成して確認する

aws cloudformation delete-stack \
  --stack-name cfn-json-error

> aws cloudformation create-stack \
  --stack-name cfn-json-error \
  --template-body file://cfn-template.yaml \
  --capabilities CAPABILITY_IAM

{
    "StackId": "arn:aws:cloudformation:us-east-1:xxxxxxxxxxxx:stack/cfn-json-error/fead9f00-97ef-11e9-b4fc-0e16aabfe77c"
}


> aws cloudformation describe-stacks \
  --stack-name cfn-json-error

{
    "Stacks": [
        {
            "StackId": "arn:aws:cloudformation:us-east-1:xxxxxxxxxxxx:stack/cfn-json-error/fead9f00-97ef-11e9-b4fc-0e16aabfe77c",
            "StackName": "cfn-json-error",
            "CreationTime": "2019-06-26T08:54:23.736Z",
            "RollbackConfiguration": {},
            "StackStatus": "CREATE_COMPLETE",
            "DisableRollback": false,
            "NotificationARNs": [],
            "Capabilities": [
                "CAPABILITY_IAM"
            ],
            "Outputs": [
                {
                    "OutputKey": "hoge",
                    "OutputValue": "2019-06-26T08:54:48.738374"
                }
            ],
            "Tags": [],
            "EnableTerminationProtection": false,
            "DriftInformation": {
                "StackDriftStatus": "NOT_CHECKED"
            }
        }
    ]
}

無事に日付が文字列に変換されてスタック作成できたのが確認できました。

参考

[Python] dateやdatetimeをjson.dumpでエラーなく出力する - YoheiM .NET
https://www.yoheim.net/blog.php?q=20170703

0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?