Help us understand the problem. What is going on with this article?

CloudWatch LogsのデータをFirehoseを使ってS3に保存するCloudFormationのテンプレート

More than 1 year has passed since last update.

こちらの、リンクを参考にしています。
CloudWatchLogsのログをFirehose経由でS3に出力してみた | Developers.IO

変更した点は、以下の通りです。

  • 既存のターゲットロググループを指定できる
  • 保存するS3のバケットおよびprefixを指定できる
  • 保存したS3のオブジェクトの有効期限をリスト選択できるようにした(デフォルトは365日)
  • Firehose実行ログの有効期限をリスト選択できるようにした(デフォルトは7日)

参考になれば幸いです。
Developers.IO最高です!

cfn-cwlogs-direct-firehose.yaml
AWSTemplateFormatVersion: '2010-09-09'
Description: Save directly to S3 from CloudWatch Logs using Firehose
Parameters:
  TargetLogGroup:
    Description: 'Input target log group'
    Type: String
    Default: ''
  TargetS3bucket:
    Description: 'Input S3 bucket name to save'
    Type: String
    Default: ''
  TargetS3bucketPrefix:
    Description: 'Input prefix in S3 bucket'
    Type: String
    Default: ''
  S3Expiredate:
    Description: 'Number of days to keep S3 file'
    Type: Number
    Default: 365
    AllowedValues: [1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653]
  CWlogsFirehoseExpiredate:
    Description: 'Number of days to keep Cloudwatch logs'
    Type: Number
    Default: 7
    AllowedValues: [1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653]
Resources:
  TargetS3bucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Delete
    Properties:
      BucketName: !Ref 'TargetS3bucket'
      LifecycleConfiguration:
        Rules:
          - Id: AutoDelete
            Status: Enabled
            ExpirationInDays: !Ref 'S3Expiredate'
  SubscriptionFilter:
    Type: AWS::Logs::SubscriptionFilter
    Properties:
      DestinationArn: !GetAtt 'Deliverystream.Arn'
      FilterPattern: ''
      LogGroupName: !Ref 'TargetLogGroup'
      RoleArn: !GetAtt 'LogsRole.Arn'
  LogsRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: !Sub 'logs.${AWS::Region}.amazonaws.com'
            Action:
              - sts:AssumeRole
      Path: /
      Policies:
        - PolicyName: root
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - firehose:*
                Resource: !GetAtt 'Deliverystream.Arn'
  Deliverystream:
    DependsOn:
      - DeliveryPolicy
    Type: AWS::KinesisFirehose::DeliveryStream
    Properties:
      ExtendedS3DestinationConfiguration:
        BucketARN: !GetAtt 'TargetS3bucket.Arn'
        BufferingHints:
          IntervalInSeconds: '60'
          SizeInMBs: '50'
        CompressionFormat: GZIP
        Prefix: !Join [ "/", [ !Ref TargetS3bucketPrefix, '' ] ]
        RoleARN: !GetAtt 'DeliveryRole.Arn'
        ProcessingConfiguration:
          Enabled: 'false'
  DeliveryRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Sid: ''
            Effect: Allow
            Principal:
              Service: firehose.amazonaws.com
            Action: sts:AssumeRole
            Condition:
              StringEquals:
                sts:ExternalId: !Ref 'AWS::AccountId'
  DeliveryPolicy:
    Type: AWS::IAM::Policy
    Properties:
      PolicyName: firehose_delivery_policy
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Action:
              - s3:AbortMultipartUpload
              - s3:GetBucketLocation
              - s3:GetObject
              - s3:ListBucket
              - s3:ListBucketMultipartUploads
              - s3:PutObject
            Resource:
              - !Sub 'arn:aws:s3:::${TargetS3bucket}'
              - !Sub 'arn:aws:s3:::${TargetS3bucket}*'
      Roles:
        - !Ref 'DeliveryRole'
  LogGroupFirehose:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub '/aws/firehose/${Deliverystream}'
      RetentionInDays: !Ref 'CWlogsFirehoseExpiredate'
takachan
Software/Infrastructure Engineer
http://takachan.hatenablog.jp
Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away