Go to Qiita Advent Calendar 2024 Top
LoginSignup
13
7

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

gumi.inc TS&DAdvent Calendar 2017Day 7gumi
@tsubaki_dev

CloudFormationとecs-cliを使って「Fargate+NLB+fluentd」環境を作成する。

Last updated at Posted at 2017-12-08

AWS Fargateが発表されたのでさっそくつかってみます。
今回はFargateの上でfluentdコンテナを実行します。

手順

  1. CloudFromationのテンプレートでECSクラスタやNLBなどのAWSリソースを作ります。
  2. fluentdのDockerfile、docker-compose.yml、ecs-params.ymlを作ります。
  3. fluentdのDockerイメージを作成し、ECRにpushします。
  4. ecs-cliコマンドでFargateにデプロイします。
  5. ecs-cli logsコマンドで接続を受け付けていることを確認します。

CloudFormationテンプレート

  • VPCから一気に作ります。
AWSTemplateFormatVersion: '2010-09-09'

Description: ""

Parameters:
  VPCCidr:
    Description: ""
    Type: String
    Default: "10.1.0.0/16"
  EnableDnsSupport:
    Description: ""
    Type: String
    Default: true
    AllowedValues:
      - true
      - false
  EnableDnsHostnames:
    Description: ""
    Type: String
    Default: true
    AllowedValues:
      - true
      - false
  InstanceTenancy:
    Description: ""
    Type: String
    Default: default
    AllowedValues:
      - default
      - dedicated
  SubnetCidrA:
    Description: ""
    Type: String
    Default: "10.1.10.0/24"
  SubnetCidrB:
    Description: ""
    Type: String
    Default: "10.1.20.0/24"
  AvailabilityZoneA:
    Description: ""
    Type: String
    Default: us-east-1a
  AvailabilityZoneB:
    Description: ""
    Type: String
    Default: us-east-1b
  MapPublicIpOnLaunch:
    Description: ""
    Type: String
    Default: true
    AllowedValues:
      - true
      - false
  AttachInternetGateway:
    Description: ""
    Type: String
    Default: "true"
    AllowedValues:
      - true
      - false
  Schema:
    Description: ""
    Type: String
    Default: "internet-facing"
    AllowedValues:
      - "internet-facing"
      - "internal"
  ListenerPort:
    Description: ""
    Type: String
    Default: "24224"
  TargetGroupPort:
    Description: ""
    Type: String
    Default: "24224"
  TargetType:
    Description: ""
    Type: String
    Default: "ip"
    AllowedValues:
      - "ip"
      - "instance"

Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: "Network Configuration"
        Parameters:
          - VPCCidr
          - EnableDnsSupport
          - EnableDnsHostnames
          - InstanceTenancy
          - SubnetCidrA
          - SubnetCidrB
          - AvailabilityZoneA
          - AvailabilityZoneB
          - MapPublicIpOnLaunch
          - AttachInternetGateway
      - Label:
          default: "NetworkLoadBalancer Configuration"
        Parameters:
          - Schema
          - ListenerPort
          - TargetGroupPort
          - TargetType

Outputs:
  Cluster:
    Value: !Ref Cluster
  TargetGroupArn:
    Value: !Ref TargetGroup
  SubnetA:
    Value: !Ref SubnetA
  SubnetB:
    Value: !Ref SubnetB
  SecurityGroup:
    Value: !GetAtt SecurityGroup.GroupId
  IAMRole:
    Value: !GetAtt IAMRole.Arn
  Repository:
    Value: !Ref Repository

Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VPCCidr
      EnableDnsSupport: !Ref EnableDnsSupport
      EnableDnsHostnames: !Ref EnableDnsHostnames
      InstanceTenancy: !Ref InstanceTenancy
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}.vpc"

  RouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}.rtable"

  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}.igw"

  AttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC

  Route:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref RouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway

  SubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: !Ref SubnetCidrA
      AvailabilityZone: !Ref AvailabilityZoneA
      MapPublicIpOnLaunch: !Ref MapPublicIpOnLaunch
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}.subnet"

  SubnetARouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref SubnetA
      RouteTableId: !Ref RouteTable

  SubnetB:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: !Ref SubnetCidrB
      AvailabilityZone: !Ref AvailabilityZoneB
      MapPublicIpOnLaunch: !Ref MapPublicIpOnLaunch
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}.subnet"

  SubnetBRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref SubnetB
      RouteTableId: !Ref RouteTable

  SecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: "SecurityGroup"
      VpcId: !Ref VPC
      SecurityGroupIngress:
        - CidrIp: !Ref VPCCidr
          IpProtocol: "tcp"
          FromPort: "24224"
          ToPort: "24224"
        - CidrIp: 0.0.0.0/0
          IpProtocol: "tcp"
          FromPort: "24224"
          ToPort: "24224"
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}.sg"

  Cluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: !Ref AWS::StackName

  LoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: !Sub "${AWS::StackName}-nlb"
      Type: "network"
      Scheme: !Ref Schema
      Subnets:
        - !Ref SubnetA
        - !Ref SubnetB
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}.nlb"

  Listener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      LoadBalancerArn: !Ref LoadBalancer
      Port: !Ref ListenerPort
      Protocol: "TCP"
      DefaultActions:
        - Type: forward
          TargetGroupArn: !Ref TargetGroup

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    DependsOn: LoadBalancer
    Properties:
      Name: !Sub "${AWS::StackName}-tg"
      Port: !Ref TargetGroupPort
      Protocol: "TCP"
      VpcId: !Ref VPC
      TargetType: !Ref TargetType
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}.targetgroup"

  IAMRole:
    Type: "AWS::IAM::Role"
    Properties:
      RoleName: !Sub "${AWS::StackName}.role"
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: "Allow"
            Action:
              - "sts:AssumeRole"
            Principal:
              Service:
                - "ecs-tasks.amazonaws.com"
      Policies:
        - PolicyName: !Sub "${AWS::StackName}.policy"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: "Allow"
                Resource: "*"
                Action:
                  - "ecr:GetAuthorizationToken"
                  - "ecr:BatchCheckLayerAvailability"
                  - "ecr:GetDownloadUrlForLayer"
                  - "ecr:BatchGetImage"
                  - "logs:CreateLogStream"
                  - "logs:PutLogEvents"

  LogGroup:
    Type: "AWS::Logs::LogGroup"
    Properties:
      LogGroupName: !Sub "${AWS::StackName}/fluentd"
      RetentionInDays: 3

  Repository:
    Type: "AWS::ECR::Repository"
    Properties:
      RepositoryName: !Sub "${AWS::StackName}/fluentd"
  • ポイント
    1. NLBにはSecurityGroupはアタッチできない。
    2. コンテナにはVPC Cidrからのアクセスを許可する必要がある。これがないとヘルスチェックが失敗する。
    3. コンテナはロググループを自動生成しない。あらかじめロググループを作成しておく必要がある。

Stackの作成

  • AWSコンソールからCloudFormationを選択してStackを作成します。

cfn

  • しばらく待ちます。

creating

  • 完成です。

created

コンテナの設定

  • 以下のディレクトリ構成で設定ファイルを書きます。
├── Makefile
└── fluentd
    ├── Dockerfile
    ├── docker-compose.yml
    ├── ecs-params.yml
    ├── fluent.conf
    └── plugins //今回は空ディレクトリでok

fluent.conf

  • 標準出力に吐き出すだけです。
  • CloudWatchLogsに転送されます。
<source>
  type forward
  bind 0.0.0.0
  port 24224
</source>

<match **>
  type stdout
</match>

Dockerfile

FROM fluent/fluentd
COPY fluent.conf /fluentd/etc

docker-compose.yml

  • ${AWS::Account}はAWSアカウントID、${AWS::StackName}はCloudFormationで作成したスタックの名前を入れます。
version: '2'
services:
  fluentd:
    image: ${AWS::Account}.dkr.ecr.us-east-1.amazonaws.com/${AWS::StackName}/fluentd
    ports:
      - "24224:24224"
    logging:
      driver: "awslogs"
      options:
        awslogs-region: "us-east-1"
        awslogs-group: "${AWS::StackName}/fluentd"
        awslogs-stream-prefix: "container"

ecs-params.yml

  • ${SubnetA}, ${SubnetB}, ${SecurityGroup}はCloudFormationで作成したリソースのIDを入れます。
  • CloudFormationの「出力」タブに作成したリソースの一覧が表示されているのでコピペします。
  • 自動生成してほしい。
version: 1
task_definition:
  ecs_network_mode: awsvpc
  task_execution_role: arn:aws:iam::${AWS::Account}:role/${AWS::StackName}.role
  task_size:
    cpu_limit: 0.25
    mem_limit: 0.5GB
  services:
    fluentd:
      essential: true

run_params:
  network_configuration:
    awsvpc_configuration:
      subnets:
        - ${SubnetA}
        - ${SubnetB}
      security_groups:
        - ${SecurityGroup}
      assign_public_ip: ENABLED
  • この設定ファイルをみて分かる通り、Fargate(正確にはawsvpcモード)ではコンテナに直接サブネットやセキュリティグループをアタッチします。
  • EC2インスタンスのような感覚でコンテナを扱えます。

Makefile

  • たくさんコマンドをうつのでMakefileをつくっておきます。
push:
	docker build -f fluentd/Dockerfile -t　${AWS::Account}.dkr.ecr.us-east-1.amazonaws.com/${AWS::StackName}/fluentd fluentd
	`aws ecr get-login --no-include-email --region us-east-1`
	docker push ${AWS::Account}.dkr.ecr.us-east-1.amazonaws.com/${AWS::StackName}/fluentd:latest

up:
	cd fluentd; \
	ecs-cli compose service up \
        --cluster ${AWS::StackName} \
        --target-group-arn ${TargetGroupArn} \
        --launch-type FARGATE \
        --container-name fluentd \
        --container-port 24224 \
        --region us-east-1 \
        --timeout 10

rm:
	cd fluentd; \
	ecs-cli compose service rm \
	--cluster ${AWS::StackName} \
	--region us-east-1 \
	--timeout 10

Deploy

  • イメージを作ってECRにpushします。
$ make push
docker build -f fluentd/Dockerfile -t ************.dkr.ecr.us-east-1.amazonaws.com/fargate/fluentd fluentd
Sending build context to Docker daemon  5.632kB
Step 1/2 : FROM fluent/fluentd
 ---> 060874232311
Step 2/2 : COPY fluent.conf /fluentd/etc
 ---> Using cache
 ---> 1ee08befeb8d
Successfully built 1ee08befeb8d
Successfully tagged ************.dkr.ecr.us-east-1.amazonaws.com/fargate/fluentd:latest
`aws ecr get-login --no-include-email --region us-east-1`
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Login Succeeded
docker push ************.dkr.ecr.us-east-1.amazonaws.com/fargate/fluentd:latest
The push refers to a repository [************.dkr.ecr.us-east-1.amazonaws.com/fargate/fluentd]
baa346e06fe3: Pushed 
fe129fa31f70: Pushed 
dcf88bef8f3a: Pushed 
b59190601542: Pushed 
56e1e5a28df0: Pushed 
9fc62b353b50: Pushed 
26fbe6ae586e: Pushed 
16174e87921f: Pushed 
latest: digest: sha256:9f8c90b5fc10c084f93c5a93c038f4d307676b4fb641a8a36d67f4573655d52f size: 1981
  • Fargateにデプロイします。
$ make up
cd fluentd; \
	ecs-cli compose service up \
	--cluster fargate \
	--target-group-arn arn:aws:elasticloadbalancing:us-east-1:************:targetgroup/fargate-tg/**** \
	--launch-type FARGATE \
	--container-name fluentd \
	--container-port 24224 \
	--region us-east-1 \
	--timeout 10
WARN[0000] Skipping unsupported YAML option...           option name=networks
WARN[0000] Skipping unsupported YAML option for service...  option name=networks service name=fluentd
INFO[0001] Using ECS task definition                     TaskDefinition="fluentd:12"
INFO[0002] Created an ECS service                        service=fluentd taskDefinition="fluentd:12"
INFO[0002] Updated ECS service successfully              desiredCount=1 serviceName=fluentd
INFO[0017] (service fluentd) has started 1 tasks: (task 7228958b-0de1-4e31-a6b2-52d35b6c7b84).  timestamp=2017-12-07 02:14:52 +0000 UTC
INFO[0139] Service status                                desiredCount=1 runningCount=1 serviceName=fluentd
INFO[0139] ECS Service has reached a stable state        desiredCount=1 runningCount=1 serviceName=fluentd

確認

  • ログを取得します。
$ ecs-cli ps --cluster fargate --region us-east-1
Name                                          State                Ports                         TaskDefinition
7228958b-0de1-4e31-a6b2-52d35b6c7b84/fluentd  RUNNING              **.**.**.**:24224->24224/tcp  fluentd:12

$ ecs-cli logs --cluster fargate --region us-east-1 --task-id 7228958b-0de1-4e31-a6b2-52d35b6c7b84
2017-12-07 02:17:03 +0000 [info]: reading config file path="/fluentd/etc/fluent.conf"
2017-12-07 02:17:03 +0000 [info]: starting fluentd-0.12.40
2017-12-07 02:17:03 +0000 [info]: gem 'fluentd' version '0.12.40'
2017-12-07 02:17:03 +0000 [info]: adding match pattern="**" type="stdout"
2017-12-07 02:17:03 +0000 [info]: adding source type="forward"
2017-12-07 02:17:03 +0000 [info]: using configuration file: <ROOT>
  <source>
    type forward
    bind 0.0.0.0
    port 24224
  </source>

  <match **>
    type stdout
  </match>
</ROOT>
2017-12-07 02:17:03 +0000 [info]: listening fluent socket on 0.0.0.0:24224
  • 正常に起動しています。
  • fluent-catやtelnetで接続するとログが出力されます。

削除

  • 作成したリソースを消しておかないとお金を取られます。
  • コンテナを削除してから、AWSリソースを削除します。
$ make rm
cd fluentd; \
	ecs-cli compose service rm \
	--cluster fargate \
	--region us-east-1 \
	--timeout 10
WARN[0000] Skipping unsupported YAML option...           option name=networks
WARN[0000] Skipping unsupported YAML option for service...  option name=networks service name=fluentd
INFO[0001] Updated ECS service successfully              desiredCount=0 serviceName=fluentd
INFO[0001] Service status                                desiredCount=0 runningCount=1 serviceName=fluentd
INFO[0017] (service fluentd) has begun draining connections on 1 tasks.  timestamp=2017-12-07 02:44:53 +0000 UTC
INFO[0017] (service fluentd) deregistered 1 targets in (target-group arn:aws:elasticloadbalancing:us-east-1:************:targetgroup/fargate-tg/****)  timestamp=2017-12-07 02:44:53 +0000 UTC
INFO[0321] Service status                                desiredCount=0 runningCount=0 serviceName=fluentd
INFO[0321] ECS Service has reached a stable state        desiredCount=0 runningCount=0 serviceName=fluentd
INFO[0321] Deleted ECS service                           service=fluentd
INFO[0322] ECS Service has reached a stable state        desiredCount=0 runningCount=0 serviceName=fluentd
  • AWSコンソールでECRリポジトリを削除します。
    • CloudFromationで作成したリポジトリにイメージが登録されていると、CloudFormationでは削除できません。
  • AWSコンソール -> CloudFromation -> Stackの削除をします。

使ってみた感想

今まではECSクラスタを構成するEC2インスタンスの制限を受けて使いにくい部分(動的ポートマッピングとかawsvpcのeni制限とかスケールとか)がありましたが、Fargateによってそれらが一気に解決しました。AWSでコンテナ運用するならFargate一択ですね。

参考

  1. AWS Fargate: サービス概要
  2. GitHub - aws/amazon-ecs-cli
13
7
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
13
7

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?