LoginSignup
12
9

More than 1 year has passed since last update.

@hatsu

AWS FargateでRailsアプリとSidekiqを動かすやり方

Posted at

FargateでRailsコンテナと別にSidekiqを動かしたかったので試してみました。
その手順を書いていきます。
なお、RailsにSidekiqを導入するやり方については、本記事では紹介しません。

前提

RailsアプリをFargateで動かすまでにやり方については以下の記事に書いております。

Sidekiqの設定確認

Sidekiq.configure_server do |config|
  config.redis = { url: "redis://#{ENV.fetch("REDIS_URL", "localhost:6379")}", namespace: "sidekiq" }
end

Sidekiq.configure_client do |config|
  config.redis = { url: "redis://#{ENV.fetch("REDIS_URL", "localhost:6379")}", namespace: "sidekiq" }
end

この REDIS_URLは環境変数で定義出来るようにしています。

Jobは以下のようなものを設置しています。

class Batch::TestJob
  include Sidekiq::Worker
  sidekiq_options queue: "default"

  def perform
    logger.info "========Batch::TestJob==========="
  end
end

Jobはこれで動きます。

Batch::TestJob.perform_async

ElastiCacheの導入

Redisを用意するため、ElastiCacheを導入します。

ElastiCache用セキュリティグループの作成

AWS管理画面からセキュリティグループの作成を行います。

セキュリティグループの入力する値は以下です。

フォーム 値
セキュリティグループ名 fargate-test-redis
説明 fargate-test-redis
VPC fargate-test
インバウンドルール タイプ カスタムTCP
インバウンドルール ポート範囲 6379
インバウンドルール ソース fargate-test

イメージ
VPC_Management_Console.png

入力が完了したら、作成して完了です。

ElastiCacheの作成

AWS管理画面の ElastiCache ページから「今すぐ始める」を押します。
ElastiCache_Management_Console.png

フォーム 値
クラスターエンジン redis
ロケーション Amazon クラウド
名前 fargate-test
説明 fargate-test-redis
エンジンバージョンの互換性 6.x(最新)
ポート 6379
パラメータグループ default.redis6.x
ノードのタイプ cache.t3.micro１
パラメータグループ default.redis6.x
レプリケーション数 1
マルチAZ チェック
サブネットグループ 新規作成
サブネットグループ名 fargate-test-redis
サブネットグループ 説明 fargate-test-redis
サブネットグループ VPC fargate-testのもの
サブネットグループ サブネット プライベートサブネット2つ(ap-northeast-1a,ap-northeast-1c)
アベイラビリティーゾーンの配置 指定なし
セキュリティグループ 先程作ったfaragate-test-redisのセキュリティグループ

イメージ

ElastiCache_Management_Console.png
ElastiCache_Management_Console.png
ElastiCache_Management_Console.png
ElastiCache_Management_Console.png

上記のように入力が完了したら、「作成」ボタンを押して作成を行います。

SSMパラメータストアにREDIS_URLを登録

ElastiCacheの作成が完了したら、ElastiCacheの「プライマリエンドポイント」をパラメータストアに登録します。(クラスタモードを有効にしている場合は「設定エンドポイント」を登録)
System Managerの画面からパラメータストアを選択します。

AWS_Systems_Manager.png

パラメータストアの作成を行います。
AWS_Systems_Manager_-_Parameter_Store.png

プライマリエンドポイント
ElastiCache_Management_Console.png

入力する内容は以下です。

名前 タイプ KMS キーソース KMS キー ID 値
/fargate-test/redis-url 安全な文字列 現在のアカウント alias/aws/ssm ElasitCacheのプライマリエンドポイント

イメージ
AWS_Systems_Manager_-_Parameter_Store.png
AWS_Systems_Manager_-_Parameter_Store.png

自動デプロイでSidekiqコンテナを動かす

アプリケーションに書いたtask-definition.jsonを編集

↑こちらの記事で自動デプロイの設定がすでに済んでいる前提で進めます。
deploy/task-definition.jsonの編集

元のtask-definition.json
{
  "ipcMode": null,
  "executionRoleArn": "arn:aws:iam::318288222771:role/ecsTaskExecutionRole",
  "containerDefinitions": [
    {
      "dnsSearchDomains": null,
      "environmentFiles": null,
      "logConfiguration": {
        "logDriver": "awslogs",
        "secretOptions": null,
        "options": {
          "awslogs-group": "/ecs/fargate-test",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "entryPoint": [],
      "portMappings": [
        {
          "hostPort": 80,
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "command": [
        "/app/entrypoint.sh"
      ],
      "linuxParameters": null,
      "cpu": 0,
      "environment": [
        {
          "name": "RAILS_ENV",
          "value": "production"
        },
        {
          "name": "RAILS_LOG_TO_STDOUT",
          "value": "true"
        },
        {
          "name": "RAILS_SERVE_STATIC_FILES",
          "value": "true"
        }
      ],
      "resourceRequirements": null,
      "ulimits": null,
      "dnsServers": null,
      "mountPoints": [],
      "workingDirectory": null,
      "secrets": [
        {
          "valueFrom": "/fargate-test/db-host",
          "name": "DB_HOST"
        },
        {
          "valueFrom": "/fargate-test/db-password",
          "name": "DB_PASSWORD"
        },
        {
          "valueFrom": "/fargate-test/db-username",
          "name": "DB_USERNAME"
        },
        {
          "valueFrom": "/fargate-test/rails-master-key",
          "name": "RAILS_MASTER_KEY"
        }
      ],
      "dockerSecurityOptions": null,
      "memory": null,
      "memoryReservation": null,
      "volumesFrom": [],
      "stopTimeout": null,
      "image": "318288222771.dkr.ecr.ap-northeast-1.amazonaws.com/fargate-test:latest",
      "startTimeout": null,
      "firelensConfiguration": null,
      "dependsOn": null,
      "disableNetworking": null,
      "interactive": null,
      "healthCheck": null,
      "essential": true,
      "links": null,
      "hostname": null,
      "extraHosts": null,
      "pseudoTerminal": null,
      "user": null,
      "readonlyRootFilesystem": null,
      "dockerLabels": null,
      "systemControls": null,
      "privileged": null,
      "name": "fargate-test"
    }
  ],
  "placementConstraints": [],
  "memory": "512",
  "taskRoleArn": "arn:aws:iam::318288222771:role/ecsTaskExecutionRole",
  "compatibilities": [
    "EC2",
    "FARGATE"
  ],
  "taskDefinitionArn": "arn:aws:ecs:ap-northeast-1:318288222771:task-definition/fargate-test:5",
  "family": "fargate-test",
  "requiresAttributes": [
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.execution-role-awslogs"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.ecr-auth"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.task-iam-role"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.execution-role-ecr-pull"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.secrets.ssm.environment-variables"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.task-eni"
    }
  ],
  "pidMode": null,
  "requiresCompatibilities": [
    "FARGATE"
  ],
  "networkMode": "awsvpc",
  "cpu": "256",
  "revision": 5,
  "status": "ACTIVE",
  "inferenceAccelerators": null,
  "proxyConfiguration": null,
  "volumes": []
}

編集後のtask-definition.json

やることは、"containerDefinitions"にSidekiqのコンテナも作ることです。

{
  "ipcMode": null,
  "executionRoleArn": "arn:aws:iam::#####:role/ecsTaskExecutionRole",
  "containerDefinitions": [
    {
      "dnsSearchDomains": null,
      "environmentFiles": null,
      "logConfiguration": {
        "logDriver": "awslogs",
        "secretOptions": null,
        "options": {
          "awslogs-group": "/ecs/fargate-test",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "entryPoint": [],
      "portMappings": [
        {
          "hostPort": 80,
          "protocol": "tcp",
          "containerPort": 80
        }
      ],
      "command": [
        "/app/entrypoint.sh"
      ],
      "linuxParameters": null,
      "cpu": 0,
      "environment": [
        {
          "name": "RAILS_ENV",
          "value": "production"
        },
        {
          "name": "RAILS_LOG_TO_STDOUT",
          "value": "true"
        },
        {
          "name": "RAILS_SERVE_STATIC_FILES",
          "value": "true"
        }
      ],
      "resourceRequirements": null,
      "ulimits": null,
      "dnsServers": null,
      "mountPoints": [],
      "workingDirectory": null,
      "secrets": [
        {
          "valueFrom": "/fargate-test/db-host",
          "name": "DB_HOST"
        },
        {
          "valueFrom": "/fargate-test/db-password",
          "name": "DB_PASSWORD"
        },
        {
          "valueFrom": "/fargate-test/db-username",
          "name": "DB_USERNAME"
        },
        {
          "valueFrom": "/fargate-test/rails-master-key",
          "name": "RAILS_MASTER_KEY"
        },
        {
          // パラメータストアからREDIS_URLを追加
          "valueFrom": "/fargate-test/redis-url",
          "name": "REDIS_URL"
        }
      ],
      "dockerSecurityOptions": null,
      "memory": null,
      "memoryReservation": null,
      "volumesFrom": [],
      "stopTimeout": null,
      "startTimeout": null,
      "firelensConfiguration": null,
      "dependsOn": null,
      "disableNetworking": null,
      "interactive": null,
      "healthCheck": null,
      "essential": true,
      "links": null,
      "hostname": null,
      "extraHosts": null,
      "pseudoTerminal": null,
      "user": null,
      "readonlyRootFilesystem": null,
      "dockerLabels": null,
      "systemControls": null,
      "privileged": null,
      "name": "fargate-test"
    },
    // ↓ここから追加
    {
      "dnsSearchDomains": null,
      "environmentFiles": null,
      "logConfiguration": {
        "logDriver": "awslogs",
        "secretOptions": null,
        "options": {
          "awslogs-group": "/ecs/fargate-test",
          "awslogs-region": "ap-northeast-1",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "entryPoint": [],
      // portMappingsの記述は削除
      // Sidekiqを動かすコマンドに変更
      "command": [
        "bundle", "exec", "sidekiq", "-C", "config/sidekiq.yml"
      ],
      "linuxParameters": null,
      "cpu": 0,
      "environment": [
        {
          "name": "RAILS_ENV",
          "value": "production"
        },
        {
          "name": "RAILS_LOG_TO_STDOUT",
          "value": "true"
        },
        {
          "name": "RAILS_SERVE_STATIC_FILES",
          "value": "true"
        }
      ],
      "resourceRequirements": null,
      "ulimits": null,
      "dnsServers": null,
      "mountPoints": [],
      "workingDirectory": null,
      "secrets": [
        {
          "valueFrom": "/fargate-test/db-host",
          "name": "DB_HOST"
        },
        {
          "valueFrom": "/fargate-test/db-password",
          "name": "DB_PASSWORD"
        },
        {
          "valueFrom": "/fargate-test/db-username",
          "name": "DB_USERNAME"
        },
        {
          "valueFrom": "/fargate-test/rails-master-key",
          "name": "RAILS_MASTER_KEY"
        },
        {
          // パラメータストアからREDIS_URLを読み込む
          "valueFrom": "/fargate-test/redis-url",
          "name": "REDIS_URL"
        }
      ],
      "dockerSecurityOptions": null,
      "memory": null,
      "memoryReservation": null,
      "volumesFrom": [],
      "stopTimeout": null,
      "image": "#####.dkr.ecr.ap-northeast-1.amazonaws.com/fargate-test:latest",
      "startTimeout": null,
      "firelensConfiguration": null,
      "dependsOn": null,
      "disableNetworking": null,
      "interactive": null,
      "healthCheck": null,
      "essential": true,
      "links": null,
      "hostname": null,
      "extraHosts": null,
      "pseudoTerminal": null,
      "user": null,
      "readonlyRootFilesystem": null,
      "dockerLabels": null,
      "systemControls": null,
      "privileged": null,
      // コンテナ名をfargate-test-sidekiqにする
      "name": "fargate-test-sidekiq"
    }
   // ↑ここまで追加
  ],
  "placementConstraints": [],
  "memory": "512",
  "taskRoleArn": "arn:aws:iam::#####:role/ecsTaskExecutionRole",
  "compatibilities": [
    "EC2",
    "FARGATE"
  ],
  "taskDefinitionArn": "arn:aws:ecs:ap-northeast-1:#####:task-definition/fargate-test:5",
  "family": "fargate-test",
  "requiresAttributes": [
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.execution-role-awslogs"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.ecr-auth"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.task-iam-role"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.execution-role-ecr-pull"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.secrets.ssm.environment-variables"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.task-eni"
    }
  ],
  "pidMode": null,
  "requiresCompatibilities": [
    "FARGATE"
  ],
  "networkMode": "awsvpc",
  "cpu": "256",
  "revision": 5,
  "status": "ACTIVE",
  "inferenceAccelerators": null,
  "proxyConfiguration": null,
  "volumes": []
}

GithubActionsの編集

元の.github/workflows/deploy.yml
on:
  pull_request:
  push:
    branches:
      - master

name: Deploy to Amazon ECS

jobs:
  deploy:
    name: Deploy
    runs-on: ubuntu-18.04
    environment: production

    steps:
    - name: Checkout
      uses: actions/checkout@v2

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: ap-northeast-1

    - name: Login to Amazon ECR
      id: login-ecr
      uses: aws-actions/amazon-ecr-login@v1

    - name: Build, tag, and push image to Amazon ECR
      id: build-image
      env:
        DOCKER_BUILDKIT: 1
        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
        ECR_REPOSITORY: fargate-test #ECRのリポジトリ名
        IMAGE_TAG: ${{ github.sha }}
      run: |
        docker build \
          -f docker/production/Dockerfile \
          --cache-from=$ECR_REGISTRY/$ECR_REPOSITORY:latest \
          --build-arg BUILDKIT_INLINE_CACHE=1 \
          -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG \
          -t $ECR_REGISTRY/$ECR_REPOSITORY:latest .
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest
        echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"

    - name: 【Rails】 Fill in the new image ID in the Amazon ECS task definition
      id: task-def-with-rails
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: deploy/task-definition.json
        container-name: fargate-test
        image: ${{ steps.build-image.outputs.image }}

    - name: Deploy Amazon ECS task definition
      uses: aws-actions/amazon-ecs-deploy-task-definition@v1
      with:
        task-definition: ${{ steps.task-def-with-rails.outputs.task-definition }} # 上のid: task-def-with-railsを利用する
        service: fargate-test
        cluster: fargate-test
        wait-for-service-stability: true
        codedeploy-appspec: deploy/appspec.yml
        codedeploy-application: AppECS-fargate-test-fargate-test
        codedeploy-deployment-group: DgpECS-fargate-test-fargate-test

編集後のdeploy.yml

元の.github/workflows/deploy.ymlを編集します。

on:
  pull_request:
  push:
    branches:
      - master

name: Deploy to Amazon ECS

jobs:
  deploy:
    name: Deploy
    runs-on: ubuntu-18.04
    environment: production

    steps:
    - name: Checkout
      uses: actions/checkout@v2

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: ap-northeast-1

    - name: Login to Amazon ECR
      id: login-ecr
      uses: aws-actions/amazon-ecr-login@v1

    - name: Build, tag, and push image to Amazon ECR
      id: build-image
      env:
        DOCKER_BUILDKIT: 1
        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
        ECR_REPOSITORY: fargate-test #ECRのリポジトリ名
        IMAGE_TAG: ${{ github.sha }}
      run: |
        docker build \
          -f docker/production/Dockerfile \
          --cache-from=$ECR_REGISTRY/$ECR_REPOSITORY:latest \
          --build-arg BUILDKIT_INLINE_CACHE=1 \
          -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG \
          -t $ECR_REGISTRY/$ECR_REPOSITORY:latest .
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest
        echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"

    - name: 【Rails】 Fill in the new image ID in the Amazon ECS task definition
      id: task-def-with-rails
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: deploy/task-definition.json
        container-name: fargate-test
        image: ${{ steps.build-image.outputs.image }}

    # ↓ここから追加
    - name: 【Sidekiq】 Fill in the new image ID in the Amazon ECS task definition
      id: task-def-with-sidekiq
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: ${{ steps.task-def-with-rails.outputs.task-definition }} # 上のid: task-def-with-railsを利用する
        container-name: fargate-test-sidekiq # コンテナ名を記述
        image: ${{ steps.build-image.outputs.image }}
    # ↑ここまで追加

    - name: Deploy Amazon ECS task definition
      uses: aws-actions/amazon-ecs-deploy-task-definition@v1
      with:
        # ↓sidekiqコンテナの設定を追加したtask定義を読み込むように編集
        task-definition: ${{ steps.task-def-with-sidekiq.outputs.task-definition }} # 上のid: task-def-with-sidekiqを利用する
        service: fargate-test
        cluster: fargate-test
        wait-for-service-stability: true
        codedeploy-appspec: deploy/appspec.yml
        codedeploy-application: AppECS-fargate-test-fargate-test
        codedeploy-deployment-group: DgpECS-fargate-test-fargate-test

これで完成

これで完成です。
Mainブランチにマージしてデプロイを行うと、Sidekiqも動くようになっているかと思います。

Amazon_ECS_と_見積もりフォーマット_numbers.png

Sidekiq.png

12
9
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
12
9