Go to Qiita Advent Calendar 2024 Top
LoginSignup
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

@teruroom(Teru Room)

【コマンドベース】Fargate + ECR + ECS 構成による FastAPI dynamoDB 連携サービスの稼働

Last updated at Posted at 2024-11-04

通常コンテナ更新時サマリー

リージョン クラスター名 タスク定義ファミリー サービス名
$REGION $CONTAINER_NAME-cluster $CONTAINER_NAME-definition $CONTAINER_NAME-service

  • ローカル端末でのコンテナサービステスト

    cd ~/aws/mongodb && source ./Local_Container_Service_Test.sh

  • 既存のクラスター、タスク定義、サービスに対して更新

    cd ~/aws/mongodb && source ./Upadte_Cluster_TaskDef_Service.sh

  • 更新したECSサービスのタスクのパブリックIPを取得しテストを実行 ※数分待ってから実行

    cd ~/aws/mongodb && source ./Get_ECS_Service_task_IP_Address.sh

  • サービスの実行例

    match_skills POST サービス 
    cd ~/aws/mongodb && source ./Get_ECS_Service_task_IP_Address.sh
INDV_ID=HOGE22222222 && INDV_NMAE="禿山 木瓜尾" && COMP_ID=BAA006 && COMP_NUM=02
echo $PUBLIC_IP':'$INDV_ID':'$INDV_NMAE':'$COMP_ID':'$COMP_NUM
curl -X 'POST' \
  'http://'$PUBLIC_IP'/match_skills/' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "individual_user": {
        "ID_manuke": "'$INDV_ID'",
        "name_ahomarudashi": "'$INDV_NMAE'"
  },
  "job_description": {
        "nukesaku_ID": "'$COMP_ID'",
        "BAKA_Number": "'$COMP_NUM'"
  }
}'

シェルスクリプト

  • ローカル端末でのコンテナサービステスト

    • シェルスクリプト

      ./Local_Container_Service_Test.sh 
      ## 0. 環境変数を設定
export AWS_ACOUNT_ID=12132323232
export REGION=us-east-1
export AWS_PROFILE=hageconf
export CONTAINER_NAME=aho_baka
export KIND=fastapi
export IMAGE_MAME=$CONTAINER_NAME'-'$KIND

## 1. GitHubリモートリモートリポジトリの更新
git add . && git commit -m "regular update" && git push -u origin main

## 2. Dockerコンテナをビルド
cd ~/aws/mongodb/ && docker ps -a && docker images && ./remove_container.sh $CONTAINER_NAME
docker build -t $IMAGE_MAME -f Dockerfile_$CONTAINER_NAME .
docker ps -a && docker images

## 3. Dockerコンテナサービスを実行
docker run -d -p 80:80 --name $IMAGE_MAME $IMAGE_MAME
docker ps -a && docker images

## 4. ローカル端末でサービスをテスト:
curl -X 'POST' \
  'http://localhost/individual/heatmap/' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{"id_individual":"C2038023082","name":"阿呆打 馬鹿打"}'
  • 既存のクラスター、タスク定義、サービスに対して更新

    • シェルスクリプト

      ./Upadte_Cluster_TaskDef_Service.sh 
      #---------------------------------------------------
# 既存のクラスター、タスク定義、サービスに対して更新をかける 
#---------------------------------------------------
## 0. 定数を設定
export AWS_ACOUNT_ID=211125541523
export REGION=us-east-1
export AWS_PROFILE=hrindconf
export CONTAINER_NAME=individual_company
export KIND=fastapi
export IMAGE_MAME=$CONTAINER_NAME'-'$KIND

## 1. GitHubリモートリモートリポジトリの更新
git add . && git commit -m "regular update" && git push -u origin main

## 2. Dockerコンテナをビルドし、コンテナイメージにタグを付与
cd ~/aws/mongodb/ && docker ps -a && docker images && ./remove_container.sh $CONTAINER_NAME
docker build -t $IMAGE_MAME -f Dockerfile_$CONTAINER_NAME .
TAG_NAMING='docker tag '$IMAGE_MAME':latest '$AWS_ACOUNT_ID'.dkr.ecr.'$REGION'.amazonaws.com/'$IMAGE_MAME':latest'
eval $TAG_NAMING
docker ps -a && docker images

## 3. 認証トークンを取得し、レジストリに対して Docker クライアントを認証
aws ecr get-login-password --region $REGION --profile $AWS_PROFILE | docker login --username AWS --password-stdin $AWS_ACOUNT_ID.dkr.ecr.us-east-1.amazonaws.com

## 4. DockerイメージをECRにプッシュ
docker push $AWS_ACOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$IMAGE_MAME

## 5. 新しいタスク定義を登録
aws ecs register-task-definition \
  --family $CONTAINER_NAME-definition \
  --network-mode awsvpc \
  --requires-compatibilities FARGATE \
  --cpu "256" \
  --memory "512" \
  --execution-role-arn arn:aws:iam::$AWS_ACOUNT_ID':role/ecsTaskExecutionRole' \
  --container-definitions '[
    {
      "name": "aho_baka_manuke-container",
      "image": "'$AWS_ACOUNT_ID'.dkr.ecr.'$REGION'.amazonaws.com/'$IMAGE_MAME':latest",
      "essential": true,
      "portMappings": [
        {
          "containerPort": 80,
          "hostPort": 80
        }
      ]
    }
  ]' \
  --region $REGION

## 6. 新しいタスク定義のARNを取得
NEW_TASK_DEF_ARN=$(aws ecs describe-task-definition --task-definition $CONTAINER_NAME-definition --query 'taskDefinition.taskDefinitionArn' --output text)

## 7. 新しいタスク定義を使用して既存のサービスを更新
aws ecs update-service \
  --cluster $CONTAINER_NAME-cluster \
  --service $CONTAINER_NAME-service \
  --task-definition $NEW_TASK_DEF_ARN \
  --force-new-deployment \
  --region $REGION
  • 更新したECSサービスのタスクのパブリックIPを取得しテストを実行

    • シェルスクリプト

      ./Get_ECS_Service_task_IP_Address.sh 
      #---------------------------------------------------
# 更新したECSサービスのタスクのパブリックIPを取得
#---------------------------------------------------
## 0. 定数を設定
AWS_ACOUNT_ID=77777777777 && REGION=us-east-1 && AWS_PROFILE=hageconf && CONTAINER_NAME=aho_baka_manuke && KIND=fastapi && IMAGE_MAME=$CONTAINER_NAME'-'$KIND

## 1. 実行中のタスクのARNを取得:
TASK_ARN=$(aws ecs list-tasks --cluster $CONTAINER_NAME-cluster --service-name $CONTAINER_NAME-service --desired-status RUNNING --query 'taskArns[0]' --output text)
echo $TASK_ARN

## 2. タスクの詳細を取得:
ENI_ID=$(aws ecs describe-tasks --cluster $CONTAINER_NAME-cluster --tasks $TASK_ARN --query 'tasks[0].attachments[0].details[?name==`networkInterfaceId`].value' --output text)
echo $ENI_ID

## 3. ENI（Elastic Network Interface）のパブリックIPを取得:
PUBLIC_IP=$(aws ec2 describe-network-interfaces --network-interface-ids $ENI_ID --query 'NetworkInterfaces[0].Association.PublicIp' --output text)
echo $PUBLIC_IP
## 4. パブリックIPを使用してサービスをテスト:
curl -X 'POST' \
  'http://'$PUBLIC_IP'/individual/heatmap/' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d {"id_individual":"C2038023082","name":"阿呆打 馬鹿打"}'
  • 動作中のコンテナを停止し、イメージを削除する

    • シェルスクリプト

      ./remove_container.sh 
      CONTAINER_NAME=$1
docker stop $CONTAINER_NAME-fastapi && docker rm -f $CONTAINER_NAME-fastapi
docker rmi $(docker images $CONTAINER_NAME-fastapi --format "{{.ID}}")
docker rmi -f $(docker images -q)
docker ps -a && docker images

初回登録

  • AWS CLIの設定情報を確認

    cat ~/.aws/config
    実行結果 
    [default]
region = us-east-1
aws_access_key_id = HHHHHHHHHHHHHHHH
aws_secret_access_key = bakabakabakamanukeshineohondara

[profile hageconf]
region = us-east-1
aws_access_key_id = HHHHHHHHHHHHHHHH
aws_secret_access_key = bakabakabakamanukeshineohondara

[profile usprod]
region = us-east-1
aws_access_key_id = HHHHHHHHHHHHHHHH
aws_secret_access_key = bakabakabakamanukeshineohondara

[profile prod]
region = ap-northeast-1
aws_access_key_id = HHHHHHHHHHHHHHHH 
aws_secret_access_key = bakabakabakamanukeshineohondara

[profile dev]
region = ap-northeast-1
aws_access_key_id = AKIAVASCR35HR3LGK7IY
aws_secret_access_key = nukesakubakamanukeahoaho

  • 認証トークンを取得し、レジストリに対して Docker クライアントを認証

    • リージョン：バージニア北部の場合

      AWS_ACOUNT_ID=77777777777 && REGION=us-east-1 && AWS_PROFILE=hageconf && CONTAINER_NAME=aho_baka_manuke && KIND=fastapi && IMAGE_MAME=$CONTAINER_NAME'-'$KIND
aws ecr get-login-password --region $REGION --profile $AWS_PROFILE | docker login --username AWS --password-stdin $AWS_ACOUNT_ID.dkr.ecr.us-east-1.amazonaws.com
      実行結果 
      Login Succeeded

  • Dockerコンテナをビルドし、コンテナイメージにタグを付与

    cd ~/aws/mongodb/ && docker ps -a && docker images && ./remove_container.sh $CONTAINER_NAME
docker build -t $IMAGE_MAME -f Dockerfile_$CONTAINER_NAME .
docker tag $IMAGE_MAME:latest $AWS_ACOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$IMAGE_MAME:latest
docker ps -a && docker images

  • FastAPIサービスのDockerコンテナをポート80でローカル端末でテスト実行

    docker run -d -p 80:80 --name $IMAGE_MAME $IMAGE_MAME

  • AWS ECRリポジトリを作成し、タグ付けしたコンテナイメージをプッシュ

    aws ecr create-repository --repository-name $IMAGE_MAME
docker push $AWS_ACOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$IMAGE_MAME':latest'
    実行結果 
    {
  "repository": {
      "repositoryArn": "arn:aws:ecr:us-east-1:77777777777:repository/aho_baka_manuke-fastapi",
      "registryId": "77777777777",
      "repositoryName": "aho_baka_manuke-fastapi",
      "repositoryUri": "77777777777.dkr.ecr.us-east-1.amazonaws.com/aho_baka_manuke-fastapi",
      "createdAt": "2024-10-06T16:47:12.317000+09:00",
      "imageTagMutability": "MUTABLE",
      "imageScanningConfiguration": {
          "scanOnPush": false
      },
      "encryptionConfiguration": {
          "encryptionType": "AES256"
      }
  }
}

    The push refers to repository [77777777777.dkr.ecr.us-east-1.amazonaws.com/aho_baka_manuke-fastapi]
f4e81d585f22: Pushed 
e46ea3317667: Pushed 
0326df620ed2: Pushed 
6ddff48d9c6e: Pushed 
5f70bf18a086: Pushed 
6d835c0b5006: Pushed 
9ce5156d8e4e: Pushed 
7cda1b8c5bbf: Pushed 
cf0e723ea047: Pushed 
0a9ccad5eb69: Pushed 
30bc2dab5899: Pushed 
20d5b086fe88: Pushed 
ba8e44410c8a: Pushed 
8628259c120e: Pushed 
e8b56a267ef1: Pushed 
8325e8413ce3: Pushed 
228e477f5903: Pushed 
dd3bb83a5529: Pushed 
5dd1f684416f: Pushed 
3a8081ce85fa: Pushed 
045d8b74bf0d: Pushed 
25879f85bbb0: Pushed 
6abe10f2f601: Pushed 
latest: digest: sha256:h9348ry83f8hwe8fh348hf84hf98h34hf3849h84389 size: 5332

  • ECSでFargateを使用する【クラスター】を作成

    aws ecs create-cluster --cluster-name $CONTAINER_NAME-cluster --region $REGION
    実行結果 
    {
  "repository": {
      "repositoryArn": "arn:aws:ecr:us-east-1:77777777777:repository/aho_baka_manuke-fastapi",
      "registryId": "77777777777",
      "repositoryName": "aho_baka_manuke-fastapi",
      "repositoryUri": "77777777777.dkr.ecr.us-east-1.amazonaws.com/aho_baka_manuke-fastapi",
      "createdAt": "2024-10-06T16:47:12.317000+09:00",
      "imageTagMutability": "MUTABLE",
      "imageScanningConfiguration": {
          "scanOnPush": false
      },
      "encryptionConfiguration": {
          "encryptionType": "AES256"
      }
  }
}

  • ECSでFargateを使用する【タスク定義】を作成

    aws ecs register-task-definition \
  --family $CONTAINER_NAME-definition \
  --network-mode awsvpc \
  --requires-compatibilities FARGATE \
  --cpu "256" \
  --memory "512" \
  --execution-role-arn arn:aws:iam::$AWS_ACOUNT_ID':role/ecsTaskExecutionRole' \
  --container-definitions '[
    {
      "name": "'$CONTAINER_NAME'-container",
      "image": "'$AWS_ACOUNT_ID'.dkr.ecr.'$REGION'.amazonaws.com/'$IMAGE_MAME':latest",
      "essential": true,
      "portMappings": [
        {
          "containerPort": 80,
          "hostPort": 80
        }
      ]
    }
  ]' \
  --region $REGION
    実行結果 
    {
  "name": "aho_baka_manuke-container",
  "image": "77777777777.dkr.ecr.us-east-1.amazonaws.com/aho_baka_manuke-fastapi:latest",
          "environment": [],
          "mountPoints": [],
          "volumesFrom": [],
          "systemControls": []
      }
  ],
  "family": "aho_baka_manuke-definition",
  "executionRoleArn": "arn:aws:iam::77777777777:role/ecsTaskExecutionRole",
  "networkMode": "awsvpc",
  "revision": 1,
  "volumes": [],
  "status": "ACTIVE",
  "requiresAttributes": [
      {
          "name": "com.amazonaws.ecs.capability.ecr-auth"
      },
      {
          "name": "ecs.capability.execution-role-ecr-pull"
      },
      {
          "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
      },
      {
          "name": "ecs.capability.task-eni"
      }
  ],
  "placementConstraints": [],
  "compatibilities": [
      "EC2",
      "FARGATE"
  ],
  "requiresCompatibilities": [
      "FARGATE"
  ],
  "cpu": "256",
  "memory": "512",
  "registeredAt": "2024-10-06T17:42:50.486000+09:00",
  "registeredBy": "arn:aws:iam::77777777777:user/AH0_Nukesaku"
}

  • ECS上に【サービス】を新規作成

リージョン クラスター名 タスク定義ファミリー サービス名
$REGION $CONTAINER_NAME-cluster $CONTAINER_NAME-definition $CONTAINER_NAME-service
タスク実行ロール セキュリティグループID プロトコル ポート範囲/ソース
arn:aws:iam::$AWS_ACOUNT_ID':role/ecsTaskExecutionRole' sg-djdijwidjweijdiewjdijew HTTP / TCP 80/0.0.0.0/0
サブネット1 サブネット2 サブネット3 パブリックIPアドレス設定
subnet-dsjfiajfijadsifjdijfi", "subnet-grergregerregerg" "subnet-ewjifjwiefjewijfiewjifjew","subnet-ewjifjwiefjewijfiewjifjew" "subnet-cdcdcdcdcdcd", "subnet-cdcdcdcdcdcd 有効

  • コマンド

    aws ecs create-service \
  --cluster $CONTAINER_NAME-cluster \
  --service-name $CONTAINER_NAME-service \
  --task-definition $CONTAINER_NAME-definition \
  --desired-count 1 \
  --launch-type FARGATE \
  --network-configuration '{
    "awsvpcConfiguration": {
      "subnets": ["subnet-grergregerregerg", "subnet-ewjifjwiefjewijfiewjifjew", "subnet-ewjifjwiefjewijfiewjifjew", "subnet-cdcdcdcdcdcd", "subnet-fidfiewifeiwjfiewjfiewji", "subnet-cdcdcdcdcdcd"],
      "securityGroups": ["sg-djdijwidjweijdiewjdijew"],
      "assignPublicIp": "ENABLED"
    }
  }' \
  --region $REGION
    実行結果 
    {
  "service": {
      "serviceArn": "arn:aws:ecs:us-east-1:77777777777:service/aho_baka_manuke-cluster/aho_baka_manuke-service",
      "serviceName": "aho_baka_manuke-service",
      "clusterArn": "arn:aws:ecs:us-east-1:77777777777:cluster/aho_baka_manuke-cluster",
      "loadBalancers": [],
      "serviceRegistries": [],
      "status": "ACTIVE",
      "desiredCount": 1,
      "runningCount": 0,
      "pendingCount": 0,
      "launchType": "FARGATE",
      "platformVersion": "LATEST",
      "platformFamily": "Linux",
      "taskDefinition": "arn:aws:ecs:us-east-1:77777777777:task-definition/aho_baka_manuke-definition:1",
      "deploymentConfiguration": {
          "deploymentCircuitBreaker": {
              "enable": false,
              "rollback": false
          },
          "maximumPercent": 200,
          "minimumHealthyPercent": 100
      },
      "deployments": [
          {
              "id": "ecs-svc/4615426311835997782",
              "status": "PRIMARY",
              "taskDefinition": "arn:aws:ecs:us-east-1:77777777777:task-definition/aho_baka_manuke-definition:1",
              "desiredCount": 0,
      :...skipping...
      {
          "service": {
              "serviceArn": "arn:aws:ecs:us-east-1:77777777777:service/aho_baka_manuke-cluster/aho_baka_manuke-service",
              "serviceName": "aho_baka_manuke-service",
              "clusterArn": "arn:aws:ecs:us-east-1:77777777777:cluster/aho_baka_manuke-cluster",
              "loadBalancers": [],
              "serviceRegistries": [],
              "status": "ACTIVE",
              "desiredCount": 1,
              "runningCount": 0,
              "pendingCount": 0,
              "launchType": "FARGATE",
              "platformVersion": "LATEST",
              "platformFamily": "Linux",
              "taskDefinition": "arn:aws:ecs:us-east-1:77777777777:task-definition/aho_baka_manuke-definition:1",
              "deploymentConfiguration": {
                  "deploymentCircuitBreaker": {
                      "enable": false,
                      "rollback": false
                  },
                  "maximumPercent": 200,
                  "minimumHealthyPercent": 100
              },
              "deployments": [
                  {
                      "id": "ecs-svc/4615426311835997782",
                      "status": "PRIMARY",
                      "taskDefinition": "arn:aws:ecs:us-east-1:77777777777:task-definition/aho_baka_manuke-definition:1",
                      "desiredCount": 0,
                      "pendingCount": 0,
                      "runningCount": 0,
                      "failedTasks": 0,
                      "createdAt": "2024-10-06T20:43:04.919000+09:00",
                      "updatedAt": "2024-10-06T20:43:04.919000+09:00",
                      "launchType": "FARGATE",
                      "platformVersion": "1.4.0",
                      "platformFamily": "Linux",
                      "networkConfiguration": {
                          "awsvpcConfiguration": {
                              "subnets": [
                                  "subnet-dsjfiajfijadsifjdijfi",
                                  "subnet-iejfijifejiewjfijewifjwi",
                                  "subnet-ewjifjwiefjewijfiewjifjew",
                                  "subnet-sdfjdijfidjiffwe",
                                  "subnet-fidfiewifeiwjfiewjfiewji",
                                  "subnet-34ht8023h08gh3hguer"
                              ],
                              "securityGroups": [
                                  "sg-djdijwidjweijdiewjdijew"
                              ],
                              "assignPublicIp": "ENABLED"
                          }
                      },
                      "rolloutState": "IN_PROGRESS",
                      "rolloutStateReason": "ECS deployment ecs-svc/4615426311835997782 in progress."
                  }
              ],
              "roleArn": "arn:aws:iam::77777777777:role/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS",
              "events": [],
              "createdAt": "2024-10-06T20:43:04.919000+09:00",
              "placementConstraints": [],
              "placementStrategy": [],
              "networkConfiguration": {
                  "awsvpcConfiguration": {
                      "subnets": [
                          "subnet-dsjfiajfijadsifjdijfi",
                          "subnet-iejfijifejiewjfijewifjwi",
                          "subnet-ewjifjwiefjewijfiewjifjew",
                          "subnet-sdfjdijfidjiffwe",
                          "subnet-fidfiewifeiwjfiewjfiewji",
                          "subnet-34ht8023h08gh3hguer"
                      ],
                      "securityGroups": [
                          "sg-djdijwidjweijdiewjdijew"
                      ],
                      "assignPublicIp": "ENABLED"
                  }
              },
              "schedulingStrategy": "REPLICA",
              "deploymentController": {
                  "type": "ECS"
              },
              "createdBy": "arn:aws:iam::77777777777:user/AH0_Nukesaku",
              "enableECSManagedTags": false,
              "propagateTags": "NONE",
              "enableExecuteCommand": false
          }
1
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?