LoginSignup
6

More than 5 years have passed since last update.

【AWS Batch】1コマンドでdockerを定期実行する

Last updated at Posted at 2017-10-05

CloudWatch EventsでLambdaを定期実行し、LambdaからAWS Batch上でdockerが動くようにする。

scheduled-docker-batch.yaml
Parameters:
  SubnetIds:
    Description: Subnets For ComputeEnvironment
    Type: List<AWS::EC2::Subnet::Id>
  SecurityGroupIds:
    Description: SecurityGroups For ComputeEnvironment
    Type: List<AWS::EC2::SecurityGroup::Id>
Resources:
  BatchComputeSpotFleetRole:
    Type: "AWS::IAM::Role"
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: spotfleet.amazonaws.com
            Action: "sts:AssumeRole"
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonEC2SpotFleetRole
  BatchComputeEnvironmentRole:
    Type: "AWS::IAM::Role"
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: batch.amazonaws.com
            Action: "sts:AssumeRole"
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSBatchServiceRole
  BatchComputeInstanceRole:
    Type: "AWS::IAM::Role"
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: "sts:AssumeRole"
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role
  BatchComputeInstanceProfile:
    Type: "AWS::IAM::InstanceProfile"
    Properties:
      Path: "/"
      Roles:
        - Ref: BatchComputeInstanceRole
  BatchJobQueue:
    Type: "AWS::Batch::JobQueue"
    Properties:
      JobQueueName: scheduled-job-queue
      Priority: 1
      ComputeEnvironmentOrder:
        - Order: 1
          ComputeEnvironment: !Ref BatchComputeEnvironment
  BatchComputeEnvironment:
    Type: AWS::Batch::ComputeEnvironment
    Properties:
      Type: MANAGED
      ServiceRole: !Ref BatchComputeEnvironmentRole
      ComputeEnvironmentName: scheduled-batch-compute-environment
      ComputeResources:
        Type: SPOT
        SecurityGroupIds: !Ref SecurityGroupIds
        Subnets: !Ref SubnetIds
        MaxvCpus: 128
        MinvCpus: 0
        DesiredvCpus: 0
        InstanceRole: !Ref BatchComputeInstanceProfile
        InstanceTypes:
          - optimal
        SpotIamFleetRole: !Ref BatchComputeSpotFleetRole
        BidPercentage: 50
      State: ENABLED
  JobDefinition:
    Type: AWS::Batch::JobDefinition
    Properties:
      Type: container
      JobDefinitionName: schedule-batch-example
      ContainerProperties:
        Command:
          - /hello
        Memory: 256
        Vcpus: 1
        Image: hello-world
      RetryStrategy:
        Attempts: 1
  LambdaExecutionRole:
    Type: "AWS::IAM::Role"
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: "sts:AssumeRole"
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Policies:
        - PolicyName: CustomLocalPolicy
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - batch:SubmitJob
                Resource:
                  - "*"
  JobSubmitFunction:
    Type: "AWS::Lambda::Function"
    Properties:
      # Handler: "index.handler"
      Handler: "index.handler"
      Role: !GetAtt [ LambdaExecutionRole, Arn ]
      Code:
        ZipFile: |
          import os
          import boto3
          def handler(event, context):
              params = {
                  'jobName': 'schedule-sample-job',
                  'jobQueue': os.environ.get('JOB_QUEUE'),
                  'jobDefinition': os.environ.get('JOB_DEFINITION')
              }
              print(params)
              client = boto3.client('batch')
              res = client.submit_job(**params)
              print(res)
      Runtime: "python3.6"
      Environment:
        Variables:
          JOB_QUEUE: !Ref BatchJobQueue
          JOB_DEFINITION: !Ref JobDefinition
  JobSubmitEventRule:
    Type: AWS::Events::Rule
    Properties:
      ScheduleExpression: rate(5 minutes)
      Targets:
        - Id: JobSubmitScheduler
          Arn:
            Fn::GetAtt:
              - JobSubmitFunction
              - Arn
  InvokeLambdaPermission:
    Type: AWS::Lambda::Permission
    Properties:
      FunctionName:
        Fn::GetAtt:
          - JobSubmitFunction
          - Arn
      Action: lambda:InvokeFunction
      Principal: events.amazonaws.com
      SourceArn:
        Fn::GetAtt:
          - JobSubmitEventRule
          - Arn

実行方法

$ aws cloudformation create-stack --stack-name schedule-docker-batch \
  --template-body file://./schedule-docker-batch.yaml \
  --capabilities CAPABILITY_IAM \
  --parameters ParameterKey=SubnetIds,ParameterValue="subnet-xxxxxxx" ParameterKey=SubnetIds,ParameterValue="subnet-yyyyyyy" ParameterKey=SecurityGroupIds,ParameterValue="sg-zzzzzzzz"

よしなにJobDefinitionを編集して好きなようにdockerを動かす。

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
6